Role responsibilities
Design and build scalable, fault-tolerant data processing pipelines and architectures to handle petabyte-scale data for Bing and MSN. Collaborate with cross-functional partners to develop data assets, anomaly detection, and real-time insights while ensuring data compliance.
Requirements
Requires a Bachelor's degree in Computer Science and at least 4 years of technical engineering experience in languages like C#, Java, or Python. Preferred candidates have 6+ years of experience with big data technologies such as Spark or Flink and experience in cloud environments like Azure.
Key skills
Spark, Flink, Kafka, EventHub, Azure, HDFS, Azure Blob Storage, C#, Java, Python, C++, Data Pipeline Design, Big Data Processing, AI Integration, Data Governance, Backend Service Development
Keywords
Spark, Flink, Kafka, EventHub, Azure, HDFS, Azure Blob, C#, Java, Python, C++, Big Data, Petabyte Scale, Data Pipelines, Real-time Insights, Data Governance, Data Compliance, Anomaly Detection, Backend Development, Artificial Intelligence, Software Development Lifecycle, SLA, Fault-tolerant Systems, Data Analytics, Bing, MSN, Cloud Computing, Distributed Systems, Data Validation