Lakehouse Development
- Design and maintain Fabric Lakehouse environments.
- Develop and optimize Delta Tables for high-performance analytics workloads.
- Implement Medallion Architecture (Bronze, Silver, Gold layers) to support data quality and governance requirements.
- Create curated datasets for reporting, analytics, and AI/ML use cases.
Data Transformation & Processing
- Develop data transformation logic using PySpark, Spark SQL, and notebooks.
- Create reusable frameworks and data engineering best practices.
- Perform data cleansing, enrichment, and standardization activities.
- Data Modeling & Analytics Support
- Design dimensional and analytical data models.
- Collaborate with Business Intelligence teams to support Power BI semantic models and reporting requirements.
- Ensure data consistency, quality, and accuracy across data platforms.
Key Responsibilities
Data Engineering & Integration
- Design, develop, and maintain scalable data ingestion and transformation pipelines using Microsoft Fabric.
- Build and manage ETL/ELT workflows using Fabric Data Factory and Data Pipelines.
- Integrate data from multiple sources including databases, APIs, flat files, cloud storage, and SaaS applications.
- Implement incremental data loads and Change Data Capture (CDC) strategies.
Graduate in Computer Science, Data Science, or related field. 4-6 years of experience in data engineering or related field.
Required Skills
Technical Skills
- Microsoft Fabric
- OneLake
- Fabric Lakehouse
- Fabric Data Factory
- Data Pipelines
- Notebook Development
- PySpark
- Spark SQL
- SQL
- Delta Lake
- Medallion Architecture
- Power BI Integration
- Git Version Control
Cloud & Data Platform Experience
- Azure Data Services
- Azure Data Lake Storage (ADLS)
- Azure Synapse Analytics (preferred)
- Azure DevOps
Data Engineering Concepts
- ETL/ELT Development
- Data Modeling
- Data Warehousing
- Data Governance
- Data Quality Frameworks
- Performance Tuning
Preferred Qualifications
- Experience in large-scale enterprise data transformation programs.
- Knowledge of Data Mesh and Modern Data Platform concepts.
- Exposure to AI/ML data preparation processes.
- Microsoft Fabric Certification (Preferred).
Experience working in Banking, Insurance, Healthcare, or Retail domains.