Job Summary
We are seeking a skilled GCP BigQuery Data Engineer to design, develop, and optimize scalable data solutions on Google Cloud Platform (GCP). The ideal candidate will have strong expertise in BigQuery, data warehousing, ETL/ELT development, and cloud-based analytics platforms. The role involves building robust data pipelines, ensuring data quality, and enabling business insights through efficient data management practices.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using GCP services.
- Build and optimize data warehouses and data marts using BigQuery.
- Develop ETL/ELT workflows for ingesting data from multiple sources.
- Implement data modeling techniques to support reporting and analytics requirements.
- Optimize BigQuery performance, query costs, and storage utilization.
- Work with structured, semi-structured, and streaming data sources.
- Develop and maintain workflows using Cloud Composer (Airflow), Dataflow, and Pub/Sub.
- Ensure data quality, consistency, security, and governance standards.
- Partner with business stakeholders, data analysts, and data scientists to understand data requirements.
- Monitor, troubleshoot, and enhance data platform performance.
- Implement CI/CD processes and automate deployment of data solutions.
- Follow data security and compliance best practices within GCP environments.
Required Skills
GCP Services
- BigQuery
- Cloud Storage (GCS)
- Dataflow
- Pub/Sub
- Cloud Composer (Airflow)
- Cloud Run / Cloud Functions
- Dataproc (preferred)
Data Engineering
- Data Warehousing Concepts
- ETL/ELT Design & Development
- Data Modeling (Star Schema, Snowflake Schema)
- Data Migration and Data Integration
- Batch and Real-time Processing
Programming
- SQL (Advanced)
- Python
- Shell Scripting
Database Technologies
- BigQuery
- SQL Databases (MySQL, PostgreSQL, SQL Server)
- NoSQL Databases (Preferred)
DevOps & Version Control
- Git
- CI/CD Pipelines
- Terraform (Preferred)
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
- 4+ years of experience in Data Engineering.
- Minimum 2+ years of hands-on experience with GCP and BigQuery.
- Strong expertise in SQL query optimization and performance tuning.
- Experience building large-scale data processing solutions.
- Knowledge of cloud security, governance, and data management best practices.
Preferred Qualifications
- Google Cloud Professional Data Engineer Certification.
- Experience with dbt.
- Experience with Looker, Tableau, or Power BI.
- Exposure to Machine Learning data pipelines.
- Experience in Agile/Scrum development methodologies.
Key Competencies
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management.
- Ability to work independently in a fast-paced environment.
- Attention to detail and commitment to data quality.
- Collaborative team player with a continuous learning mindset.