28 Aug
|
J M Group
|
Mississauga
28 Aug
J M Group
Mississauga
We are looking for a GCP Senior Data Engineer with strong experience in GCP, Big
Query, Dataflow, Dataproc, Pub/Sub, Spark, Python, SQL, Terraform & CI/CD. **Required Skills** - Technical: Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam). - GCP Expertise: Hands-on with Big
Query, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform. - Dev
Ops: Familiarity with CI/CD, Git, and infrastructure-as-code. **Nice to Have** - GCP Certifications(e.g., Skilled Data Engineer). - Experience with real-time processing, Vertex AI, or multi-cloud environments. **Key Responsibilities** - Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions. - Data Infrastructure: Architect and manage Big
Query, Cloud Storage,
and Data Fusion for efficient data ingestion, transformation, and storage. - Team Leadership: Mentor junior engineers, review code, and establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging). - Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost managemen (e.g., slot reservations, partitioning). - Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases. - Data Quality & Governance: Implement data validation, lineage, and metadata management(e.g., Data Catalog, dbt).
📌 GCP Senior Data Engineer (Mississauga)
🏢 J M Group
📍 Mississauga