Python and PySpark expertise define this Data Engineering role, focused on AI applications, cloud platforms, and production-grade software engineering.

Build and operationalize data and AI solutions using Python, PySpark, SQL, REST APIs, and FastAPI. The work spans ETL/ELT, batch and streaming pipelines, data quality, metadata, lineage, observability, and DataOps practices. You will also support MLOps and LLMOps capabilities across delivery and production.

Key Responsibilities:

- Profile unfamiliar client data using Python, pandas, PySpark, and SQL
- Build thin solutions with real client data and APIs
- Implement automated tests, security controls, and error handling
- Create evaluation datasets and measure ML or LLM quality
- Measure adoption and business impact using agreed KPIs

Requirements:

- Experience in Data Engineering and production-grade software engineering
- Hands-on Python, PySpark, SQL, REST APIs, and FastAPI expertise
- Experience with ETL/ELT, streaming pipelines, and data quality
- Knowledge of Git, Docker, Kubernetes, Airflow, and MLflow
- Bachelor's degree in a related discipline

Apply Python, PySpark, data discovery, and production engineering skills across measurable AI and data solutions.

📌 Senior Python PySpark Data Engineer (Toronto)
🏢 Atlantis IT Group
📍 Toronto