30 Jul
|
Virtusa
|
Canada
Job Description
Data Engineer and Generative AI/ML Specialist
Skill Cluster/Practice: Data Engineering / Emerging Technologies (GenAI)
We are seeking an cutting-edge and results-driven Data Engineer and GenAI/ML Specialist with expertise in large-scale data engineering, LLM-based automation, and the implementation of intelligent systems. The ideal candidate will leverage proven experience to develop robust, compliant, and scalable data solutions, contributing to both data infrastructure and advanced AI applications.
Responsibilities
Design, build, and maintain large‑scale, high‑volume real‑time data ingestion and streaming pipelines using technologies such as Apache Spark/PySpark, Python, and Kafka on cloud platforms like Databricks, GCP, and AWS.
Lead the development and deployment of LLM‑based automation, Retrieval‑Augmented Generation (RAG) pipelines, and AI agent systems (using tools like OpenAI/HuggingFace).
Apply and productionize Machine Learning and Deep Learning models, including building classification models, predictive models, time‑series forecasting,
and sentiment analysis solutions using frameworks such as PyTorch, TensorFlow, and scikit‑learn.
Mine and re‑architect data models to build and maintain data lakes and data warehouses (e.g., Snowflake), streamlining ETL/ELT workflows using tools like dbt and Apache Airflow.
Ensure data ingestion and processing workflows adhere to data quality standards, compliance, and data privacy requirements.
Optimize SQL and Spark workloads to improve query performance and reduce compute costs.
Required Technical Skills and Experience
Experience and hands‑on experience in Data Engineering, Machine Learning, or AI Solution Engineering.
Deep proficiency in Python and SQL for data processing, scripting, and optimization.
Expertise in distributed computing (e.g., Apache Spark/PySpark, Databricks), real‑time streaming technologies (e.g., Kafka), and cloud platforms (AWS, GCP, Snowflake).
J-18808-Ljbffr
📌 Data Engineer Toronto (Canada)
🏢 Virtusa
📍 Canada