Become an integral part of a global IT consulting company as a Data Engineer in Brampton, Ontario. Specialize in building data pipelines and maintaining effective data governance practices.
In this onsite role, you’ll leverage 5-7 years of experience in distributed data systems, focusing on ETL flows and real-time processing. Your tasks will involve managing hybrid search systems utilizing technologies such as pgvector and Redis, ensuring compliance in data handling procedures.
Key Responsibilities:
• Create ETL processes for both structured and unstructured data
• Manage Azure AI Search and vector databases
• Implement data governance and privacy measures
• Develop high-performance event-driven architectures
• Ensure normalization and deduplication throughout data pipelines
Requirements:
• Proven deep expertise in SQL and distributed systems
• Experience in high-throughput database infrastructure
• Knowledge of Google's GECX is beneficial
• Familiarity with data chunking strategies
• Competence in using Spark or Kafka for real-time data flow
Apply your skills to influence data strategy within this cutting-edge IT company.
#J-18808-Ljbffr