Become an integral part of a global IT consulting company as a Data Engineer in Brampton, Ontario. Specialize in building data pipelines and maintaining effective data governance practices.
In this onsite role, you’ll leverage 5-7 years of experience in distributed data systems, focusing on ETL flows and real-time processing. Your tasks will involve managing hybrid search systems utilizing technologies such as pgvector and Redis, ensuring compliance in data handling procedures.
Key Responsibilities: • Create ETL processes for both structured and unstructured data • Manage Azure AI Search and vector databases • Implement data governance and privacy measures • Develop high-performance event-driven architectures • Ensure normalization and deduplication throughout data pipelines
Requirements: • Proven deep expertise in SQL and distributed systems • Experience in high-throughput database infrastructure • Knowledge of Google's GECX is beneficial • Familiarity with data chunking strategies • Competence in using Spark or Kafka for real-time data flow
Apply your skills to influence data strategy within this creative IT company. #J-18808-Ljbffr