Job Summary:
We are seeking a highly skilled Data Engineer with extensive experience in agentic frameworks and generative AI. The ideal candidate will be responsible for designing and implementing data solutions that leverage LangChain and LangGraph, while integrating with Azure AI Foundry. The role requires a strong understanding of metric-based evaluations and the ability to work with LLMs as judges in data processing workflows.
Responsibilities
Design and implement data pipelines using agentic design patterns.
Utilize LangChain and LangGraph for data processing and analysis.
Integrate and optimize data solutions with Azure AI Foundry.
Conduct metric-based evaluations to assess data quality and performance.
Implement multithreading and asynchronous programming using Python and AsyncIO.
Collaborate with DevOps teams to ensure seamless CI/CD processes.
Manage infrastructure using Docker and Terraform.
Mandatory Skills
Proven experience with generative AI technologies.
Solid proficiency in Python, particularly in AsyncIO and multithreading.
Experience with LangChain and LangGraph frameworks.
Familiarity with Azure AI Foundry and its integration capabilities.
Solid understanding of metric-based evaluation methodologies.
Preferred Skills
Experience with DevOps practices and tools.
Knowledge of Docker and Terraform for infrastructure management.
Familiarity with CI/CD pipelines and best practices.
Qualifications
Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
10-15 years of relevant experience in data engineering or a similar role.
Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.