Position OverviewAs a Software Developer on the Fusion platform services team within Product Development and Manufacturing Solutions (PDMS), you'll be part of a team of technologists dedicated to creating cutting-edge AI and generative AI solutions that enhance developer productivity and experience. You'll work closely with AI engineers, software architects, and product engineering teams to build and rigorously evaluate intelligent agentic systems — including benchmarking AI agents against commercial solvers — and develop MCP (Model Context Protocol)-based tooling that integrates seamlessly with IDEs such as VS Code and Cursor.ResponsibilitiesDevelop and orchestrate multi-agent AI systems for automated test generation, test execution, and end-to-end development workflow optimization using frameworks like LangGraph, AutoGen, or the Anthropic Agent SDK (Claude Code)Design and implement agentic workflows that coordinate multiple AI agents to autonomously drive test automation across UI, API, integration, and system levels, from test case synthesis to result evaluation, ensuring seamless integration with existing developer tools and MCP-compatible servicesBuild evaluation frameworks and custom benchmarks for agentic systems, including comparisons of AI agents against commercial solvers, using tools like AgentBench and LangfuseEvaluate MCP server and tool performance across agentic pipelines, measuring latency,



accuracy, context fidelity, and end-to-end task completion ratesMinimum QualificationsBS/MS in Computer Science, Machine Learning, or a related applied AI fieldExpertise in Python and ML frameworks (PyTorch, Transformers, scikit-learn)Experience with Large Language Models applied to software understanding or test generationKnowledge of AI evaluation methodologies and metrics for agentic task completion and test qualityStrong foundation in statistical analysis and experimental designExperience with developer workflow and productivity measurement frameworksPreferred QualificationsBackground in software engineering or QA with close collaboration with development teamsFamiliarity with test automation frameworks (e.G., Playwright, Selenium, Pytest, Appium) and CI/CD pipelinesExperience designing benchmarks that compare AI agents against commercial or domain-specific solversHands-on experience with MCP (Model Context Protocol), building, evaluating, and optimizing MCP servers and tool integrations within agentic pipelinesExperience with agentic AI frameworks including LangGraph, AutoGen, or the Anthropic Agent SDK / Claude CodeKnowledge in vision-language models or multi-modal AI for UI and system-level understanding and evaluationExperience with Azure AI Foundry/ML or AWS cloud ML platforms #J-18808-Ljbffr

📌 Applied Ai Developer (Agent Evaluation) (Toronto)
🏢 Autodesk
📍 Toronto