Headquarters: RemoteURL: https://www.Toptal.Com/About the RoleWe're looking for a Senior Data/ML Engineer to refactor, operationalize, and improve an existing time series forecasting platform that's already live and driving real business value. This is not a greenfield build — the focus is modernizing a Databricks-based forecasting system that's been maintained primarily by a single developer for years. You'll reduce technical debt, strengthen testing and observability, and raise the engineering bar on a system the business already depends on. If you'd rather bring discipline and maturity to an existing production system than start from a blank slate, this is built for that.What You'll DoReview the current forecasting platform architecture and identify areas for improvementRefactor existing Databricks, Python, and PySpark implementationsMove business logic out of Databricks notebooks and into reusable Python modules or packagesImprove separation of concerns between orchestration and core business logicEstablish stronger engineering standards and help define what "positive" looks like for the platformImplement or improve automated testing practices and validation mechanisms for forecasting workflowsBuild or improve monitoring and observability, increasing visibility into how predictions are generatedHelp monitor model behavior and operational health over timeImprove reliability of scheduled training workflows, reducing manual intervention on failureImprove failure handling, retries,



and overall workflow resilienceMaintain and extend existing forecasting capabilities as neededWhat You BringStrong professional experience with Databricks, including workspaces, notebooks, scheduled workflows, and CI/CD processesStrong Python engineering experience, including designing reusable modules or packagesStrong PySpark experience with production data pipelines or distributed data processingExperience refactoring production code and improving maintainabilityFamiliarity with time series forecasting concepts and workflowsAbility to understand and work effectively within an existing, unfamiliar codebaseExperience improving software quality, testing strategy, and engineering standardsExperience implementing automated testing practicesExperience improving monitoring, observability, or operational visibility for production systemsStrong judgment around technical debt, refactoring priorities, and maintainable architectureAbility to work with existing systems rather than only building from scratchWhy This RoleReal production impact: Improve a system the business already relies on, not a proof-of-conceptEngineering maturity focus: Bring testing, observability, and maintainability to a platform that’s outgrown its current stateMeaningful ownership: Help define engineering standards for the forecasting platform going forwardFlexible location: Preference for Toronto or St. Louis, but open to remote consultants globally with North American working-hours overlap#J-18808-Ljbffr

📌 Senior Data/Ml Engineer — Databricks Forecasting Platform | Remote - Remote (Toronto)
🏢 Databricks
📍 Toronto