Elevate an existing time series forecasting platform as a Senior Data/ML Engineer at Toptal, working remotely. Focus on refactoring and operationalizing to drive business value efficiently.

In this senior role, you will modernize a production Databricks-based forecasting system, which is already in use but requires updates and enhancements. Your expertise in Python, Databricks, and PySpark will help reduce technical debt, improve testing, and enhance observability, ensuring the system meets high engineering standards. You'll contribute to the ongoing success and reliability of the platform.

Key Responsibilities:
• Review current architecture for improvement opportunities
• Refactor Databricks and Python implementations efficiently
• Develop reusable Python modules from existing notebooks




• Enhance separation of concerns in the system
• Implement better automated testing and validation practices

Requirements:
• Extensive experience with Databricks and CI/CD processes
• Robust Python skills for designing reusable modules
• Proficiency in PySpark for large-scale data pipelines
• Experience with refactoring production code for maintainability
• Familiarity with time series forecasting and monitoring

Contribute to a vital forecasting system at Toptal, enhancing its testing, observability, and engineering standards.
#J-18808-Ljbffr

📌 Senior Data Engineer Remote Position (Ontario)
🏢 Neura Market
📍 Ontario