Headquarters: Remote

URL: https://www.toptal.com/

About the Role

We're looking for a Senior Data/ML Engineer to refactor, operationalize, and improve an existing time series forecasting platform that's already live and driving real business value. This is not a greenfield build — the focus is modernizing a Databricks-based forecasting system that's been maintained primarily by a single developer for years. You'll reduce technical debt, strengthen testing and observability, and raise the engineering bar on a system the business already depends on. If you'd rather bring discipline and maturity to an existing production system than start from a blank slate, this is built for that.

What You'll Do

•
Review the current forecasting platform architecture and identify areas for improvement

•
Refactor existing Databricks, Python, and PySpark implementations

•
Move business logic out of Databricks notebooks and into reusable Python modules or packages

•
Improve separation of concerns between orchestration and core business logic

•
Establish stronger engineering standards and help define what "good" looks like for the platform

•
Implement or improve automated testing practices and validation mechanisms for forecasting workflows

•
Build or improve monitoring and observability, increasing visibility into how predictions are generated

•
Help monitor model behavior and operational health over time

•
Improve reliability of scheduled training workflows, reducing manual intervention on failure

•
Improve failure handling, retries, and overall workflow resilience

•
Maintain and extend existing forecasting capabilities as needed

What You Bring

•
Strong professional experience with Databricks, including workspaces, notebooks, scheduled workflows, and CI/CD processes

•
Strong Python engineering experience, including designing reusable modules or packages

•
Strong PySpark experience with production data pipelines or distributed data processing

•
Experience refactoring production code and improving maintainability

•
Familiarity with time series forecasting concepts and workflows

•
Ability to understand and work effectively within an existing, unfamiliar codebase

•
Experience improving software quality, testing strategy, and engineering standards

•
Experience implementing automated testing practices

•
Experience improving monitoring, observability, or operational visibility for production systems

•
Strong judgment around technical debt, refactoring priorities, and maintainable architecture

•
Ability to work with existing systems rather than only building from scratch

Why This Role

•
Real production impact: Improve a system the business already relies on, not a proof-of-concept

•
Engineering maturity focus: Bring testing, observability, and maintainability to a platform that's outgrown its current state

•
Meaningful ownership: Help define engineering standards for the forecasting platform going forward

•
Flexible location: Preference for Toronto or St. Louis, but open to remote consultants globally with North American working-hours overlap

How to Apply

Ready to bring engineering rigor to a production forecasting platform? Apply through Toptal here: https://www.toptal.com/talent/apply

To apply: https://weworkremotely.com/remote-jobs/toptal-senior-data-ml-engineer-databricks-forecasting-platform-remote