Over the past two years, our team at Tobiko have worked on creating, refining, and expanding SQLMesh, our open-source data transformation product. As the SQLMesh community grew, so did the demand for a solution that extends the SQLMesh experience and can be easily deployed across teams. 

To meet that need, we’re excited to announce the general availability of Tobiko Cloud—our hosted, enterprise-ready version of SQLMesh. Tobiko Cloud brings enterprise-grade security, simplifies deployment, and amplifies SQLMesh’s time- and resource-saving capabilities to help teams operate at scale with increasingly complex and interdependent data workloads.

Tobiko Cloud: elevating SQLMesh with new features and benefits

Granular, actionable observability and insights 

Tobiko Cloud includes a suite of three observability features—runtime alerts, a unified debugger, and warehouse cost tracking. 

Alerts are simple to configure. They flag failed runs and trigger when custom thresholds, such as metrics exceeding certain values, are reached. Notifications are sent through Slack, PagerDuty, or email, with Webhook and Datadog support to follow. Deep links into the Tobiko Cloud UI allow users to quickly identify root causes and address disruptions. 

Fixing pipeline issues is made easier with our debugger, which provides an all-in-one view that consolidates critical context for error resolution. It surfaces recent upstream and downstream activity, code definitions and schema at the time of failure, and key details, including pull requests, logs, and query history, among other information that guides developers in tracing, containing, and fixing data incidents.

Tobiko Cloud also offers a cost tracker. Cutting transformation costs is a core mission of SQLMesh, and we are further productizing and reinforcing that benefit in Tobiko Cloud with a tool dedicated to keeping compute transparent and manageable. Warehouse costs are often difficult to untangle from other expenses. Our cost tracker isolates and brings them into focus, breaking down spend on BigQuery, Snowflake, and other engines, pinpointing the most costly models, and giving teams a clear line-of-sight into usage patterns and opportunities to optimize.

Intuitive, efficient built-in orchestration

We created Tobiko Cloud on the basis of SQLMesh’s state-aware architecture, which imbues both products with a comprehensive understanding of cron scheduling and run history. This foundation made it possible for us to launch Tobiko Cloud with a native scheduler, which automatically groups jobs and sets them to run at defined times and intervals—without the overhead of manual setup or CI/CD configuration.

We also designed Tobiko Cloud’s scheduler with common pipeline bottlenecks in mind, introducing additional features—concurrency and pause model runs—to minimize idle downtime. 

With concurrency, Tobiko Cloud’s scheduler will not hold up data workflows in queues. Any number of models can run in parallel as soon as they are ready—execution cannot be blocked by their long-running counterparts. With fewer delays and no need to force-fit job schedules based on expected runtimes, data teams can better meet SLAs and redirect efforts to higher-value work.

Pause model runs add another layer of control and flexibility. Users can temporarily halt production runs or individual models to perform maintenance, investigate issues, or apply updates—all without needing to wait for jobs to complete. When transformations don’t go as planned, Tobiko Cloud primes teams for more agile iterations and real-time troubleshooting.

Best-in-class security and data governance

Enterprises are facing ever stricter security requirements—and supporting them means making it easy for their data teams to meet tightening standards for data governance. Tobiko Cloud delivers a pair of features designed to build guardrails into the development process: isolated Python environments and the option for hybrid deployment.

Isolated Python environments ensure that each run installs only the dependencies it needs, keeping environments clean and discrete. Teams can run tests with Python code that will not interfere with, or accidentally contaminate one another, making experiments safer and more predictable.

And for organizations with the most rigorous data policies, hybrid deployment provides a secure alternative to fully cloud-hosted solutions, such as that offered by dbt Labs. Tobiko Cloud can run in hybrid mode, maintaining all data and warehouse operations inside users’ own infrastructure. We will require only limited access to your SQL metadata, without compromising on the caliber of your transformation experience.

Advanced impact and change analysis

To round out the arsenal of tools in our first enterprise-ready release of Tobiko Cloud, we are launching two additions that deepen and enrich SQLMesh’s ability to detect and analyze changes in data.

SQLMesh distinguishes breaking from non-breaking changes when new columns are added. Advanced change categorization in Tobiko Cloud takes that a substantial leap further, evaluating whether modified columns are used downstream. Advanced change categorization more precisely classifies changes as non-breaking, and reduces the number of unnecessary backfills.

Tobiko Cloud also supports cross-database diffing, building on SQLMesh’s existing mechanisms for comparing dataset versions to identify records added, removed, or updated over time—within the same warehouse. Tobiko Cloud elevates this with a hashing algorithm, bypassing expensive full joins to make the same comparison both at a lower cost and across multiple warehouses, making it significantly more efficient to migrate among vendors, detect data drift, and maintain consistency across environments.

In conclusion

Our team hails from some of the world’s largest companies, where we recognized that the transformation tools widely adopted today make it impossible for teams with big data projects and volumes to scale. 

We founded Tobiko because we felt that should not have to be the case. With our formal launch of Tobiko Cloud today, we can now give data teams the automated scheduling, security safeguards, and detailed, decision-ready insights they need to perform and collaborate at peak productivity. We are ready to support enterprises of any size - talk to us to learn how we can help you implement Tobiko Cloud and reinvent your team’s data engineering experience.