Combine SQLMesh DataOps workflows with SYNQ’s Data Product Observability layer to deliver reliable data products
Figure 1. Announcing the SYNQ and SQLMesh integration.
We can count on the SYNQ team to move fast and add integrations for new functionality as they’re added in SQLMesh
Supporting business-critical use cases is at the heart of what we do - and it starts with building reliability into data assets from day one. At Tobiko, we have experience collaborating with SYNQ, a leading data observability platform, on supporting shared enterprise customers. We’ve seen firsthand the reliability and value these customers gain by using SQLMesh and SYNQ - together. To continue building on this success, we’re integrating with SYNQ to deliver those same great results to more users.
SYNQ’s observability platform is built with data models as first-class citizens, and with the assumption that tables will freely change in the background. This approach sets SYNQ apart from other platforms on the market, and is a great fit with the virtual data environments central to SQLMesh and our hosted offering, Tobiko Cloud. Data teams working in Tobiko’s virtual data environments can create exact replicas of production for development, and promote changes made in these environments back in production at close to no warehouse processing cost. SYNQ and Tobiko share a common philosophy of accommodating and optimizing for the dynamic changes data engineers need to make, making our solutions naturally complementary.
SQLMesh now integrates with SYNQ to support data transformations with modern data observability. We’re committed to continue supporting data and analytics engineers across platforms, and look forward to expanding this new integration together with SYNQ and our shared customers.
What is SYNQ
SYNQ is a ‘Data Product Observability Platform’ built for teams that own business-critical data. By embedding the concept of Data Products into observability, SYNQ helps teams manage data quality in the context of business-critical use cases. With models and metrics as core constructs, SYNQ enables observability workflows that go beyond traditional table-centric approaches, supported by features like anomaly monitoring, data domains, ownership, incident management, and data quality reporting.
Why SYNQ is a good fit
SYNQ enables targeted monitoring of critical data through data products, and is designed to fit how data teams already work. Here are some key initiatives we’re particularly excited to explore as part of our partnership:
💡 Use data products to manage customer data platforms’ health from the perspective of business priorities
💡 Combine SQLMesh tests and SYNQ monitors for unified issue management workflows
💡 Treat critical issues as incidents and manage them across the data stack
💡 Empower engineers to proactively address SQLMesh issues by routing them to the appropriate individuals and tracking data quality by owner
SQLMesh + SYNQ
Integrating both platforms lets you model your data using DevOps best practices, while adding an observability layer to ensure your data is reliable. SYNQ automatically ingests all SQLMesh model properties and metadata, so you can seamlessly manage data products and ownership definitions while deploying monitors through code.
Figure 1. SQLMesh + SYNQ – build and model reliable data with a unified workflow.
Core workflows
Here are a few workflows where SQLMesh and SYNQ play well together:
- Define data products– SYNQ automatically ingests data assets from SQLMesh, including SQL models, snapshots, and seeds, and lets you build data products that give you visibility into data health across your full stack.
- Enhance tests and audits with monitors– use SQLMesh model tags to define SYNQ monitor deployments and combine tests and audits with SYNQ’s advanced anomaly detection to catch both ‘Known Unknowns’ and ‘Unknown Unknowns’. Together, SQLMesh and SYNQ ensure both types of anomalies are surfaced and addressed before they escalate.
- ‘Known Unknowns’- are expected issues e.g. where you have predefined rules or thresholds
- ‘Unknown Unknowns’- are unexpected anomalies that traditional tests might not catch, such as a sudden but subtle shift in user behavior patterns or an upstream schema change causing silent data corruption.
- Activate ownership– use SQLMesh model properties, such as owner or tags, to bring ownership into SYNQ. Route test or audit issues alongside other data issues and avoid dispersed alerting workflows.
- Manage data incidents– treat test or audit failures as incidents, track downstream impacts, and manage communication from detection to resolution.
To learn more, join us for our fireside chat on January 23rd where Petr Janda (Founder and CEO of SYNQ) and Tobias (Toby) Mao (Co-Founder and CTO of Tobiko) will discuss our partnership in more detail.
Or learn more about SQLMesh here and watch SYNQ’s CEO, Petr Janda, talk about a strategic approach to testing monitoring with Data Products.