Making SQLMesh Faster
Reflecting on SQLMesh's evolution over the past two years and highlighting the significant performance breakthroughs we've recently achieved.
Tobiko Articles
-
-
Project Variables in SQLMesh: Non-obvious Challenges
SQLMesh now supports project-wide variables to help you parameterize your projects. This feature has some interesting implementation challenges, which we dive into in this post.
-
SQLGlot Jumps on the Rust Bandwagon
Introducing the Rust implementation of the SQLGlot tokenizer, which significantly improves parsing performance.
-
Introducing WAP Pattern Support with Apache Iceberg
SQLMesh now supports the Write-Audit-Publish pattern when model tables are stored in the Apache Iceberg format.
-
Why Data Teams Are Adopting Declarative Pipelines
Explaining the surge in popularity of declarative data transformation while drawing some parallels with DevOps.
-
Data Completeness: To Stream or Not to Stream
The tremendous push towards streaming pipelines lately has led some to believe that the days of batch processing are either over or will soon be coming to an end. But is this actually true? In this post, I explore the reasons for keeping some of your batch pipelines.
-
Evolving Large Datasets with SQLMesh
Learn about what SQLMesh has to offer when it comes to evolving really large datasets.
-
Traps and Pitfalls of Using SQL with Jinja
The practice of augmenting SQL code with Jinja templates has gained significant traction and is consistently growing in popularity. However, is this truly the optimal approach? In this article, we examine the common challenges that arise when combining SQL and Jinja, while also suggesting a more effective method for metaprogramming in SQL.
-
Simplicity or Efficiency: How dbt Makes You Choose
Comparing development experience and runtime efficiency of dbt and SQLMesh.
-
Semantic Understanding of SQL
Exploring what semantic understanding is and how it helps improve user experience when using SQL.