Tobiko Data Articles

1. SQLGlot Jumps on the Rust Bandwagon

   Introducing the Rust implementation of the SQLGlot tokenizer, which significantly improves parsing performance.

2. Introducing WAP Pattern Support with Apache Iceberg

   SQLMesh now supports the Write-Audit-Publish pattern when model tables are stored in the Apache Iceberg format.

3. Why Data Teams Are Adopting Declarative Pipelines

   Explaining the surge in popularity of declarative data transformation while drawing some parallels with DevOps.

4. Data Completeness: To Stream or Not to Stream

   The tremendous push towards streaming pipelines lately has led some to believe that the days of batch processing are either over or will soon be coming to an end. But is this actually true? In this post, I explore the reasons for keeping some of your batch pipelines.

5. Evolving Large Datasets with SQLMesh

   Learn about what SQLMesh has to offer when it comes to evolving really large datasets.

6. Traps and Pitfalls of Using SQL with Jinja

   The practice of augmenting SQL code with Jinja templates has gained significant traction and is consistently growing in popularity. However, is this truly the optimal approach? In this article, we examine the common challenges that arise when combining SQL and Jinja, while also suggesting a more effective method for metaprogramming in SQL.

7. Simplicity or Efficiency: How dbt Makes You Choose

   Comparing development experience and runtime efficiency of dbt and SQLMesh.

8. Semantic Understanding of SQL

   Exploring what semantic understanding is and how it helps improve user experience when using SQL.

9. Automatically detecting breaking changes in SQL queries

   Categorizing changes in SQL queries to automatically establish data contracts between producers and consumers.

10. Virtual Data Environments

    Creating highly efficient and scalable development environments for data.