SQLMesh now supports dev_only VDE mode, where the Virtual Data Environment (VDE) is applied only in development.
What’s new
SQLMesh now supports dev_only VDE mode, where the Virtual Data Environment (VDE) is applied only in development. In production, SQLMesh bypasses the virtual layer and materializes models directly as physical tables using the original, unversioned model names. The option is virtual_environment_mode=dev_only. The default remains full, so nothing changes unless you opt in. Let’s get into it!
Why we made it
Use dev_only when there are specific benefits only available to physical tables in prod. Here are two examples: some catalogs and third‑party platforms don’t play well with a virtual view layer, and some warehouse features are table‑centric, such as time travel. If those integration points matter to you, dev_only removes the indirection in prod.
The SQLMesh default remains full VDE mode. With full VDE, you have versioned snapshots, instant view swaps, and rollbacks without recompute. You also get automatic change categorization and reuse of built data across environments. These capabilities reduce compute and keep prod and dev aligned without relying on engine‑specific features.
When to use it
The decision to use dev_only VDE mode depends on your specific needs and technology. Here are some examples:
• Your catalog or governance tooling expects tables and does not handle a virtual view layer cleanly.
• You want to lean on table‑first warehouse capabilities at the physical layer. Time travel is one example that can be easier to use with physical-based tables.
What stays the same
Don’t worry. The defaults are unchanged. If you do nothing, SQLMesh continues to use full VDE across dev and prod environments. Your modeling workflow and CLI are the same. This is a config switch.
How to enable it
virtual_environment_mode: dev_only
#python
from sqlmesh.core.config import Config
config = Config(
virtual_environment_mode="dev_only",
)See the docs for additional details.
How it behaves
In development, nothing changes about how you preview and validate. SQLMesh still categorizes changes as breaking or non‑breaking, and uses that to decide what needs to be computed for testing and preview. Indirect non‑breaking changes are skipped automatically in dev previews, which keeps compute lean.
In production, there is no virtual layer. Promotions update physical tables using the original model names rather than creating new versioned tables. Reverts are applied going forward and may require a restatement.
Considerations before you flip the switch
• One‑time rebuild of existing projects - Switching an existing project to dev_only triggers a complete rebuild of all models. Plan a maintenance window or use the Table Migration Guide to avoid recomputing from scratch.
• Preview‑only in dev - Data inserted in development is for preview and is not reused in prod. If you depend on dev‑to‑prod reuse, adjust your process.
• Forward‑only reverts - Reverting to a previous version is applied going forward and may require explicit restatement.
What benefits still remain with full VDE
If you maintain the default full VDE mode, you keep data reuse from prod to dev, reuse of changes built in one dev environment across other dev environments, automatic change categorization, and smart builds that minimize recompute while preserving correctness. Those benefits continue to be strong.
Summary
dev_only VDE mode keeps virtual environments where you need them most, in development, and gives production a clean, table‑first surface that plays nicely with catalogs and table‑centric warehouse features. It is off by default, easy to enable, and comes with clear tradeoffs: a one‑time rebuild on first switch, preview‑only data in dev, and forward‑only style reverts in prod. Use it when your platform, governance, and specific needs in prod are table‑centric. Keep full VDE if you want maximum data reuse and virtual‑layer swaps as your default path.
{{banner-slack}}
