Dolt: When Databases Marry Git – An A-Z Guide to Data Version Control – ITFROMZERO

Table of Contents

Why is Dolt Catching the Attention of Data Engineers?

As developers, we are all too familiar with Git for source code management. But with databases, things are much more complicated. Previously, when working with MySQL or PostgreSQL, every time I wanted to experiment with a new schema, I usually had to dump SQL files or clone tens of gigabytes of data, which is extremely resource-intensive. A single UPDATE command without a WHERE clause on production is enough to cause sleepless nights.

Dolt was born to end those pain points. It is a SQL database management system (100% MySQL compatible) but with Git’s DNA. You can dolt commit, dolt branch, and dolt merge data instantly.

After testing it on Machine Learning datasets and complex configuration systems, I realized Dolt is a lifesaver for staging environments that require absolute safety.

Quick Start: Get to Know Dolt in 5 Minutes

Installation is very straightforward because Dolt consists of a single binary file.

1. Installing Dolt

On Linux or macOS, you only need a single command:

sudo curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash

2. Initializing the Database

Similar to git init, create a directory and initialize the environment:

mkdir project_db && cd project_db
dolt init

3. Basic SQL Operations

You can execute SQL directly from the terminal without installing additional clients:

dolt sql -q "CREATE TABLE users (id INT PRIMARY KEY, name VARCHAR(50))"
dolt sql -q "INSERT INTO users VALUES (1, 'Hoang IT')"

4. Committing Changes

This is when we save a “snapshot” of the data:

dolt add .
dolt commit -m "Create users table and add sample data"

Branching and Merging: Isolating Changes for Stress-Free Testing

In real-world projects, schema migrations often come with high risks. With traditional MySQL, you usually pray before running a script. With Dolt, just create a new branch and go for it.

Creating a Branch for Testing

Suppose I need to add a phone_number column but I’m not sure if this schema is optimal yet:

dolt checkout -b feature/add-phone
dolt sql -q "ALTER TABLE users ADD COLUMN phone VARCHAR(15)"
dolt commit -a -m "Add phone column on a separate branch"

If you switch back to the main branch, the users table remains clean. This allows you to test new features without worrying about breaking the team’s data.

Smart Data Merging

Once everything looks good, merge the changes into the main branch:

dolt checkout main
dolt merge feature/add-phone

Dolt handles merges at the cell level. If two people update two different rows, Dolt will automatically merge the data smoothly, without causing conflicts like the way we merge code.

Comparing Data with Dolt Diff

The dolt diff feature is my favorite. Instead of using expensive database comparison tools, you just need to type:

dolt diff

The screen will clearly show: which rows were deleted (red) and which were added (green). When debugging a data bug from two hours ago, this feature helps you find the culprit in seconds.

Connecting Like a Real MySQL Server

Dolt isn’t just limited to the CLI. You can run it as a server to connect via tools like DBeaver or TablePlus:

dolt sql-server --host 0.0.0.0 --port 3306

In particular, you can perform versioning directly using SQL commands via procedures:

-- Commit directly within your SQL client
CALL DOLT_COMMIT('-am', 'Update from DBeaver');

Practical Experience: Don’t Overlook Performance

I have deployed Dolt for a product catalog management system, and here are a few important notes:

Performance: Due to storing history via Prolly Trees, Dolt’s write speed is about 2-3 times slower than MySQL. However, read speed shows almost no significant difference.
Storage: Database size will grow faster due to history tracking. Don’t forget to run dolt gc periodically to optimize disk space.
DoltHub: If your team works remotely, use DoltHub. It’s like GitHub for data, allowing you to push/pull and review changes very intuitively.

Conclusion

Dolt provides powerful “rollback” capabilities for your database. If your project frequently requires configuration changes or involves AI/ML that needs dataset version tracking, Dolt is an option you cannot ignore. Try installing it and experience the feeling of managing a database as securely as managing source code.