The Problem: When Your Git Repository Bloats and Holds Things It Shouldn’t
Many developers have run into this: you’re reviewing an old repo one day and discover a .env file with a real API key sitting right there in the git history from six months ago. Or maybe the repo is ballooning at 200MB even though the actual code is only a few dozen MB — because someone once accidentally committed a video file, a dataset, or a SQL backup.
Deleting the file from the working tree and committing again doesn’t solve the problem. Git stores the entire history — the file still lives in the object database and can still be checked out using the old commit hash. Anyone who knows to use git log --all and git checkout can still retrieve that content without any trouble.
Why git filter-branch Isn’t a Good Option
The classic solution is git filter-branch — Git’s built-in command for rewriting history. The problem is it’s extremely slow. I once ran git filter-branch on a 500MB repo with around 3,000 commits and it took nearly 2 hours. For repositories with tens of thousands of commits, you’re looking at days.
BFG Repo-Cleaner was built to solve exactly this pain point. According to the author’s benchmarks, BFG is 10 to 720 times faster than git filter-branch depending on the case. Written in Scala, it runs multithreaded and processes the object graph in parallel — that’s why the speed difference is so dramatic.
Installing BFG Repo-Cleaner
Prerequisite: Java Runtime
BFG runs on the JVM and requires Java 8 or later:
java -version
# java version "17.0.8" or similar is fineIf you don’t have Java installed:
# Ubuntu/Debian
sudo apt install default-jre
# macOS (Homebrew)
brew install openjdk
# Windows (Chocolatey)
choco install openjdkDownload the BFG JAR
BFG is a single JAR file — no complex installation needed:
# Download version 1.14.0 (latest stable)
wget https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar -O bfg.jar
# Verify
java -jar bfg.jar --versionFor everyday convenience, add an alias to your shell config:
# Add to ~/.bashrc or ~/.zshrc
alias bfg='java -jar /path/to/bfg.jar'
source ~/.bashrcStep-by-Step: Real-World Use Cases
Preparation: Clone the Repository as a Mirror
BFG works directly on a bare repository. Before doing anything, clone it as a mirror to your local machine:
# Clone as bare (mirror) — do NOT use a regular clone
git clone --mirror [email protected]:username/my-repo.git my-repo.git
# Inspect the bare directory
ls my-repo.git/Working on a bare clone is safer — the original remote repo is unaffected until you deliberately force push. This is an important safety net.
Case 1: Remove a Sensitive File That Was Accidentally Committed
The most common use case. For example, accidentally committing a .env file, secrets.json, or a private key:
cd my-repo.git
# Delete a specific file from the entire history
java -jar bfg.jar --delete-files .env
# Delete multiple files (glob pattern)
java -jar bfg.jar --delete-files '*.pem'
java -jar bfg.jar --delete-files '{.env,.env.local,.env.production}'
# After BFG finishes, you must run these steps
git reflog expire --expire=now --all
git gc --prune=now --aggressiveThe git gc command is what actually frees up disk space — BFG only marks objects as orphaned; it’s gc that removes them from disk.
Case 2: Remove Large Files to Shrink Repo Size
I once encountered a machine learning project repo where someone accidentally committed a 1.5GB dataset folder, ballooning the repo from 50MB to 1.6GB. BFG handles this cleanly:
# Remove all blobs larger than 50MB from history
java -jar bfg.jar --strip-blobs-bigger-than 50M
# Or use a stricter limit
java -jar bfg.jar --strip-blobs-bigger-than 10MImportant note: BFG automatically protects the latest commit (HEAD). If the large file still exists in HEAD, you need to remove it manually first:
# Remove the file from HEAD before running BFG
git rm --cached dataset.zip
git commit -m "chore: remove accidentally committed large file"
# Then run BFG
java -jar bfg.jar --strip-blobs-bigger-than 50MCase 3: Replace Hardcoded Passwords or Tokens in Code
Sometimes you don’t need to delete an entire file — you just need to replace a sensitive string, like a hardcoded API key in a config file:
# Create a file containing the strings to replace (one per line)
cat > passwords.txt << 'EOF'
sk-ant-api03-xxx-yyy-zzz
ghp_xxxxxxxxxxxxxxxxxxxx
AKIAIOSFODNN7EXAMPLE
EOF
# BFG replaces all occurrences of those strings with ***REMOVED***
java -jar bfg.jar --replace-text passwords.txtBFG scans the entire history and precisely replaces those strings. The file structure and all other content remain intact.
Case 4: Remove Entire Directories from History
# Remove the logs/ and node_modules/ directories from the entire history
java -jar bfg.jar --delete-folders logs
java -jar bfg.jar --delete-folders node_modules
# Remove multiple directories at once
java -jar bfg.jar --delete-folders '{logs,tmp,cache,__pycache__}'Verify and Push to Remote
Verify Results Before Pushing
BFG prints a report after each run — read it carefully to see how many commits were rewritten and which files were processed. You should also verify manually:
# Check if the sensitive file has been removed from history
git log --all --full-history -- .env
# No output = successfully removed
# Compare size before and after
git count-objects -vH
# Search for sensitive strings across the entire history
git log -p --all | grep -n "sk-ant-api03"
# No results = cleanForce Push to Remote
Once you’ve verified everything, push to remote. This step rewrites history on the remote — make sure to notify your entire team beforehand, as everyone’s local repos will be out of sync:
cd my-repo.git
# Push all branches and tags
git push --forceEvery team member will need to resync:
# Safest option: re-clone
git clone [email protected]:username/my-repo.git
# Or if you have local changes you want to keep
git fetch --all
git reset --hard origin/mainRotate Credentials Immediately
This is something many people overlook: even after removing credentials from git history, you must still revoke and regenerate all exposed credentials. GitHub, GitLab, and most platforms have automated crawlers that scan public repos — there’s no way to know whether those credentials were already harvested between when they went public and when you removed them. Revoke first, clean second — in that order:
- GitHub Token: Settings → Developer settings → Personal access tokens → Delete
- AWS Key: IAM → Access keys → Deactivate → Delete
- Anthropic API Key: console.anthropic.com → API Keys → Revoke
Prevent Recurrence with .gitignore
Add these to your .gitignore right away so no one accidentally commits them again:
# .gitignore
.env
.env.local
.env.*.local
*.pem
*.key
*.p12
secrets/
config/database.ymlIn the 8-person team I manage, after the accidental .env commit incident, I added a git hook to block commits matching sensitive patterns — combined with a pre-commit check. It hasn’t happened since. Automated prevention at the point of commit is far more effective than catching it in review after the fact.
BFG or git filter-repo?
Since Git 2.24+, there’s also git filter-repo — a Python tool officially recommended by Git as a replacement for git filter-branch, faster than filter-branch and more flexible than BFG. But BFG still holds an advantage in many situations:
- BFG: Simpler syntax for common use cases (deleting files, replacing text, stripping large blobs). No Python dependency. A single JAR — ready to run immediately.
- git filter-repo: Better suited for complex operations like restructuring a repo, splitting a subdirectory into its own repo, or filtering by author.
If your goal is simply to remove sensitive files or shrink a repo’s size, BFG remains the fastest and lowest-risk option — installed in 2 minutes, running in seconds.
