Sparse, Partial, Shallow, and LFS

Sparse, partial, shallow, LFS — four ways to handle big

The Linux kernel repo is 5 GB. The Chromium repo is 30+ GB. Game asset repos can be hundreds of GB. Plain git clone on these is painful or impossible on a normal laptop. Git has four mechanisms to deal with size, each solving a different problem. Knowing which one to reach for matters.

Shallow clone (git clone --depth N) downloads only the most recent N commits. Fastest clone, smallest download, useful for one-off CI runs and demos. The cost: git blame, git bisect, and most history queries become inaccurate or impossible. Use shallow only when you genuinely do not need history. Promote to full history with git fetch --unshallow when needed.

Partial clone (git clone --filter=blob:none and friends) downloads commit + tree objects but defers blob fetching until checkout. Modern, available since Git 2.20. Better than shallow for most workflows: history queries that do not need blob content stay fast, and blobs are fetched on-demand when you actually look at files. --filter=tree:0 defers tree downloads too, useful for very narrow checkouts.

Sparse checkout chooses which paths from the repo's tree are materialized in your working directory. Combined with partial clone, it gives you a tiny working tree even from a giant repo. git sparse-checkout init --cone and git sparse-checkout set frontend/ docs/ populate only those paths. Used by monorepos to give each engineer a focused workspace.

Git LFS ("Large File Storage") replaces big binary files (videos, datasets, design assets) with small pointer files in the Git repo, while the actual binaries live on a dedicated LFS server. Required for binary-heavy projects because Git's diffing/packing is optimized for text. Track filetypes with git lfs track "*.psd", install git-lfs hooks once, and pushes/pulls automatically route binaries to LFS storage.

Code

Shallow vs partial clone·bash

# Shallow — fast, no history depth:
git clone --depth 1 https://github.com/torvalds/linux.git
# git blame and bisect become limited.

# Partial — modern, lazy blob fetch:
git clone --filter=blob:none https://github.com/torvalds/linux.git
# Blobs are fetched on demand when you actually open files.

# Tree-also-deferred (best with sparse-checkout):
git clone --filter=tree:0 https://github.com/...

# Promote shallow → full history when needed:
git fetch --unshallow

Sparse-checkout for monorepo focus·bash

# Inside a cloned repo, init sparse-checkout in cone mode:
git sparse-checkout init --cone

# Materialize only these paths in the working tree:
git sparse-checkout set frontend/web docs/api

# Add another path later:
git sparse-checkout add backend/auth

# Disable sparse-checkout (full tree):
git sparse-checkout disable

# Combined with partial clone is the monorepo combo:
git clone --filter=blob:none --sparse https://...
cd repo
git sparse-checkout init --cone
git sparse-checkout set my/team/path

Git LFS for big binary assets·bash

# Install git-lfs once on your machine:
brew install git-lfs        # macOS
git lfs install              # registers hooks for the current user

# In a repo, mark filetypes for LFS storage:
git lfs track "*.psd"
git lfs track "*.mp4"
git add .gitattributes
git commit -m "Configure LFS for design + video assets"

# Push as normal — git-lfs intercepts and stores binaries on the LFS server.

# Inspect what is in LFS for this repo:
git lfs ls-files

Exercise

Clone any large public repo (e.g. https://github.com/torvalds/linux.git) two ways: --depth 1 shallow and --filter=blob:none partial. Time both, note disk usage with du -sh. In the partial clone, run git log -p on a recent file and observe the lazy-fetch behavior. Then on a smaller repo, configure sparse-checkout to materialize only one subdirectory. Note in writing which of the four mechanisms you would actually adopt at work and why.

Sparse, Partial, Shallow, and LFS

Sparse, partial, shallow, LFS — four ways to handle big

Code

External links

Exercise

Progress

Comments 0