Git Packfiles

TL;DR: Git can store objects as individual zlib-compressed loose files or group them into binary packfiles (.pack) with a matching index (.idx). A pack may store similar objects using delta compression, but Git still presents commits as snapshots; the pack is a storage optimization, not a change-history format.

Object format note: The examples here use Git’s default SHA-1 object format, so you will see 40-character object IDs and 20-byte binary IDs inside .idx files. With Git’s optional SHA-256 object format, IDs are 64 characters (32 bytes) and the packfile index scales accordingly — the delta-compression strategy itself is unchanged.

If you’ve spent any time reading about Git’s architecture, you’ve probably heard that Git stores full snapshots of your files rather than file deltas. As we discuss in how Git differs and Git objects, this snapshot model is what makes Git clean, fast, and conceptually simple.

However, storing each changed file version as a separate loose object can use substantial disk space. For example, changing one line in a 10 MB file creates a new blob object; repeated versions can add up. Git’s packfiles can reduce this cost by grouping objects and delta-compressing similar ones, though the savings depend on the repository and the objects being packed.

Why Git stores thousands of objects in one file

To understand packfiles, you must understand the two distinct states in which Git objects exist: Loose Objects and Packed Objects.

graph TD
    Loose1[Loose Blob: version1.txt <br> 10MB zlib compressed]
    Loose2[Loose Blob: version2.txt <br> 10MB zlib compressed]
    GC[Run git gc / repack] --> PackFile[Packfile: pack-xxx.pack <br> Stores object data; some objects may be deltas]
    IdxFile[Index File: pack-xxx.idx <br> Maps object IDs to pack offsets]

Loose Objects

When you run git add or git commit, Git takes file contents, adds an object header, hashes and compresses the object, and writes it into .git/objects/. These are called loose objects.

• Loose objects are straightforward to write; actual staging and commit performance depends on the filesystem and repository.
• However, because each file version is a standalone zlib-compressed file, Git cannot compress duplicates across history efficiently in this state.

Packed Objects

When Git packs objects, it writes object data into a binary packfile (ending in .pack) and builds a corresponding index file (ending in .idx) to find objects efficiently.

The pack’s physical representation does not change the logical snapshot model.

Delta Compression: Bases and Deltas

Delta compression lets a pack encode one object as instructions to reconstruct it from a similar base object. Git chooses bases while packing; a base can be older or newer, and some objects are stored without deltas. A delta chain may require Git to reconstruct intermediate objects, so Git limits chain depth. Use git verify-pack -v <pack>.idx to inspect the representation in a particular pack rather than assuming the newest version is always stored in full.

Packing a repository and measuring the result

Let’s build a repository, watch loose files pile up, pack them manually, and inspect the resulting binary files using plumbing commands.

Step 1: Create a Repository and Build Loose Objects

Let’s initialize a repository:

$ mkdir git-packfile-demo
$ cd git-packfile-demo
$ git init
Initialized empty Git repository in /path/to/git-packfile-demo/.git/

Now, let’s create a text file containing some lines of text, and modify it in several consecutive commits.

$ echo -e "Line 1: Git Internals\nLine 2: Understanding Packfiles" > content.txt
$ git add content.txt
$ git commit -m "First revision"

$ echo -e "Line 1: Git Internals\nLine 2: Understanding Packfiles\nLine 3: Added some new info" > content.txt
$ git commit -am "Second revision"

$ echo -e "Line 1: Git Internals\nLine 2: Understanding Packfiles\nLine 3: Added some new info\nLine 4: A third revision" > content.txt
$ git commit -am "Third revision"

Let’s look at the loose files inside .git/objects:

(The object IDs in this listing are representative rather than reproducible — a commit or tree hash depends on its contents and timestamps, and no file contents are shown here. The structure, counts, and git gc behaviour are exactly what you will see.)

$ find .git/objects -type f | sort
.git/objects/06/de3410657519a4e41ff34ab7987e9154a4f8df
.git/objects/09/b35e29fa757fb1e4a6435c249a5b3992850cd3
.git/objects/42/bdbe49dbb0ffb8b209b55502c77d9cbb28d09f
.git/objects/59/cdbc68fbfbdcbe812bb1e24749f90263f350c3
.git/objects/a5/d87bc0ee734a7809622d99214b72605f624d77
.git/objects/ba/92b678fa42cb869ab12be3f5bc41a9bc9c3b88
.git/objects/c9/791c2fc886c99c8bf8a89ee1be2a09c2a382e2
.git/objects/d6/f90659d007c6f2df83f4b48bc4a5b060d4b9bf
.git/objects/fa/01d43d1a6015bd67634f19b8813a7c64a3d4f8

We have nine loose objects (three commits, three tree objects, and three blobs).

Step 2: Repacking Loose Objects

Let’s run Git’s garbage collector, which repackages these loose objects:

$ git gc
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 8 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (9/9), done.
Total 9 (delta 1), reused 0 (delta 0), pack-reused 0

Let’s check the .git/objects folder again:

$ find .git/objects -type f
.git/objects/info/commits
.git/objects/info/packs
.git/objects/pack/pack-98e9a2b8e398188fc836859846cd4d34eb51897d.idx
.git/objects/pack/pack-98e9a2b8e398188fc836859846cd4d34eb51897d.pack

All the individual loose object subdirectories are gone! They have been cleaned up and consolidated into two files:

1. pack-98e9a2....pack (the actual data pack)
2. pack-98e9a2....idx (the index mapping hashes to their offset locations inside the .pack file)

Step 3: Inspecting the Packfile contents

We can verify the contents of our new packfile using the plumbing command git verify-pack -v. This reads the .idx file to report on the state of the packfile.

$ git verify-pack -v .git/objects/pack/pack-98e9a2b8e398188fc836859846cd4d34eb51897d.idx

Output:

ba92b678fa42cb869ab12be3f5bc41a9bc9c3b88 commit 172 110 12
09b35e29fa757fb1e4a6435c249a5b3992850cd3 commit 224 140 122
06de3410657519a4e41ff34ab7987e9154a4f8df commit 224 140 262
fa01d43d1a6015bd67634f19b8813a7c64a3d4f8 tree   64 45 402
42bdbe49dbb0ffb8b209b55502c77d9cbb28d09f tree   64 45 447
d6f90659d007c6f2df83f4b48bc4a5b060d4b9bf tree   64 45 492
59cdbc68fbfbdcbe812bb1e24749f90263f350c3 blob   92 68 537
a5d87bc0ee734a7809622d99214b72605f624d77 blob   62 53 605 1 59cdbc68fbfbdcbe812bb1e24749f90263f350c3
c9791c2fc886c99c8bf8a89ee1be2a09c2a382e2 blob   43 38 658 2 a5d87bc0ee734a7809622d99214b72605f624d77
non-delta: 8 objects
chain length = 1: 1 object
chain length = 2: 1 object

Let’s dissect this output:

• Columns read: [SHA-1] [Type] [Decompressed Size] [Compressed Size] [Byte Offset] [Delta Depth] [Parent Base SHA-1].
• In this sample, blob 59cdbc68... is stored as a non-delta object at offset 537.
• Blob a5d87bc0... is a depth-1 delta based on 59cdbc68...; c9791c2... is a depth-2 delta based on a5d87bc0....

These rows show how this pack chose to store these objects; they do not mean Git always stores the latest version in full or uses reverse deltas. Delta compression can reduce size, while git verify-pack reveals the base and depth for the specific pack you inspect.

Delta chains, bitmaps, and gc flags

Git provides underlying mechanics and parameters to tweak how aggressively objects are repacked.

1. The underlying plumbing: git repack

While developers use git gc to clean repositories, git gc is actually a wrapper that calls git repack under the hood. You can use git repack directly:

• git repack -a -d: Packs all loose objects in the repository into a single packfile and deletes (-d) the loose files that were just packed.
• git repack -f: Forces Git to ignore all existing delta associations and re-evaluate every object from scratch. This takes more time and processing power but can shrink your repository significantly if history was modified.

2. Aggressive Garbage Collection

If your repository is getting sluggish or has accumulated a large amount of history, you can trigger an aggressive repack:

$ git gc --aggressive --prune=now

This flag tells Git to spend more CPU time analyzing file histories to construct deeper and more efficient delta chains. It adjusts the window size and depth configurations.

3. Understanding Pack Windows and Depth

Two Git configuration values govern how delta compression behaves:

• pack.window (Default: 10): When packing, Git sorts objects by filename, extension, size, and type, and loads them into a sliding queue (the “window”). Git only compares an object against other objects currently in this window. A larger window allows Git to compare more files to find better deltas, but uses more memory.
• pack.depth (Default: 50): Specifies the maximum length of a delta chain (how many diffs are layered on top of a base object). If the depth is too shallow, files aren’t compressed as much. If it is too deep, Git has to execute too many backward instructions to reconstruct older versions of a file, slowing down commands like git checkout or git log -p.

You can modify these configurations globally or for a specific run:

$ git -c pack.window=250 -c pack.depth=250 repack -a -d -f

How packfiles support clone, fetch, and push

Packfiles are commonly used to transfer objects for git clone, git fetch, and git push. The exact negotiation and packing steps depend on the Git protocol and server implementation.

sequenceDiagram
    participant Client as Local Client
    participant Server as Remote Server
    Client->>Server: git clone / fetch
    Server->>Server: Determine missing objects
    Server->>Server: Select objects the client needs
    Server-->>Client: Transfer pack data
    Client->>Client: Verify and index received objects

1. Negotiate objects: During a fetch, the client and server determine which reachable objects the client is missing.
2. Transfer a pack: Smart transports commonly send the needed objects in a pack stream rather than as a set of separate loose files. Server-side packing and transport details vary.
3. Index received objects: Git processes the incoming pack and makes its objects searchable locally, typically by creating or updating an index. The .idx is an index into the pack, not a record of file changes.

Repacking a repository that grew too large

Scenario A: Rebuilding a Lost Index File

A developer’s computer crashes during a write operation. The file .git/objects/pack/pack-xxx.idx becomes corrupted or is completely deleted. The developer runs git status and gets a crash error because Git can no longer read the objects stored inside the corresponding .pack file.

A cautious recovery path:

If the .pack file itself is intact, its index can often be rebuilt. Make a copy of the repository or pack first, then try:

1. Locate the .pack file path:

   $ ls .git/objects/pack/
   pack-98e9a2b8e398188fc836859846cd4d34eb51897d.pack

2. Reconstruct the .idx index file using git index-pack:

   $ git index-pack .git/objects/pack/pack-98e9a2b8e398188fc836859846cd4d34eb51897d.pack

3. Git scans the pack and writes an index. Then run git fsck and verify that the repository can read the objects you need. If the pack is corrupt or other metadata is missing, rebuilding the index alone will not repair everything.

Scenario B: Shrinking a Bloated Repository

A repository has ballooned to 2GB because developers frequently commit compiled build assets. Even though these assets were deleted in the latest commits, they are still taking up space in the history.

1. Rewrite history with a tool such as git-filter-repo only after coordinating with collaborators; rewriting changes commit IDs and requires them to resynchronize.
2. Make and verify a backup before cleanup. Normal garbage collection may retain unreachable objects for a period; reflogs and configuration affect when they become eligible for pruning.
3. Avoid git reflog expire --expire=now --all and git gc --prune=now on a shared or active repository unless you understand that they can remove recovery paths. If immediate pruning is necessary, stop concurrent writes and work from a verified backup.

When an aggressive gc makes things worse

1. Believing Deleting a Large File and Committing Solves Storage Bloat

The Mistake: Deleting a large 1GB movie file, committing the deletion, and expecting the .git folder size to drop. Why it fails: Git preserves your complete history. Since the commit containing the 1GB file is still part of the repository graph, the 1GB blob remains inside your packfile forever. The Fix: If you must remove the file from history, coordinate the rewrite with collaborators and back up first. Repacking or pruning is a separate step; do not expire reflogs or use --prune=now casually, because those operations can remove recovery data.

2. Manually Deleting Packfiles

The Mistake: Opening .git/objects/pack/ and deleting large .pack files to save disk space. Why it fails: Deleting a packfile deletes the actual object data. If you delete a packfile, you permanently corrupt your repository history. Git will fail to check out files or run commits because the required history is missing. The Fix: Never touch packfiles manually. If you need to reduce space, use git gc or prune operations.

3. Creating Too Deep Delta Chains (pack.depth too high)

The Mistake: Configuring pack.depth to a very high number (like 1000) hoping to get the smallest possible repository size. Why it fails: While this might save a few extra megabytes, checking out older files or running git log -p requires Git to compute a chain of 1000 deltas sequentially. This degrades performance significantly, making the developer experience slow. Stick to reasonable defaults unless you have specific constraints.

Packfile commands at a glance

Packfile questions, answered

What is the difference between loose and packed objects in Git?

Loose objects are individual files under .git/objects/??/, each holding one zlib-compressed Git object. A packfile groups object data in a binary file; some objects may be delta-compressed against a similar base, while others are stored whole. The .idx file maps object IDs to locations in the pack.

What is delta compression in Git packfiles?

Delta compression represents one object as instructions for reconstructing it from a similar base object. Git chooses the base while packing; the base is not always newer, and not every object is stored as a delta. git verify-pack -v shows the delta depth and base for objects in a particular pack.

Can you open or edit a packfile directly?

No. Packfiles are binary files composed of zlib-compressed blocks, offsets, and delta instructions. To read or inspect them, you must use plumbing commands like git verify-pack or git cat-file. Editing them directly will corrupt the entire package.

How does git index-pack work during clone?

In a typical smart-protocol clone or fetch, the client receives pack data for objects it needs and Git indexes those objects locally. The exact transport and pack handling vary; git index-pack can inspect a pack and build its index. An .idx is a lookup structure, not a history of file changes.

Check your understanding