freshlybakedca.ke
Git fork
1git-pack-objects(1)
2===================
3
4NAME
5----
6git-pack-objects - Create a packed archive of objects
7
8
9SYNOPSIS
10--------
11[verse]
12'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied]
13 [--no-reuse-delta] [--delta-base-offset] [--non-empty]
14 [--local] [--incremental] [--window=<n>] [--depth=<n>]
15 [--revs [--unpacked | --all]] [--keep-pack=<pack-name>]
16 [--cruft] [--cruft-expiration=<time>]
17 [--stdout [--filter=<filter-spec>] | <base-name>]
18 [--shallow] [--keep-true-parents] [--[no-]sparse]
19 [--name-hash-version=<n>] [--path-walk] < <object-list>
20
21
22DESCRIPTION
23-----------
24Reads list of objects from the standard input, and writes either one or
25more packed archives with the specified base-name to disk, or a packed
26archive to the standard output.
27
28A packed archive is an efficient way to transfer a set of objects
29between two repositories as well as an access efficient archival
30format. In a packed archive, an object is either stored as a
31compressed whole or as a difference from some other object.
32The latter is often called a delta.
33
34The packed archive format (.pack) is designed to be self-contained
35so that it can be unpacked without any further information. Therefore,
36each object that a delta depends upon must be present within the pack.
37
38A pack index file (.idx) is generated for fast, random access to the
39objects in the pack. Placing both the index file (.idx) and the packed
40archive (.pack) in the pack/ subdirectory of $GIT_OBJECT_DIRECTORY (or
41any of the directories on $GIT_ALTERNATE_OBJECT_DIRECTORIES)
42enables Git to read from the pack archive.
43
44The 'git unpack-objects' command can read the packed archive and
45expand the objects contained in the pack into "one-file
46one-object" format; this is typically done by the smart-pull
47commands when a pack is created on-the-fly for efficient network
48transport by their peers.
49
50
51OPTIONS
52-------
53base-name::
54 Write into pairs of files (.pack and .idx), using
55 <base-name> to determine the name of the created file.
56 When this option is used, the two files in a pair are written in
57 <base-name>-<SHA-1>.{pack,idx} files. <SHA-1> is a hash
58 based on the pack content and is written to the standard
59 output of the command.
60
61--stdout::
62 Write the pack contents (what would have been written to
63 .pack file) out to the standard output.
64
65--revs::
66 Read the revision arguments from the standard input, instead of
67 individual object names. The revision arguments are processed
68 the same way as 'git rev-list' with the `--objects` flag
69 uses its `commit` arguments to build the list of objects it
70 outputs. The objects on the resulting list are packed.
71 Besides revisions, `--not` or `--shallow <SHA-1>` lines are
72 also accepted.
73
74--unpacked::
75 This implies `--revs`. When processing the list of
76 revision arguments read from the standard input, limit
77 the objects packed to those that are not already packed.
78
79--all::
80 This implies `--revs`. In addition to the list of
81 revision arguments read from the standard input, pretend
82 as if all refs under `refs/` are specified to be
83 included.
84
85--include-tag::
86 Include unasked-for annotated tags if the object they
87 reference was included in the resulting packfile. This
88 can be useful to send new tags to native Git clients.
89
90--stdin-packs[=<mode>]::
91 Read the basenames of packfiles (e.g., `pack-1234abcd.pack`)
92 from the standard input, instead of object names or revision
93 arguments. The resulting pack contains all objects listed in the
94 included packs (those not beginning with `^`), excluding any
95 objects listed in the excluded packs (beginning with `^`).
96+
97When `mode` is "follow", objects from packs not listed on stdin receive
98special treatment. Objects within unlisted packs will be included if
99those objects are (1) reachable from the included packs, and (2) not
100found in any excluded packs. This mode is useful, for example, to
101resurrect once-unreachable objects found in cruft packs to generate
102packs which are closed under reachability up to the boundary set by the
103excluded packs.
104+
105Incompatible with `--revs`, or options that imply `--revs` (such as
106`--all`), with the exception of `--unpacked`, which is compatible.
107
108--cruft::
109 Packs unreachable objects into a separate "cruft" pack, denoted
110 by the existence of a `.mtimes` file. Typically used by `git
111 repack --cruft`. Callers provide a list of pack names and
112 indicate which packs will remain in the repository, along with
113 which packs will be deleted (indicated by the `-` prefix). The
114 contents of the cruft pack are all objects not contained in the
115 surviving packs which have not exceeded the grace period (see
116 `--cruft-expiration` below), or which have exceeded the grace
117 period, but are reachable from an other object which hasn't.
118+
119When the input lists a pack containing all reachable objects (and lists
120all other packs as pending deletion), the corresponding cruft pack will
121contain all unreachable objects (with mtime newer than the
122`--cruft-expiration`) along with any unreachable objects whose mtime is
123older than the `--cruft-expiration`, but are reachable from an
124unreachable object whose mtime is newer than the `--cruft-expiration`).
125+
126Incompatible with `--unpack-unreachable`, `--keep-unreachable`,
127`--pack-loose-unreachable`, `--stdin-packs`, as well as any other
128options which imply `--revs`.
129
130--cruft-expiration=<approxidate>::
131 If specified, objects are eliminated from the cruft pack if they
132 have an mtime older than `<approxidate>`. If unspecified (and
133 given `--cruft`), then no objects are eliminated.
134
135--window=<n>::
136--depth=<n>::
137 These two options affect how the objects contained in
138 the pack are stored using delta compression. The
139 objects are first internally sorted by type, size and
140 optionally names and compared against the other objects
141 within --window to see if using delta compression saves
142 space. --depth limits the maximum delta depth; making
143 it too deep affects the performance on the unpacker
144 side, because delta data needs to be applied that many
145 times to get to the necessary object.
146+
147The default value for --window is 10 and --depth is 50. The maximum
148depth is 4095.
149
150--window-memory=<n>::
151 This option provides an additional limit on top of `--window`;
152 the window size will dynamically scale down so as to not take
153 up more than '<n>' bytes in memory. This is useful in
154 repositories with a mix of large and small objects to not run
155 out of memory with a large window, but still be able to take
156 advantage of the large window for the smaller objects. The
157 size can be suffixed with "k", "m", or "g".
158 `--window-memory=0` makes memory usage unlimited. The default
159 is taken from the `pack.windowMemory` configuration variable.
160
161--max-pack-size=<n>::
162 In unusual scenarios, you may not be able to create files
163 larger than a certain size on your filesystem, and this option
164 can be used to tell the command to split the output packfile
165 into multiple independent packfiles, each not larger than the
166 given size. The size can be suffixed with
167 "k", "m", or "g". The minimum size allowed is limited to 1 MiB.
168 The default is unlimited, unless the config variable
169 `pack.packSizeLimit` is set. Note that this option may result in
170 a larger and slower repository; see the discussion in
171 `pack.packSizeLimit`.
172
173--honor-pack-keep::
174 This flag causes an object already in a local pack that
175 has a .keep file to be ignored, even if it would have
176 otherwise been packed.
177
178--keep-pack=<pack-name>::
179 This flag causes an object already in the given pack to be
180 ignored, even if it would have otherwise been
181 packed. `<pack-name>` is the pack file name without
182 leading directory (e.g. `pack-123.pack`). The option could be
183 specified multiple times to keep multiple packs.
184
185--incremental::
186 This flag causes an object already in a pack to be ignored
187 even if it would have otherwise been packed.
188
189--local::
190 This flag causes an object that is borrowed from an alternate
191 object store to be ignored even if it would have otherwise been
192 packed.
193
194--non-empty::
195 Only create a packed archive if it would contain at
196 least one object.
197
198--progress::
199 Progress status is reported on the standard error stream
200 by default when it is attached to a terminal, unless -q
201 is specified. This flag forces progress status even if
202 the standard error stream is not directed to a terminal.
203
204--all-progress::
205 When --stdout is specified then progress report is
206 displayed during the object count and compression phases
207 but inhibited during the write-out phase. The reason is
208 that in some cases the output stream is directly linked
209 to another command which may wish to display progress
210 status of its own as it processes incoming pack data.
211 This flag is like --progress except that it forces progress
212 report for the write-out phase as well even if --stdout is
213 used.
214
215--all-progress-implied::
216 This is used to imply --all-progress whenever progress display
217 is activated. Unlike --all-progress this flag doesn't actually
218 force any progress display by itself.
219
220-q::
221 This flag makes the command not to report its progress
222 on the standard error stream.
223
224--no-reuse-delta::
225 When creating a packed archive in a repository that
226 has existing packs, the command reuses existing deltas.
227 This sometimes results in a slightly suboptimal pack.
228 This flag tells the command not to reuse existing deltas
229 but compute them from scratch.
230
231--no-reuse-object::
232 This flag tells the command not to reuse existing object data at all,
233 including non deltified object, forcing recompression of everything.
234 This implies --no-reuse-delta. Useful only in the obscure case where
235 wholesale enforcement of a different compression level on the
236 packed data is desired.
237
238--compression=<n>::
239 Specifies compression level for newly-compressed data in the
240 generated pack. If not specified, pack compression level is
241 determined first by pack.compression, then by core.compression,
242 and defaults to -1, the zlib default, if neither is set.
243 Add --no-reuse-object if you want to force a uniform compression
244 level on all data no matter the source.
245
246--sparse::
247--no-sparse::
248 Toggle the "sparse" algorithm to determine which objects to include in
249 the pack, when combined with the "--revs" option. This algorithm
250 only walks trees that appear in paths that introduce new objects.
251 This can have significant performance benefits when computing
252 a pack to send a small change. However, it is possible that extra
253 objects are added to the pack-file if the included commits contain
254 certain types of direct renames. If this option is not included,
255 it defaults to the value of `pack.useSparse`, which is true unless
256 otherwise specified.
257
258--thin::
259 Create a "thin" pack by omitting the common objects between a
260 sender and a receiver in order to reduce network transfer. This
261 option only makes sense in conjunction with --stdout.
262+
263Note: A thin pack violates the packed archive format by omitting
264required objects and is thus unusable by Git without making it
265self-contained. Use `git index-pack --fix-thin`
266(see linkgit:git-index-pack[1]) to restore the self-contained property.
267
268--shallow::
269 Optimize a pack that will be provided to a client with a shallow
270 repository. This option, combined with --thin, can result in a
271 smaller pack at the cost of speed.
272
273--delta-base-offset::
274 A packed archive can express the base object of a delta as
275 either a 20-byte object name or as an offset in the
276 stream, but ancient versions of Git don't understand the
277 latter. By default, 'git pack-objects' only uses the
278 former format for better compatibility. This option
279 allows the command to use the latter format for
280 compactness. Depending on the average delta chain
281 length, this option typically shrinks the resulting
282 packfile by 3-5 per-cent.
283+
284Note: Porcelain commands such as `git gc` (see linkgit:git-gc[1]),
285`git repack` (see linkgit:git-repack[1]) pass this option by default
286in modern Git when they put objects in your repository into pack files.
287So does `git bundle` (see linkgit:git-bundle[1]) when it creates a bundle.
288
289--threads=<n>::
290 Specifies the number of threads to spawn when searching for best
291 delta matches. This requires that pack-objects be compiled with
292 pthreads otherwise this option is ignored with a warning.
293 This is meant to reduce packing time on multiprocessor machines.
294 The required amount of memory for the delta search window is
295 however multiplied by the number of threads.
296 Specifying 0 will cause Git to auto-detect the number of CPU's
297 and set the number of threads accordingly.
298
299--index-version=<version>[,<offset>]::
300 This is intended to be used by the test suite only. It allows
301 to force the version for the generated pack index, and to force
302 64-bit index entries on objects located above the given offset.
303
304--keep-true-parents::
305 With this option, parents that are hidden by grafts are packed
306 nevertheless.
307
308--filter=<filter-spec>::
309 Omits certain objects (usually blobs) from the resulting
310 packfile. See linkgit:git-rev-list[1] for valid
311 `<filter-spec>` forms.
312
313--no-filter::
314 Turns off any previous `--filter=` argument.
315
316--missing=<missing-action>::
317 A debug option to help with future "partial clone" development.
318 This option specifies how missing objects are handled.
319+
320The form '--missing=error' requests that pack-objects stop with an error if
321a missing object is encountered. If the repository is a partial clone, an
322attempt to fetch missing objects will be made before declaring them missing.
323This is the default action.
324+
325The form '--missing=allow-any' will allow object traversal to continue
326if a missing object is encountered. No fetch of a missing object will occur.
327Missing objects will silently be omitted from the results.
328+
329The form '--missing=allow-promisor' is like 'allow-any', but will only
330allow object traversal to continue for EXPECTED promisor missing objects.
331No fetch of a missing object will occur. An unexpected missing object will
332raise an error.
333
334--exclude-promisor-objects::
335 Omit objects that are known to be in the promisor remote. (This
336 option has the purpose of operating only on locally created objects,
337 so that when we repack, we still maintain a distinction between
338 locally created objects [without .promisor] and objects from the
339 promisor remote [with .promisor].) This is used with partial clone.
340
341--keep-unreachable::
342 Objects unreachable from the refs in packs named with
343 --unpacked= option are added to the resulting pack, in
344 addition to the reachable objects that are not in packs marked
345 with *.keep files. This implies `--revs`.
346
347--pack-loose-unreachable::
348 Pack unreachable loose objects (and their loose counterparts
349 removed). This implies `--revs`.
350
351--unpack-unreachable::
352 Keep unreachable objects in loose form. This implies `--revs`.
353
354--delta-islands::
355 Restrict delta matches based on "islands". See DELTA ISLANDS
356 below.
357
358--name-hash-version=<n>::
359 While performing delta compression, Git groups objects that may be
360 similar based on heuristics using the path to that object. While
361 grouping objects by an exact path match is good for paths with
362 many versions, there are benefits for finding delta pairs across
363 different full paths. Git collects objects by type and then by a
364 "name hash" of the path and then by size, hoping to group objects
365 that will compress well together.
366+
367The default name hash version is `1`, which prioritizes hash locality by
368considering the final bytes of the path as providing the maximum magnitude
369to the hash function. This version excels at distinguishing short paths
370and finding renames across directories. However, the hash function depends
371primarily on the final 16 bytes of the path. If there are many paths in
372the repo that have the same final 16 bytes and differ only by parent
373directory, then this name-hash may lead to too many collisions and cause
374poor results. At the moment, this version is required when writing
375reachability bitmap files with `--write-bitmap-index`.
376+
377The name hash version `2` has similar locality features as version `1`,
378except it considers each path component separately and overlays the hashes
379with a shift. This still prioritizes the final bytes of the path, but also
380"salts" the lower bits of the hash using the parent directory names. This
381method allows for some of the locality benefits of version `1` while
382breaking most of the collisions from a similarly-named file appearing in
383many different directories. At the moment, this version is not allowed
384when writing reachability bitmap files with `--write-bitmap-index` and it
385will be automatically changed to version `1`.
386
387--path-walk::
388 Perform compression by first organizing objects by path, then a
389 second pass that compresses across paths as normal. This has the
390 potential to improve delta compression especially in the presence
391 of filenames that cause collisions in Git's default name-hash
392 algorithm.
393+
394Incompatible with `--delta-islands`, `--shallow`, or `--filter`. The
395`--use-bitmap-index` option will be ignored in the presence of
396`--path-walk.`
397
398
399DELTA ISLANDS
400-------------
401
402When possible, `pack-objects` tries to reuse existing on-disk deltas to
403avoid having to search for new ones on the fly. This is an important
404optimization for serving fetches, because it means the server can avoid
405inflating most objects at all and just send the bytes directly from
406disk. This optimization can't work when an object is stored as a delta
407against a base which the receiver does not have (and which we are not
408already sending). In that case the server "breaks" the delta and has to
409find a new one, which has a high CPU cost. Therefore it's important for
410performance that the set of objects in on-disk delta relationships match
411what a client would fetch.
412
413In a normal repository, this tends to work automatically. The objects
414are mostly reachable from the branches and tags, and that's what clients
415fetch. Any deltas we find on the server are likely to be between objects
416the client has or will have.
417
418But in some repository setups, you may have several related but separate
419groups of ref tips, with clients tending to fetch those groups
420independently. For example, imagine that you are hosting several "forks"
421of a repository in a single shared object store, and letting clients
422view them as separate repositories through `GIT_NAMESPACE` or separate
423repos using the alternates mechanism. A naive repack may find that the
424optimal delta for an object is against a base that is only found in
425another fork. But when a client fetches, they will not have the base
426object, and we'll have to find a new delta on the fly.
427
428A similar situation may exist if you have many refs outside of
429`refs/heads/` and `refs/tags/` that point to related objects (e.g.,
430`refs/pull` or `refs/changes` used by some hosting providers). By
431default, clients fetch only heads and tags, and deltas against objects
432found only in those other groups cannot be sent as-is.
433
434Delta islands solve this problem by allowing you to group your refs into
435distinct "islands". Pack-objects computes which objects are reachable
436from which islands, and refuses to make a delta from an object `A`
437against a base which is not present in all of `A`'s islands. This
438results in slightly larger packs (because we miss some delta
439opportunities), but guarantees that a fetch of one island will not have
440to recompute deltas on the fly due to crossing island boundaries.
441
442When repacking with delta islands the delta window tends to get
443clogged with candidates that are forbidden by the config. Repacking
444with a big --window helps (and doesn't take as long as it otherwise
445might because we can reject some object pairs based on islands before
446doing any computation on the content).
447
448Islands are configured via the `pack.island` option, which can be
449specified multiple times. Each value is a left-anchored regular
450expressions matching refnames. For example:
451
452-------------------------------------------
453[pack]
454island = refs/heads/
455island = refs/tags/
456-------------------------------------------
457
458puts heads and tags into an island (whose name is the empty string; see
459below for more on naming). Any refs which do not match those regular
460expressions (e.g., `refs/pull/123`) is not in any island. Any object
461which is reachable only from `refs/pull/` (but not heads or tags) is
462therefore not a candidate to be used as a base for `refs/heads/`.
463
464Refs are grouped into islands based on their "names", and two regexes
465that produce the same name are considered to be in the same
466island. The names are computed from the regexes by concatenating any
467capture groups from the regex, with a '-' dash in between. (And if
468there are no capture groups, then the name is the empty string, as in
469the above example.) This allows you to create arbitrary numbers of
470islands. Only up to 14 such capture groups are supported though.
471
472For example, imagine you store the refs for each fork in
473`refs/virtual/ID`, where `ID` is a numeric identifier. You might then
474configure:
475
476-------------------------------------------
477[pack]
478island = refs/virtual/([0-9]+)/heads/
479island = refs/virtual/([0-9]+)/tags/
480island = refs/virtual/([0-9]+)/(pull)/
481-------------------------------------------
482
483That puts the heads and tags for each fork in their own island (named
484"1234" or similar), and the pull refs for each go into their own
485"1234-pull".
486
487Note that we pick a single island for each regex to go into, using "last
488one wins" ordering (which allows repo-specific config to take precedence
489over user-wide config, and so forth).
490
491
492CONFIGURATION
493-------------
494
495Various configuration variables affect packing, see
496linkgit:git-config[1] (search for "pack" and "delta").
497
498Notably, delta compression is not used on objects larger than the
499`core.bigFileThreshold` configuration variable and on files with the
500attribute `delta` set to false.
501
502SEE ALSO
503--------
504linkgit:git-rev-list[1]
505linkgit:git-repack[1]
506linkgit:git-prune-packed[1]
507
508GIT
509---
510Part of the linkgit:git[1] suite