unpack-objects: use the bulk-checkin infrastructure

Git fork

The unpack-objects functionality is used by fetch, push, and fast-import
to turn the transfered data into object database entries when there are
fewer objects than the 'unpacklimit' setting.

By enabling an odb-transaction when unpacking objects, we can take advantage
of batched fsyncs.

Here are some performance numbers to justify batch mode for
unpack-objects, collected on a WSL2 Ubuntu VM.

Fsync Mode | Time for 90 objects (ms)
-------------------------------------
Off | 170
On,fsync | 760
On,batch | 230

Note that the default unpackLimit is 100 objects, so there's a 3x
benefit in the worst case. The non-batch mode fsync scales linearly
with the number of objects, so there are significant benefits even with
smaller numbers of objects.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

authored by

Neeraj Singh and committed by

Junio C Hamano 4 years ago 425d290c 23a3a303

1 changed file

expand all

builtin

unpack-objects.c

builtin/unpack-objects.c

··· 1 1 #include "builtin.h" 2 2 #include "cache.h" 3 + #include "bulk-checkin.h" 3 4 #include "config.h" 4 5 #include "object-store.h" 5 6 #include "object.h" ··· 503 504 if (!quiet) 504 505 progress = start_progress(_("Unpacking objects"), nr_objects); 505 506 CALLOC_ARRAY(obj_list, nr_objects); 507 + begin_odb_transaction(); 506 508 for (i = 0; i < nr_objects; i++) { 507 509 unpack_one(i); 508 510 display_progress(progress, i + 1); 509 511 } 512 + end_odb_transaction(); 510 513 stop_progress(&progress); 511 514 512 515 if (delta_list)