freshlybakedca.ke / git
Git fork
fork atom

Add 'promisor-remote' capability to protocol v2

When a server S knows that some objects from a repository are available
from a promisor remote X, S might want to suggest to a client C cloning
or fetching the repo from S that C may use X directly instead of S for
these objects.

Note that this could happen both in the case S itself doesn't have the
objects and borrows them from X, and in the case S has the objects but
knows that X is better connected to the world (e.g., it is in a
$LARGEINTERNETCOMPANY datacenter with petabit/s backbone connections)
than S. Implementation of the latter case, which would require S to
omit in its response the objects available on X, is left for future
improvement though.

Then C might or might not, want to get the objects from X. If S and C
can agree on C using X directly, S can then omit objects that can be
obtained from X when answering C's request.

To allow S and C to agree and let each other know about C using X or
not, let's introduce a new "promisor-remote" capability in the
protocol v2, as well as a few new configuration variables:

- "promisor.advertise" on the server side, and:
- "promisor.acceptFromServer" on the client side.

By default, or if "promisor.advertise" is set to 'false', a server S will
not advertise the "promisor-remote" capability.

If S doesn't advertise the "promisor-remote" capability, then a client C
replying to S shouldn't advertise the "promisor-remote" capability
either.

If "promisor.advertise" is set to 'true', S will advertise its promisor
remotes with a string like:

promisor-remote=<pr-info>[;<pr-info>]...

where each <pr-info> element contains information about a single
promisor remote in the form:

name=<pr-name>[,url=<pr-url>]

where <pr-name> is the urlencoded name of a promisor remote and
<pr-url> is the urlencoded URL of the promisor remote named <pr-name>.

For now, the URL is passed in addition to the name. In the future, it
might be possible to pass other information like a filter-spec that the
client may use when cloning from S, or a token that the client may use
when retrieving objects from X.

It is C's responsibility to arrange how it can reach X though, so pieces
of information that are usually outside Git's concern, like proxy
configuration, must not be distributed over this protocol.

It might also be possible in the future for "promisor.advertise" to have
other values. For example a value like "onlyName" could prevent S from
advertising URLs, which could help in case C should use a different URL
for X than the URL S is using. (The URL S is using might be an internal
one on the server side for example.)

By default or if "promisor.acceptFromServer" is set to "None", C will
not accept to use the promisor remotes that might have been advertised
by S. In this case, C will not advertise any "promisor-remote"
capability in its reply to S.

If "promisor.acceptFromServer" is set to "All" and S advertised some
promisor remotes, then on the contrary, C will accept to use all the
promisor remotes that S advertised and C will reply with a string like:

promisor-remote=<pr-name>[;<pr-name>]...

where the <pr-name> elements are the urlencoded names of all the
promisor remotes S advertised.

In a following commit, other values for "promisor.acceptFromServer" will
be implemented, so that C will be able to decide the promisor remotes it
accepts depending on the name and URL it received from S. So even if
that name and URL information is not used much right now, it will be
needed soon.

Helped-by: Taylor Blau <me@ttaylorr.com>
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

authored by

Christian Couder and committed by
Junio C Hamano d4602676 03944513

+584 -1
unified split
+17
Documentation/config/promisor.adoc
··· 1 1 promisor.quiet:: 2 2 If set to "true" assume `--quiet` when fetching additional 3 3 objects for a partial clone. 4 + 5 + promisor.advertise:: 6 + If set to "true", a server will use the "promisor-remote" 7 + capability, see linkgit:gitprotocol-v2[5], to advertise the 8 + promisor remotes it is using, if it uses some. Default is 9 + "false", which means the "promisor-remote" capability is not 10 + advertised. 11 + 12 + promisor.acceptFromServer:: 13 + If set to "all", a client will accept all the promisor remotes 14 + a server might advertise using the "promisor-remote" 15 + capability. Default is "none", which means no promisor remote 16 + advertised by a server will be accepted. By accepting a 17 + promisor remote, the client agrees that the server might omit 18 + objects that are lazily fetchable from this promisor remote 19 + from its responses to "fetch" and "clone" requests from the 20 + client. See linkgit:gitprotocol-v2[5].
+54
Documentation/gitprotocol-v2.adoc
··· 781 781 save themselves and the server(s) the request(s) needed to inspect the 782 782 headers of that bundle or bundles. 783 783 784 + promisor-remote=<pr-infos> 785 + ~~~~~~~~~~~~~~~~~~~~~~~~~~ 786 + 787 + The server may advertise some promisor remotes it is using or knows 788 + about to a client which may want to use them as its promisor remotes, 789 + instead of this repository. In this case <pr-infos> should be of the 790 + form: 791 + 792 + pr-infos = pr-info | pr-infos ";" pr-info 793 + 794 + pr-info = "name=" pr-name | "name=" pr-name "," "url=" pr-url 795 + 796 + where `pr-name` is the urlencoded name of a promisor remote, and 797 + `pr-url` the urlencoded URL of that promisor remote. 798 + 799 + In this case, if the client decides to use one or more promisor 800 + remotes the server advertised, it can reply with 801 + "promisor-remote=<pr-names>" where <pr-names> should be of the form: 802 + 803 + pr-names = pr-name | pr-names ";" pr-name 804 + 805 + where `pr-name` is the urlencoded name of a promisor remote the server 806 + advertised and the client accepts. 807 + 808 + Note that, everywhere in this document, `pr-name` MUST be a valid 809 + remote name, and the ';' and ',' characters MUST be encoded if they 810 + appear in `pr-name` or `pr-url`. 811 + 812 + If the server doesn't know any promisor remote that could be good for 813 + a client to use, or prefers a client not to use any promisor remote it 814 + uses or knows about, it shouldn't advertise the "promisor-remote" 815 + capability at all. 816 + 817 + In this case, or if the client doesn't want to use any promisor remote 818 + the server advertised, the client shouldn't advertise the 819 + "promisor-remote" capability at all in its reply. 820 + 821 + The "promisor.advertise" and "promisor.acceptFromServer" configuration 822 + options can be used on the server and client side to control what they 823 + advertise or accept respectively. See the documentation of these 824 + configuration options for more information. 825 + 826 + Note that in the future it would be nice if the "promisor-remote" 827 + protocol capability could be used by the server, when responding to 828 + `git fetch` or `git clone`, to advertise better-connected remotes that 829 + the client can use as promisor remotes, instead of this repository, so 830 + that the client can lazily fetch objects from these other 831 + better-connected remotes. This would require the server to omit in its 832 + response the objects available on the better-connected remotes that 833 + the client has accepted. This hasn't been implemented yet though. So 834 + for now this "promisor-remote" capability is useful only when the 835 + server advertises some promisor remotes it already uses to borrow 836 + objects from. 837 + 784 838 GIT 785 839 --- 786 840 Part of the linkgit:git[1] suite
+9
connect.c
··· 22 22 #include "protocol.h" 23 23 #include "alias.h" 24 24 #include "bundle-uri.h" 25 + #include "promisor-remote.h" 25 26 26 27 static char *server_capabilities_v1; 27 28 static struct strvec server_capabilities_v2 = STRVEC_INIT; ··· 487 488 static void send_capabilities(int fd_out, struct packet_reader *reader) 488 489 { 489 490 const char *hash_name; 491 + const char *promisor_remote_info; 490 492 491 493 if (server_supports_v2("agent")) 492 494 packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized()); ··· 499 501 packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); 500 502 } else { 501 503 reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; 504 + } 505 + if (server_feature_v2("promisor-remote", &promisor_remote_info)) { 506 + char *reply = promisor_remote_reply(promisor_remote_info); 507 + if (reply) { 508 + packet_write_fmt(fd_out, "promisor-remote=%s", reply); 509 + free(reply); 510 + } 502 511 } 503 512 } 504 513
+194
promisor-remote.c
··· 11 11 #include "strvec.h" 12 12 #include "packfile.h" 13 13 #include "environment.h" 14 + #include "url.h" 15 + #include "version.h" 14 16 15 17 struct promisor_remote_config { 16 18 struct promisor_remote *promisors; ··· 221 223 return !!repo_promisor_remote_find(r, NULL); 222 224 } 223 225 226 + int repo_has_accepted_promisor_remote(struct repository *r) 227 + { 228 + struct promisor_remote *p; 229 + 230 + promisor_remote_init(r); 231 + 232 + for (p = r->promisor_remote_config->promisors; p; p = p->next) 233 + if (p->accepted) 234 + return 1; 235 + return 0; 236 + } 237 + 224 238 static int remove_fetched_oids(struct repository *repo, 225 239 struct object_id **oids, 226 240 int oid_nr, int to_free) ··· 292 306 if (to_free) 293 307 free(remaining_oids); 294 308 } 309 + 310 + static int allow_unsanitized(char ch) 311 + { 312 + if (ch == ',' || ch == ';' || ch == '%') 313 + return 0; 314 + return ch > 32 && ch < 127; 315 + } 316 + 317 + static void promisor_info_vecs(struct repository *repo, 318 + struct strvec *names, 319 + struct strvec *urls) 320 + { 321 + struct promisor_remote *r; 322 + 323 + promisor_remote_init(repo); 324 + 325 + for (r = repo->promisor_remote_config->promisors; r; r = r->next) { 326 + char *url; 327 + char *url_key = xstrfmt("remote.%s.url", r->name); 328 + 329 + strvec_push(names, r->name); 330 + strvec_push(urls, git_config_get_string(url_key, &url) ? NULL : url); 331 + 332 + free(url); 333 + free(url_key); 334 + } 335 + } 336 + 337 + char *promisor_remote_info(struct repository *repo) 338 + { 339 + struct strbuf sb = STRBUF_INIT; 340 + int advertise_promisors = 0; 341 + struct strvec names = STRVEC_INIT; 342 + struct strvec urls = STRVEC_INIT; 343 + 344 + git_config_get_bool("promisor.advertise", &advertise_promisors); 345 + 346 + if (!advertise_promisors) 347 + return NULL; 348 + 349 + promisor_info_vecs(repo, &names, &urls); 350 + 351 + if (!names.nr) 352 + return NULL; 353 + 354 + for (size_t i = 0; i < names.nr; i++) { 355 + if (i) 356 + strbuf_addch(&sb, ';'); 357 + strbuf_addstr(&sb, "name="); 358 + strbuf_addstr_urlencode(&sb, names.v[i], allow_unsanitized); 359 + if (urls.v[i]) { 360 + strbuf_addstr(&sb, ",url="); 361 + strbuf_addstr_urlencode(&sb, urls.v[i], allow_unsanitized); 362 + } 363 + } 364 + 365 + strvec_clear(&names); 366 + strvec_clear(&urls); 367 + 368 + return strbuf_detach(&sb, NULL); 369 + } 370 + 371 + enum accept_promisor { 372 + ACCEPT_NONE = 0, 373 + ACCEPT_ALL 374 + }; 375 + 376 + static int should_accept_remote(enum accept_promisor accept, 377 + const char *remote_name UNUSED, 378 + const char *remote_url UNUSED) 379 + { 380 + if (accept == ACCEPT_ALL) 381 + return 1; 382 + 383 + BUG("Unhandled 'enum accept_promisor' value '%d'", accept); 384 + } 385 + 386 + static void filter_promisor_remote(struct strvec *accepted, const char *info) 387 + { 388 + struct strbuf **remotes; 389 + const char *accept_str; 390 + enum accept_promisor accept = ACCEPT_NONE; 391 + 392 + if (!git_config_get_string_tmp("promisor.acceptfromserver", &accept_str)) { 393 + if (!*accept_str || !strcasecmp("None", accept_str)) 394 + accept = ACCEPT_NONE; 395 + else if (!strcasecmp("All", accept_str)) 396 + accept = ACCEPT_ALL; 397 + else 398 + warning(_("unknown '%s' value for '%s' config option"), 399 + accept_str, "promisor.acceptfromserver"); 400 + } 401 + 402 + if (accept == ACCEPT_NONE) 403 + return; 404 + 405 + /* Parse remote info received */ 406 + 407 + remotes = strbuf_split_str(info, ';', 0); 408 + 409 + for (size_t i = 0; remotes[i]; i++) { 410 + struct strbuf **elems; 411 + const char *remote_name = NULL; 412 + const char *remote_url = NULL; 413 + char *decoded_name = NULL; 414 + char *decoded_url = NULL; 415 + 416 + strbuf_strip_suffix(remotes[i], ";"); 417 + elems = strbuf_split(remotes[i], ','); 418 + 419 + for (size_t j = 0; elems[j]; j++) { 420 + int res; 421 + strbuf_strip_suffix(elems[j], ","); 422 + res = skip_prefix(elems[j]->buf, "name=", &remote_name) || 423 + skip_prefix(elems[j]->buf, "url=", &remote_url); 424 + if (!res) 425 + warning(_("unknown element '%s' from remote info"), 426 + elems[j]->buf); 427 + } 428 + 429 + if (remote_name) 430 + decoded_name = url_percent_decode(remote_name); 431 + if (remote_url) 432 + decoded_url = url_percent_decode(remote_url); 433 + 434 + if (decoded_name && should_accept_remote(accept, decoded_name, decoded_url)) 435 + strvec_push(accepted, decoded_name); 436 + 437 + strbuf_list_free(elems); 438 + free(decoded_name); 439 + free(decoded_url); 440 + } 441 + 442 + strbuf_list_free(remotes); 443 + } 444 + 445 + char *promisor_remote_reply(const char *info) 446 + { 447 + struct strvec accepted = STRVEC_INIT; 448 + struct strbuf reply = STRBUF_INIT; 449 + 450 + filter_promisor_remote(&accepted, info); 451 + 452 + if (!accepted.nr) 453 + return NULL; 454 + 455 + for (size_t i = 0; i < accepted.nr; i++) { 456 + if (i) 457 + strbuf_addch(&reply, ';'); 458 + strbuf_addstr_urlencode(&reply, accepted.v[i], allow_unsanitized); 459 + } 460 + 461 + strvec_clear(&accepted); 462 + 463 + return strbuf_detach(&reply, NULL); 464 + } 465 + 466 + void mark_promisor_remotes_as_accepted(struct repository *r, const char *remotes) 467 + { 468 + struct strbuf **accepted_remotes = strbuf_split_str(remotes, ';', 0); 469 + 470 + for (size_t i = 0; accepted_remotes[i]; i++) { 471 + struct promisor_remote *p; 472 + char *decoded_remote; 473 + 474 + strbuf_strip_suffix(accepted_remotes[i], ";"); 475 + decoded_remote = url_percent_decode(accepted_remotes[i]->buf); 476 + 477 + p = repo_promisor_remote_find(r, decoded_remote); 478 + if (p) 479 + p->accepted = 1; 480 + else 481 + warning(_("accepted promisor remote '%s' not found"), 482 + decoded_remote); 483 + 484 + free(decoded_remote); 485 + } 486 + 487 + strbuf_list_free(accepted_remotes); 488 + }
+36 -1
promisor-remote.h
··· 9 9 * Promisor remote linked list 10 10 * 11 11 * Information in its fields come from remote.XXX config entries or 12 - * from extensions.partialclone. 12 + * from extensions.partialclone, except for 'accepted' which comes 13 + * from protocol v2 capabilities exchange. 13 14 */ 14 15 struct promisor_remote { 15 16 struct promisor_remote *next; 16 17 char *partial_clone_filter; 18 + unsigned int accepted : 1; 17 19 const char name[FLEX_ARRAY]; 18 20 }; 19 21 ··· 31 33 void promisor_remote_get_direct(struct repository *repo, 32 34 const struct object_id *oids, 33 35 int oid_nr); 36 + 37 + /* 38 + * Prepare a "promisor-remote" advertisement by a server. 39 + * Check the value of "promisor.advertise" and maybe the configured 40 + * promisor remotes, if any, to prepare information to send in an 41 + * advertisement. 42 + * Return value is NULL if no promisor remote advertisement should be 43 + * made. Otherwise it contains the names and urls of the advertised 44 + * promisor remotes separated by ';'. See gitprotocol-v2(5). 45 + */ 46 + char *promisor_remote_info(struct repository *repo); 47 + 48 + /* 49 + * Prepare a reply to a "promisor-remote" advertisement from a server. 50 + * Check the value of "promisor.acceptfromserver" and maybe the 51 + * configured promisor remotes, if any, to prepare the reply. 52 + * Return value is NULL if no promisor remote from the server 53 + * is accepted. Otherwise it contains the names of the accepted promisor 54 + * remotes separated by ';'. See gitprotocol-v2(5). 55 + */ 56 + char *promisor_remote_reply(const char *info); 57 + 58 + /* 59 + * Set the 'accepted' flag for some promisor remotes. Useful on the 60 + * server side when some promisor remotes have been accepted by the 61 + * client. 62 + */ 63 + void mark_promisor_remotes_as_accepted(struct repository *repo, const char *remotes); 64 + 65 + /* 66 + * Has any promisor remote been accepted by the client? 67 + */ 68 + int repo_has_accepted_promisor_remote(struct repository *r); 34 69 35 70 #endif /* PROMISOR_REMOTE_H */
+26
serve.c
··· 10 10 #include "upload-pack.h" 11 11 #include "bundle-uri.h" 12 12 #include "trace2.h" 13 + #include "promisor-remote.h" 13 14 14 15 static int advertise_sid = -1; 15 16 static int advertise_object_info = -1; ··· 28 29 strbuf_addstr(value, git_user_agent_sanitized()); 29 30 return 1; 30 31 } 32 + 33 + static int promisor_remote_advertise(struct repository *r, 34 + struct strbuf *value) 35 + { 36 + if (value) { 37 + char *info = promisor_remote_info(r); 38 + if (!info) 39 + return 0; 40 + strbuf_addstr(value, info); 41 + free(info); 42 + } 43 + return 1; 44 + } 45 + 46 + static void promisor_remote_receive(struct repository *r, 47 + const char *remotes) 48 + { 49 + mark_promisor_remotes_as_accepted(r, remotes); 50 + } 51 + 31 52 32 53 static int object_format_advertise(struct repository *r, 33 54 struct strbuf *value) ··· 154 175 .name = "bundle-uri", 155 176 .advertise = bundle_uri_advertise, 156 177 .command = bundle_uri_command, 178 + }, 179 + { 180 + .name = "promisor-remote", 181 + .advertise = promisor_remote_advertise, 182 + .receive = promisor_remote_receive, 157 183 }, 158 184 }; 159 185
+1
t/meson.build
··· 728 728 't5703-upload-pack-ref-in-want.sh', 729 729 't5704-protocol-violations.sh', 730 730 't5705-session-id-in-capabilities.sh', 731 + 't5710-promisor-remote-capability.sh', 731 732 't5730-protocol-v2-bundle-uri-file.sh', 732 733 't5731-protocol-v2-bundle-uri-git.sh', 733 734 't5732-protocol-v2-bundle-uri-http.sh',
+244
t/t5710-promisor-remote-capability.sh
··· 1 + #!/bin/sh 2 + 3 + test_description='handling of promisor remote advertisement' 4 + 5 + . ./test-lib.sh 6 + 7 + GIT_TEST_MULTI_PACK_INDEX=0 8 + GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL=0 9 + 10 + # Setup the repository with three commits, this way HEAD is always 11 + # available and we can hide commit 1 or 2. 12 + test_expect_success 'setup: create "template" repository' ' 13 + git init template && 14 + test_commit -C template 1 && 15 + test_commit -C template 2 && 16 + test_commit -C template 3 && 17 + test-tool genrandom foo 10240 >template/foo && 18 + git -C template add foo && 19 + git -C template commit -m foo 20 + ' 21 + 22 + # A bare repo will act as a server repo with unpacked objects. 23 + test_expect_success 'setup: create bare "server" repository' ' 24 + git clone --bare --no-local template server && 25 + mv server/objects/pack/pack-* . && 26 + packfile=$(ls pack-*.pack) && 27 + git -C server unpack-objects --strict <"$packfile" 28 + ' 29 + 30 + check_missing_objects () { 31 + git -C "$1" rev-list --objects --all --missing=print > all.txt && 32 + perl -ne 'print if s/^[?]//' all.txt >missing.txt && 33 + test_line_count = "$2" missing.txt && 34 + if test "$2" -lt 2 35 + then 36 + test "$3" = "$(cat missing.txt)" 37 + else 38 + test -f "$3" && 39 + sort <"$3" >expected_sorted && 40 + sort <missing.txt >actual_sorted && 41 + test_cmp expected_sorted actual_sorted 42 + fi 43 + } 44 + 45 + initialize_server () { 46 + count="$1" 47 + missing_oids="$2" 48 + 49 + # Repack everything first 50 + git -C server -c repack.writebitmaps=false repack -a -d && 51 + 52 + # Remove promisor file in case they exist, useful when reinitializing 53 + rm -rf server/objects/pack/*.promisor && 54 + 55 + # Repack without the largest object and create a promisor pack on server 56 + git -C server -c repack.writebitmaps=false repack -a -d \ 57 + --filter=blob:limit=5k --filter-to="$(pwd)/pack" && 58 + promisor_file=$(ls server/objects/pack/*.pack | sed "s/\.pack/.promisor/") && 59 + >"$promisor_file" && 60 + 61 + # Check objects missing on the server 62 + check_missing_objects server "$count" "$missing_oids" 63 + } 64 + 65 + copy_to_lop () { 66 + oid_path="$(test_oid_to_path $1)" && 67 + path="server/objects/$oid_path" && 68 + path2="lop/objects/$oid_path" && 69 + mkdir -p $(dirname "$path2") && 70 + cp "$path" "$path2" 71 + } 72 + 73 + test_expect_success "setup for testing promisor remote advertisement" ' 74 + # Create another bare repo called "lop" (for Large Object Promisor) 75 + git init --bare lop && 76 + 77 + # Copy the largest object from server to lop 78 + obj="HEAD:foo" && 79 + oid="$(git -C server rev-parse $obj)" && 80 + copy_to_lop "$oid" && 81 + 82 + initialize_server 1 "$oid" && 83 + 84 + # Configure lop as promisor remote for server 85 + git -C server remote add lop "file://$(pwd)/lop" && 86 + git -C server config remote.lop.promisor true && 87 + 88 + git -C lop config uploadpack.allowFilter true && 89 + git -C lop config uploadpack.allowAnySHA1InWant true && 90 + git -C server config uploadpack.allowFilter true && 91 + git -C server config uploadpack.allowAnySHA1InWant true 92 + ' 93 + 94 + test_expect_success "clone with promisor.advertise set to 'true'" ' 95 + git -C server config promisor.advertise true && 96 + 97 + # Clone from server to create a client 98 + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ 99 + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ 100 + -c remote.lop.url="file://$(pwd)/lop" \ 101 + -c promisor.acceptfromserver=All \ 102 + --no-local --filter="blob:limit=5k" server client && 103 + test_when_finished "rm -rf client" && 104 + 105 + # Check that the largest object is still missing on the server 106 + check_missing_objects server 1 "$oid" 107 + ' 108 + 109 + test_expect_success "clone with promisor.advertise set to 'false'" ' 110 + git -C server config promisor.advertise false && 111 + 112 + # Clone from server to create a client 113 + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ 114 + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ 115 + -c remote.lop.url="file://$(pwd)/lop" \ 116 + -c promisor.acceptfromserver=All \ 117 + --no-local --filter="blob:limit=5k" server client && 118 + test_when_finished "rm -rf client" && 119 + 120 + # Check that the largest object is not missing on the server 121 + check_missing_objects server 0 "" && 122 + 123 + # Reinitialize server so that the largest object is missing again 124 + initialize_server 1 "$oid" 125 + ' 126 + 127 + test_expect_success "clone with promisor.acceptfromserver set to 'None'" ' 128 + git -C server config promisor.advertise true && 129 + 130 + # Clone from server to create a client 131 + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ 132 + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ 133 + -c remote.lop.url="file://$(pwd)/lop" \ 134 + -c promisor.acceptfromserver=None \ 135 + --no-local --filter="blob:limit=5k" server client && 136 + test_when_finished "rm -rf client" && 137 + 138 + # Check that the largest object is not missing on the server 139 + check_missing_objects server 0 "" && 140 + 141 + # Reinitialize server so that the largest object is missing again 142 + initialize_server 1 "$oid" 143 + ' 144 + 145 + test_expect_success "init + fetch with promisor.advertise set to 'true'" ' 146 + git -C server config promisor.advertise true && 147 + 148 + test_when_finished "rm -rf client" && 149 + mkdir client && 150 + git -C client init && 151 + git -C client config remote.lop.promisor true && 152 + git -C client config remote.lop.fetch "+refs/heads/*:refs/remotes/lop/*" && 153 + git -C client config remote.lop.url "file://$(pwd)/lop" && 154 + git -C client config remote.server.url "file://$(pwd)/server" && 155 + git -C client config remote.server.fetch "+refs/heads/*:refs/remotes/server/*" && 156 + git -C client config promisor.acceptfromserver All && 157 + GIT_NO_LAZY_FETCH=0 git -C client fetch --filter="blob:limit=5k" server && 158 + 159 + # Check that the largest object is still missing on the server 160 + check_missing_objects server 1 "$oid" 161 + ' 162 + 163 + test_expect_success "clone with promisor.advertise set to 'true' but don't delete the client" ' 164 + git -C server config promisor.advertise true && 165 + 166 + # Clone from server to create a client 167 + GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ 168 + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ 169 + -c remote.lop.url="file://$(pwd)/lop" \ 170 + -c promisor.acceptfromserver=All \ 171 + --no-local --filter="blob:limit=5k" server client && 172 + 173 + # Check that the largest object is still missing on the server 174 + check_missing_objects server 1 "$oid" 175 + ' 176 + 177 + test_expect_success "setup for subsequent fetches" ' 178 + # Generate new commit with large blob 179 + test-tool genrandom bar 10240 >template/bar && 180 + git -C template add bar && 181 + git -C template commit -m bar && 182 + 183 + # Fetch new commit with large blob 184 + git -C server fetch origin && 185 + git -C server update-ref HEAD FETCH_HEAD && 186 + git -C server rev-parse HEAD >expected_head && 187 + 188 + # Repack everything twice and remove .promisor files before 189 + # each repack. This makes sure everything gets repacked 190 + # into a single packfile. The second repack is necessary 191 + # because the first one fetches from lop and creates a new 192 + # packfile and its associated .promisor file. 193 + 194 + rm -f server/objects/pack/*.promisor && 195 + git -C server -c repack.writebitmaps=false repack -a -d && 196 + rm -f server/objects/pack/*.promisor && 197 + git -C server -c repack.writebitmaps=false repack -a -d && 198 + 199 + # Unpack everything 200 + rm pack-* && 201 + mv server/objects/pack/pack-* . && 202 + packfile=$(ls pack-*.pack) && 203 + git -C server unpack-objects --strict <"$packfile" && 204 + 205 + # Copy new large object to lop 206 + obj_bar="HEAD:bar" && 207 + oid_bar="$(git -C server rev-parse $obj_bar)" && 208 + copy_to_lop "$oid_bar" && 209 + 210 + # Reinitialize server so that the 2 largest objects are missing 211 + printf "%s\n" "$oid" "$oid_bar" >expected_missing.txt && 212 + initialize_server 2 expected_missing.txt && 213 + 214 + # Create one more client 215 + cp -r client client2 216 + ' 217 + 218 + test_expect_success "subsequent fetch from a client when promisor.advertise is true" ' 219 + git -C server config promisor.advertise true && 220 + 221 + GIT_NO_LAZY_FETCH=0 git -C client pull origin && 222 + 223 + git -C client rev-parse HEAD >actual && 224 + test_cmp expected_head actual && 225 + 226 + cat client/bar >/dev/null && 227 + 228 + check_missing_objects server 2 expected_missing.txt 229 + ' 230 + 231 + test_expect_success "subsequent fetch from a client when promisor.advertise is false" ' 232 + git -C server config promisor.advertise false && 233 + 234 + GIT_NO_LAZY_FETCH=0 git -C client2 pull origin && 235 + 236 + git -C client2 rev-parse HEAD >actual && 237 + test_cmp expected_head actual && 238 + 239 + cat client2/bar >/dev/null && 240 + 241 + check_missing_objects server 1 "$oid" 242 + ' 243 + 244 + test_done
+3
upload-pack.c
··· 32 32 #include "write-or-die.h" 33 33 #include "json-writer.h" 34 34 #include "strmap.h" 35 + #include "promisor-remote.h" 35 36 36 37 /* Remember to update object flag allocation in object.h */ 37 38 #define THEY_HAVE (1u << 11) ··· 319 320 strvec_push(&pack_objects.args, "--delta-base-offset"); 320 321 if (pack_data->use_include_tag) 321 322 strvec_push(&pack_objects.args, "--include-tag"); 323 + if (repo_has_accepted_promisor_remote(the_repository)) 324 + strvec_push(&pack_objects.args, "--missing=allow-promisor"); 322 325 if (pack_data->filter_options.choice) { 323 326 const char *spec = 324 327 expand_list_objects_filter_spec(&pack_data->filter_options);