freshlybakedca.ke
Git fork
promisor-remote: allow a server to advertise more fields
For now the "promisor-remote" protocol capability can only pass "name"
and "url" information from a server to a client in the form
"name=<remote_name>,url=<remote_url>".
To allow clients to make more informed decisions about which promisor
remotes they accept, let's make it possible to pass more information
by introducing a new "promisor.sendFields" configuration variable.
On the server side, information about a remote `foo` is stored in
configuration variables named `remote.foo.<variable-name>`. To make
it clearer and simpler, we use `field` and `field name` like this:
* `field name` refers to the <variable-name> part of such a
configuration variable, and
* `field` refers to both the `field name` and the value of such a
configuration variable.
The "promisor.sendFields" configuration variable should contain a
comma or space separated list of field names that will be looked up
in the configuration of the remote on the server to find the values
that will be passed to the client.
Only a set of predefined field names are allowed. The only field
names in this set are "partialCloneFilter" and "token". The
"partialCloneFilter" field name specifies the filter definition used
by the promisor remote, and the "token" field name can provide an
authentication credential for accessing it.
For example, if "promisor.sendFields" is set to "partialCloneFilter",
and the server has the "remote.foo.partialCloneFilter" config
variable set to a value, then that value will be passed in the
"partialCloneFilter" field in the form "partialCloneFilter=<value>"
after the "name" and "url" fields.
A following commit will allow the client to use the information to
decide if it accepts the remote or not. For now the client doesn't do
anything with the additional information it receives.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
authored by
Christian Couder
and committed by
Junio C Hamano
4bf7ae31
57af9cc2
+22
Documentation/config/promisor.adoc
+22
Documentation/config/promisor.adoc
···
9
9
"false", which means the "promisor-remote" capability is not
10
10
advertised.
11
11
12
+
promisor.sendFields::
13
+
A comma or space separated list of additional remote related
14
+
field names. A server sends these field names and the
15
+
associated field values from its configuration when
16
+
advertising its promisor remotes using the "promisor-remote"
17
+
capability, see linkgit:gitprotocol-v2[5]. Currently, only the
18
+
"partialCloneFilter" and "token" field names are supported.
19
+
+
20
+
`partialCloneFilter`:: contains the partial clone filter
21
+
used for the remote.
22
+
+
23
+
`token`:: contains an authentication token for the remote.
24
+
+
25
+
When a field name is part of this list and a corresponding
26
+
"remote.foo.<field-name>" config variable is set on the server to a
27
+
non-empty value, then the field name and value are sent when
28
+
advertising the promisor remote "foo".
29
+
+
30
+
This list has no effect unless the "promisor.advertise" config
31
+
variable is set to "true", and the "name" and "url" fields are always
32
+
advertised regardless of this setting.
33
+
12
34
promisor.acceptFromServer::
13
35
If set to "all", a client will accept all the promisor remotes
14
36
a server might advertise using the "promisor-remote"
+48
-16
Documentation/gitprotocol-v2.adoc
+48
-16
Documentation/gitprotocol-v2.adoc
···
785
785
save themselves and the server(s) the request(s) needed to inspect the
786
786
headers of that bundle or bundles.
787
787
788
-
promisor-remote=<pr-infos>
789
-
~~~~~~~~~~~~~~~~~~~~~~~~~~
788
+
promisor-remote=<pr-info>
789
+
~~~~~~~~~~~~~~~~~~~~~~~~~
790
790
791
791
The server may advertise some promisor remotes it is using or knows
792
792
about to a client which may want to use them as its promisor remotes,
793
-
instead of this repository. In this case <pr-infos> should be of the
793
+
instead of this repository. In this case <pr-info> should be of the
794
794
form:
795
795
796
-
pr-infos = pr-info | pr-infos ";" pr-info
796
+
pr-info = pr-fields | pr-info ";" pr-fields
797
797
798
-
pr-info = "name=" pr-name | "name=" pr-name "," "url=" pr-url
798
+
pr-fields = pr-field | pr-fields "," pr-field
799
799
800
-
where `pr-name` is the urlencoded name of a promisor remote, and
801
-
`pr-url` the urlencoded URL of that promisor remote.
800
+
pr-field = field-name "=" field-value
802
801
803
-
In this case, if the client decides to use one or more promisor
804
-
remotes the server advertised, it can reply with
805
-
"promisor-remote=<pr-names>" where <pr-names> should be of the form:
802
+
where all the `field-name` and `field-value` in a given `pr-fields`
803
+
are field names and values related to a single promisor remote. A
804
+
given `field-name` MUST NOT appear more than once in given
805
+
`pr-fields`.
806
+
807
+
The server MUST advertise at least the "name" and "url" field names
808
+
along with the associated field values, which are the name of a valid
809
+
remote and its URL, in each `pr-fields`. The "name" and "url" fields
810
+
MUST appear first in each pr-fields, in that order.
811
+
812
+
After these mandatory fields, the server MAY advertise the following
813
+
optional fields in any order:
814
+
815
+
`partialCloneFilter`:: The filter specification used by the remote.
816
+
Clients can use this to determine if the remote's filtering strategy
817
+
is compatible with their needs (e.g., checking if both use "blob:none").
818
+
It corresponds to the "remote.<name>.partialCloneFilter" config setting.
819
+
820
+
`token`:: An authentication token that clients can use when
821
+
connecting to the remote. It corresponds to the "remote.<name>.token"
822
+
config setting.
823
+
824
+
No other fields are defined by the protocol at this time. Field names
825
+
are case-sensitive and MUST be transmitted exactly as specified
826
+
above. Clients MUST ignore fields they don't recognize to allow for
827
+
future protocol extensions.
828
+
829
+
For now, the client can only use information transmitted through these
830
+
fields to decide if it accepts the advertised promisor remote. In the
831
+
future that information might be used for other purposes though.
832
+
833
+
Field values MUST be urlencoded.
834
+
835
+
If the client decides to use one or more promisor remotes the server
836
+
advertised, it can reply with "promisor-remote=<pr-names>" where
837
+
<pr-names> should be of the form:
806
838
807
839
pr-names = pr-name | pr-names ";" pr-name
808
840
809
841
where `pr-name` is the urlencoded name of a promisor remote the server
810
842
advertised and the client accepts.
811
843
812
-
Note that, everywhere in this document, `pr-name` MUST be a valid
813
-
remote name, and the ';' and ',' characters MUST be encoded if they
814
-
appear in `pr-name` or `pr-url`.
844
+
Note that, everywhere in this document, the ';' and ',' characters
845
+
MUST be encoded if they appear in `pr-name` or `field-value`.
815
846
816
847
If the server doesn't know any promisor remote that could be good for
817
848
a client to use, or prefers a client not to use any promisor remote it
···
822
853
the server advertised, the client shouldn't advertise the
823
854
"promisor-remote" capability at all in its reply.
824
855
825
-
The "promisor.advertise" and "promisor.acceptFromServer" configuration
826
-
options can be used on the server and client side to control what they
827
-
advertise or accept respectively. See the documentation of these
856
+
On the server side, the "promisor.advertise" and "promisor.sendFields"
857
+
configuration options can be used to control what it advertises. On
858
+
the client side, the "promisor.acceptFromServer" configuration option
859
+
can be used to control what it accepts. See the documentation of these
828
860
configuration options for more information.
829
861
830
862
Note that in the future it would be nice if the "promisor-remote"
+120
-9
promisor-remote.c
+120
-9
promisor-remote.c
···
314
314
return ch > 32 && ch < 127;
315
315
}
316
316
317
+
static const char promisor_field_filter[] = "partialCloneFilter";
318
+
static const char promisor_field_token[] = "token";
319
+
320
+
/*
321
+
* List of optional field names that can be used in the
322
+
* "promisor-remote" protocol capability (others must be
323
+
* ignored). Each field should correspond to a configurable property
324
+
* of a remote that can be relevant for the client.
325
+
*/
326
+
static const char *known_fields[] = {
327
+
promisor_field_filter, /* Filter used for partial clone */
328
+
promisor_field_token, /* Authentication token for the remote */
329
+
NULL
330
+
};
331
+
332
+
/*
333
+
* Check if 'field' is in the list of the known field names for the
334
+
* "promisor-remote" protocol capability.
335
+
*/
336
+
static int is_known_field(const char *field)
337
+
{
338
+
const char **p;
339
+
340
+
for (p = known_fields; *p; p++)
341
+
if (!strcasecmp(*p, field))
342
+
return 1;
343
+
return 0;
344
+
}
345
+
346
+
static int is_valid_field(struct string_list_item *item, void *cb_data)
347
+
{
348
+
const char *field = item->string;
349
+
const char *config_key = (const char *)cb_data;
350
+
351
+
if (!is_known_field(field)) {
352
+
warning(_("unsupported field '%s' in '%s' config"), field, config_key);
353
+
return 0;
354
+
}
355
+
return 1;
356
+
}
357
+
358
+
static char *fields_from_config(struct string_list *fields_list, const char *config_key)
359
+
{
360
+
char *fields = NULL;
361
+
362
+
if (!repo_config_get_string(the_repository, config_key, &fields) && *fields) {
363
+
string_list_split_in_place_f(fields_list, fields, ",", -1,
364
+
STRING_LIST_SPLIT_TRIM |
365
+
STRING_LIST_SPLIT_NONEMPTY);
366
+
filter_string_list(fields_list, 0, is_valid_field, (void *)config_key);
367
+
}
368
+
369
+
return fields;
370
+
}
371
+
372
+
static struct string_list *fields_sent(void)
373
+
{
374
+
static struct string_list fields_list = STRING_LIST_INIT_NODUP;
375
+
static int initialized;
376
+
377
+
if (!initialized) {
378
+
fields_list.cmp = strcasecmp;
379
+
fields_from_config(&fields_list, "promisor.sendFields");
380
+
initialized = 1;
381
+
}
382
+
383
+
return &fields_list;
384
+
}
385
+
317
386
/*
318
387
* Struct for promisor remotes involved in the "promisor-remote"
319
388
* protocol capability.
···
326
395
struct promisor_info {
327
396
const char *name;
328
397
const char *url;
398
+
const char *filter;
399
+
const char *token;
329
400
};
330
401
331
402
static void promisor_info_list_clear(struct string_list *list)
···
334
405
struct promisor_info *p = list->items[i].util;
335
406
free((char *)p->name);
336
407
free((char *)p->url);
408
+
free((char *)p->filter);
409
+
free((char *)p->token);
337
410
}
338
411
string_list_clear(list, 1);
339
412
}
340
413
414
+
static void set_one_field(struct promisor_info *p,
415
+
const char *field, const char *value)
416
+
{
417
+
if (!strcasecmp(field, promisor_field_filter))
418
+
p->filter = xstrdup(value);
419
+
else if (!strcasecmp(field, promisor_field_token))
420
+
p->token = xstrdup(value);
421
+
else
422
+
BUG("invalid field '%s'", field);
423
+
}
424
+
425
+
static void set_fields(struct promisor_info *p,
426
+
struct string_list *field_names)
427
+
{
428
+
struct string_list_item *item;
429
+
430
+
for_each_string_list_item(item, field_names) {
431
+
char *key = xstrfmt("remote.%s.%s", p->name, item->string);
432
+
const char *val;
433
+
if (!repo_config_get_string_tmp(the_repository, key, &val) && *val)
434
+
set_one_field(p, item->string, val);
435
+
free(key);
436
+
}
437
+
}
438
+
341
439
/*
342
440
* Populate 'list' with promisor remote information from the config.
343
-
* The 'util' pointer of each list item will hold a 'struct promisor_info'.
441
+
* The 'util' pointer of each list item will hold a 'struct
442
+
* promisor_info'. Except "name" and "url", only members of that
443
+
* struct specified by the 'field_names' list are set (using values
444
+
* from the configuration).
344
445
*/
345
-
static void promisor_config_info_list(struct repository *repo, struct string_list *list)
446
+
static void promisor_config_info_list(struct repository *repo,
447
+
struct string_list *list,
448
+
struct string_list *field_names)
346
449
{
347
450
struct promisor_remote *r;
348
451
···
360
463
new_info->name = xstrdup(r->name);
361
464
new_info->url = xstrdup(url);
362
465
466
+
if (field_names)
467
+
set_fields(new_info, field_names);
468
+
363
469
item = string_list_append(list, new_info->name);
364
470
item->util = new_info;
365
471
}
···
380
486
if (!advertise_promisors)
381
487
return NULL;
382
488
383
-
promisor_config_info_list(repo, &config_info);
489
+
promisor_config_info_list(repo, &config_info, fields_sent());
384
490
385
491
if (!config_info.nr)
386
492
return NULL;
···
395
501
strbuf_addstr_urlencode(&sb, p->name, allow_unsanitized);
396
502
strbuf_addstr(&sb, ",url=");
397
503
strbuf_addstr_urlencode(&sb, p->url, allow_unsanitized);
504
+
505
+
if (p->filter) {
506
+
strbuf_addf(&sb, ",%s=", promisor_field_filter);
507
+
strbuf_addstr_urlencode(&sb, p->filter, allow_unsanitized);
508
+
}
509
+
if (p->token) {
510
+
strbuf_addf(&sb, ",%s=", promisor_field_token);
511
+
strbuf_addstr_urlencode(&sb, p->token, allow_unsanitized);
512
+
}
398
513
}
399
514
400
515
promisor_info_list_clear(&config_info);
···
475
590
return;
476
591
477
592
if (accept != ACCEPT_ALL) {
478
-
promisor_config_info_list(repo, &config_info);
593
+
promisor_config_info_list(repo, &config_info, NULL);
479
594
string_list_sort(&config_info);
480
595
}
481
596
···
494
609
elems = strbuf_split(remotes[i], ',');
495
610
496
611
for (size_t j = 0; elems[j]; j++) {
497
-
int res;
498
612
strbuf_strip_suffix(elems[j], ",");
499
-
res = skip_prefix(elems[j]->buf, "name=", &remote_name) ||
613
+
if (!skip_prefix(elems[j]->buf, "name=", &remote_name))
500
614
skip_prefix(elems[j]->buf, "url=", &remote_url);
501
-
if (!res)
502
-
warning(_("unknown element '%s' from remote info"),
503
-
elems[j]->buf);
504
615
}
505
616
506
617
if (remote_name)
+31
t/t5710-promisor-remote-capability.sh
+31
t/t5710-promisor-remote-capability.sh
···
295
295
check_missing_objects server 1 "$oid"
296
296
'
297
297
298
+
test_expect_success "clone with promisor.sendFields" '
299
+
git -C server config promisor.advertise true &&
300
+
test_when_finished "rm -rf client" &&
301
+
302
+
git -C server remote add otherLop "https://invalid.invalid" &&
303
+
git -C server config remote.otherLop.token "fooBar" &&
304
+
git -C server config remote.otherLop.stuff "baz" &&
305
+
git -C server config remote.otherLop.partialCloneFilter "blob:limit=10k" &&
306
+
test_when_finished "git -C server remote remove otherLop" &&
307
+
test_config -C server promisor.sendFields "partialCloneFilter, token" &&
308
+
test_when_finished "rm trace" &&
309
+
310
+
# Clone from server to create a client
311
+
GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \
312
+
-c remote.lop.promisor=true \
313
+
-c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \
314
+
-c remote.lop.url="file://$(pwd)/lop" \
315
+
-c promisor.acceptfromserver=All \
316
+
--no-local --filter="blob:limit=5k" server client &&
317
+
318
+
# Check that fields are properly transmitted
319
+
ENCODED_URL=$(echo "file://$(pwd)/lop" | sed -e "s/ /%20/g") &&
320
+
PR1="name=lop,url=$ENCODED_URL,partialCloneFilter=blob:none" &&
321
+
PR2="name=otherLop,url=https://invalid.invalid,partialCloneFilter=blob:limit=10k,token=fooBar" &&
322
+
test_grep "clone< promisor-remote=$PR1;$PR2" trace &&
323
+
test_grep "clone> promisor-remote=lop;otherLop" trace &&
324
+
325
+
# Check that the largest object is still missing on the server
326
+
check_missing_objects server 1 "$oid"
327
+
'
328
+
298
329
test_expect_success "clone with promisor.advertise set to 'true' but don't delete the client" '
299
330
git -C server config promisor.advertise true &&
300
331