freshlybakedca.ke / git
Git fork
fork atom

fast-(import|export): improve on commit signature output format

A recent commit, d9cb0e6ff8 (fast-export, fast-import: add support for
signed-commits, 2025-03-10), added support for signed commits to
fast-export and fast-import.

When a signed commit is processed, fast-export can output either
"gpgsig sha1" or "gpgsig sha256" depending on whether the signed
commit uses the SHA-1 or SHA-256 Git object format.

However, this implementation has a number of limitations:

- the output format was not properly described in the documentation,
- the output format is not very informative as it doesn't even say
if the signature is an OpenPGP, an SSH, or an X509 signature,
- the implementation doesn't support having both one signature on
the SHA-1 object and one on the SHA-256 object.

Let's improve on these limitations by improving fast-export and
fast-import so that:

- all the signatures are exported,
- at most one signature on the SHA-1 object and one on the SHA-256
are imported,
- if there is more than one signature on the SHA-1 object or on
the SHA-256 object, fast-import emits a warning for each
additional signature,
- the output format is "gpgsig <git-hash-algo> <signature-format>",
where <git-hash-algo> is the Git object format as before, and
<signature-format> is the signature type ("openpgp", "x509",
"ssh" or "unknown"),
- the output is properly documented.

About the output format:

- <git-hash-algo> allows to know which representation of the commit
was signed (the SHA-1 or the SHA-256 version) which helps with
both signature verification and interoperability between repos
with different hash functions,

- <signature-format> helps tools that process the fast-export
stream, so they don't have to parse the ASCII armor to identify
the signature type.

It could be even better to be able to import more than one signature
on the SHA-1 object and on the SHA-256 object, but other parts of
Git don't handle that well for now, so this is left for future
improvements.

Helped-by: brian m. carlson <sandals@crustytoothpaste.net>
Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

authored by

Christian Couder and committed by
Junio C Hamano b5b3ddbe cb3b4038

+312 -44
+17
Documentation/git-fast-export.adoc
··· 50 50 is the same as how earlier versions of this command without 51 51 this option behaved. 52 52 + 53 + When exported, a signature starts with: 54 + + 55 + gpgsig <git-hash-algo> <signature-format> 56 + + 57 + where <git-hash-algo> is the Git object hash so either "sha1" or 58 + "sha256", and <signature-format> is the signature type, so "openpgp", 59 + "x509", "ssh" or "unknown". 60 + + 61 + For example, an OpenPGP signature on a SHA-1 commit starts with 62 + `gpgsig sha1 openpgp`, while an SSH signature on a SHA-256 commit 63 + starts with `gpgsig sha256 ssh`. 64 + + 65 + While all the signatures of a commit are exported, an importer may 66 + choose to accept only some of them. For example 67 + linkgit:git-fast-import[1] currently stores at most one signature per 68 + Git hash algorithm in each commit. 69 + + 53 70 NOTE: This is highly experimental and the format of the data stream may 54 71 change in the future without compatibility guarantees. 55 72
+32 -6
Documentation/git-fast-import.adoc
··· 445 445 original-oid? 446 446 ('author' (SP <name>)? SP LT <email> GT SP <when> LF)? 447 447 'committer' (SP <name>)? SP LT <email> GT SP <when> LF 448 - ('gpgsig' SP <alg> LF data)? 448 + ('gpgsig' SP <algo> SP <format> LF data)? 449 449 ('encoding' SP <encoding> LF)? 450 450 data 451 451 ('from' SP <commit-ish> LF)? ··· 518 518 ^^^^^^^^ 519 519 520 520 The optional `gpgsig` command is used to include a PGP/GPG signature 521 - that signs the commit data. 521 + or other cryptographic signature that signs the commit data. 522 522 523 - Here <alg> specifies which hashing algorithm is used for this 524 - signature, either `sha1` or `sha256`. 523 + .... 524 + 'gpgsig' SP <git-hash-algo> SP <signature-format> LF data 525 + .... 525 526 526 - NOTE: This is highly experimental and the format of the data stream may 527 - change in the future without compatibility guarantees. 527 + The `gpgsig` command takes two arguments: 528 + 529 + * `<git-hash-algo>` specifies which Git object format this signature 530 + applies to, either `sha1` or `sha256`. This allows to know which 531 + representation of the commit was signed (the SHA-1 or the SHA-256 532 + version) which helps with both signature verification and 533 + interoperability between repos with different hash functions. 534 + 535 + * `<signature-format>` specifies the type of signature, such as 536 + `openpgp`, `x509`, `ssh`, or `unknown`. This is a convenience for 537 + tools that process the stream, so they don't have to parse the ASCII 538 + armor to identify the signature type. 539 + 540 + A commit may have at most one signature for the SHA-1 object format 541 + (stored in the "gpgsig" header) and one for the SHA-256 object format 542 + (stored in the "gpgsig-sha256" header). 543 + 544 + See below for a detailed description of the `data` command which 545 + contains the raw signature data. 546 + 547 + Signatures are not yet checked in the current implementation 548 + though. (Already setting the `extensions.compatObjectFormat` 549 + configuration option might help with verifying both SHA-1 and SHA-256 550 + object format signatures when it will be implemented.) 551 + 552 + NOTE: This is highly experimental and the format of the `gpgsig` 553 + command may change in the future without compatibility guarantees. 528 554 529 555 `encoding` 530 556 ^^^^^^^^^^
+48 -14
builtin/fast-export.c
··· 29 29 #include "quote.h" 30 30 #include "remote.h" 31 31 #include "blob.h" 32 + #include "gpg-interface.h" 32 33 33 34 static const char *const fast_export_usage[] = { 34 35 N_("git fast-export [<rev-list-opts>]"), ··· 652 653 return strbuf_detach(&val, NULL); 653 654 } 654 655 656 + static void print_signature(const char *signature, const char *object_hash) 657 + { 658 + if (!signature) 659 + return; 660 + 661 + printf("gpgsig %s %s\ndata %u\n%s\n", 662 + object_hash, 663 + get_signature_format(signature), 664 + (unsigned)strlen(signature), 665 + signature); 666 + } 667 + 668 + static const char *append_signatures_for_header(struct string_list *signatures, 669 + const char *pos, 670 + const char *header, 671 + const char *object_hash) 672 + { 673 + const char *signature; 674 + const char *start = pos; 675 + const char *end = pos; 676 + 677 + while ((signature = find_commit_multiline_header(start + 1, 678 + header, 679 + &end))) { 680 + string_list_append(signatures, signature)->util = (void *)object_hash; 681 + free((char *)signature); 682 + start = end; 683 + } 684 + 685 + return end; 686 + } 687 + 655 688 static void handle_commit(struct commit *commit, struct rev_info *rev, 656 689 struct string_list *paths_of_changed_objects) 657 690 { ··· 660 693 const char *author, *author_end, *committer, *committer_end; 661 694 const char *encoding = NULL; 662 695 size_t encoding_len; 663 - const char *signature_alg = NULL, *signature = NULL; 696 + struct string_list signatures = STRING_LIST_INIT_DUP; 664 697 const char *message; 665 698 char *reencoded = NULL; 666 699 struct commit_list *p; ··· 700 733 } 701 734 702 735 if (*commit_buffer_cursor == '\n') { 703 - if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig", &commit_buffer_cursor))) 704 - signature_alg = "sha1"; 705 - else if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig-sha256", &commit_buffer_cursor))) 706 - signature_alg = "sha256"; 736 + const char *after_sha1 = append_signatures_for_header(&signatures, commit_buffer_cursor, 737 + "gpgsig", "sha1"); 738 + const char *after_sha256 = append_signatures_for_header(&signatures, commit_buffer_cursor, 739 + "gpgsig-sha256", "sha256"); 740 + commit_buffer_cursor = (after_sha1 > after_sha256) ? after_sha1 : after_sha256; 707 741 } 708 742 709 743 message = strstr(commit_buffer_cursor, "\n\n"); ··· 769 803 printf("%.*s\n%.*s\n", 770 804 (int)(author_end - author), author, 771 805 (int)(committer_end - committer), committer); 772 - if (signature) { 806 + if (signatures.nr) { 773 807 switch (signed_commit_mode) { 774 808 case SIGN_ABORT: 775 809 die("encountered signed commit %s; use " 776 810 "--signed-commits=<mode> to handle it", 777 811 oid_to_hex(&commit->object.oid)); 778 812 case SIGN_WARN_VERBATIM: 779 - warning("exporting signed commit %s", 780 - oid_to_hex(&commit->object.oid)); 813 + warning("exporting %"PRIuMAX" signature(s) for commit %s", 814 + (uintmax_t)signatures.nr, oid_to_hex(&commit->object.oid)); 781 815 /* fallthru */ 782 816 case SIGN_VERBATIM: 783 - printf("gpgsig %s\ndata %u\n%s", 784 - signature_alg, 785 - (unsigned)strlen(signature), 786 - signature); 817 + for (size_t i = 0; i < signatures.nr; i++) { 818 + struct string_list_item *item = &signatures.items[i]; 819 + print_signature(item->string, item->util); 820 + } 787 821 break; 788 822 case SIGN_WARN_STRIP: 789 - warning("stripping signature from commit %s", 823 + warning("stripping signature(s) from commit %s", 790 824 oid_to_hex(&commit->object.oid)); 791 825 /* fallthru */ 792 826 case SIGN_STRIP: 793 827 break; 794 828 } 795 - free((char *)signature); 829 + string_list_clear(&signatures, 0); 796 830 } 797 831 if (!reencoded && encoding) 798 832 printf("encoding %.*s\n", (int)encoding_len, encoding);
+91 -22
builtin/fast-import.c
··· 29 29 #include "commit-reach.h" 30 30 #include "khash.h" 31 31 #include "date.h" 32 + #include "gpg-interface.h" 32 33 33 34 #define PACK_ID_BITS 16 34 35 #define MAX_PACK_ID ((1<<PACK_ID_BITS)-1) ··· 2718 2719 return list; 2719 2720 } 2720 2721 2722 + struct signature_data { 2723 + char *hash_algo; /* "sha1" or "sha256" */ 2724 + char *sig_format; /* "openpgp", "x509", "ssh", or "unknown" */ 2725 + struct strbuf data; /* The actual signature data */ 2726 + }; 2727 + 2728 + static void parse_one_signature(struct signature_data *sig, const char *v) 2729 + { 2730 + char *args = xstrdup(v); /* Will be freed when sig->hash_algo is freed */ 2731 + char *space = strchr(args, ' '); 2732 + 2733 + if (!space) 2734 + die("Expected gpgsig format: 'gpgsig <hash-algo> <signature-format>', " 2735 + "got 'gpgsig %s'", args); 2736 + *space = '\0'; 2737 + 2738 + sig->hash_algo = args; 2739 + sig->sig_format = space + 1; 2740 + 2741 + /* Validate hash algorithm */ 2742 + if (strcmp(sig->hash_algo, "sha1") && 2743 + strcmp(sig->hash_algo, "sha256")) 2744 + die("Unknown git hash algorithm in gpgsig: '%s'", sig->hash_algo); 2745 + 2746 + /* Validate signature format */ 2747 + if (!valid_signature_format(sig->sig_format)) 2748 + die("Invalid signature format in gpgsig: '%s'", sig->sig_format); 2749 + if (!strcmp(sig->sig_format, "unknown")) 2750 + warning("'unknown' signature format in gpgsig"); 2751 + 2752 + /* Read signature data */ 2753 + read_next_command(); 2754 + parse_data(&sig->data, 0, NULL); 2755 + } 2756 + 2757 + static void add_gpgsig_to_commit(struct strbuf *commit_data, 2758 + const char *header, 2759 + struct signature_data *sig) 2760 + { 2761 + struct string_list siglines = STRING_LIST_INIT_NODUP; 2762 + 2763 + if (!sig->hash_algo) 2764 + return; 2765 + 2766 + strbuf_addstr(commit_data, header); 2767 + string_list_split_in_place(&siglines, sig->data.buf, "\n", -1); 2768 + strbuf_add_separated_string_list(commit_data, "\n ", &siglines); 2769 + strbuf_addch(commit_data, '\n'); 2770 + string_list_clear(&siglines, 1); 2771 + strbuf_release(&sig->data); 2772 + free(sig->hash_algo); 2773 + } 2774 + 2775 + static void store_signature(struct signature_data *stored_sig, 2776 + struct signature_data *new_sig, 2777 + const char *hash_type) 2778 + { 2779 + if (stored_sig->hash_algo) { 2780 + warning("multiple %s signatures found, " 2781 + "ignoring additional signature", 2782 + hash_type); 2783 + strbuf_release(&new_sig->data); 2784 + free(new_sig->hash_algo); 2785 + } else { 2786 + *stored_sig = *new_sig; 2787 + } 2788 + } 2789 + 2721 2790 static void parse_new_commit(const char *arg) 2722 2791 { 2723 - static struct strbuf sig = STRBUF_INIT; 2724 2792 static struct strbuf msg = STRBUF_INIT; 2725 - struct string_list siglines = STRING_LIST_INIT_NODUP; 2793 + struct signature_data sig_sha1 = { NULL, NULL, STRBUF_INIT }; 2794 + struct signature_data sig_sha256 = { NULL, NULL, STRBUF_INIT }; 2726 2795 struct branch *b; 2727 2796 char *author = NULL; 2728 2797 char *committer = NULL; 2729 - char *sig_alg = NULL; 2730 2798 char *encoding = NULL; 2731 2799 struct hash_list *merge_list = NULL; 2732 2800 unsigned int merge_count; ··· 2750 2818 } 2751 2819 if (!committer) 2752 2820 die("Expected committer but didn't get one"); 2753 - if (skip_prefix(command_buf.buf, "gpgsig ", &v)) { 2754 - sig_alg = xstrdup(v); 2821 + 2822 + /* Process signatures (up to 2: one "sha1" and one "sha256") */ 2823 + while (skip_prefix(command_buf.buf, "gpgsig ", &v)) { 2824 + struct signature_data sig = { NULL, NULL, STRBUF_INIT }; 2825 + 2826 + parse_one_signature(&sig, v); 2827 + 2828 + if (!strcmp(sig.hash_algo, "sha1")) 2829 + store_signature(&sig_sha1, &sig, "SHA-1"); 2830 + else if (!strcmp(sig.hash_algo, "sha256")) 2831 + store_signature(&sig_sha256, &sig, "SHA-256"); 2832 + else 2833 + BUG("parse_one_signature() returned unknown hash algo"); 2834 + 2755 2835 read_next_command(); 2756 - parse_data(&sig, 0, NULL); 2757 - read_next_command(); 2758 - } else 2759 - strbuf_setlen(&sig, 0); 2836 + } 2837 + 2760 2838 if (skip_prefix(command_buf.buf, "encoding ", &v)) { 2761 2839 encoding = xstrdup(v); 2762 2840 read_next_command(); ··· 2830 2908 strbuf_addf(&new_data, 2831 2909 "encoding %s\n", 2832 2910 encoding); 2833 - if (sig_alg) { 2834 - if (!strcmp(sig_alg, "sha1")) 2835 - strbuf_addstr(&new_data, "gpgsig "); 2836 - else if (!strcmp(sig_alg, "sha256")) 2837 - strbuf_addstr(&new_data, "gpgsig-sha256 "); 2838 - else 2839 - die("Expected gpgsig algorithm sha1 or sha256, got %s", sig_alg); 2840 - string_list_split_in_place(&siglines, sig.buf, "\n", -1); 2841 - strbuf_add_separated_string_list(&new_data, "\n ", &siglines); 2842 - strbuf_addch(&new_data, '\n'); 2843 - } 2911 + 2912 + add_gpgsig_to_commit(&new_data, "gpgsig ", &sig_sha1); 2913 + add_gpgsig_to_commit(&new_data, "gpgsig-sha256 ", &sig_sha256); 2914 + 2844 2915 strbuf_addch(&new_data, '\n'); 2845 2916 strbuf_addbuf(&new_data, &msg); 2846 - string_list_clear(&siglines, 1); 2847 2917 free(author); 2848 2918 free(committer); 2849 - free(sig_alg); 2850 2919 free(encoding); 2851 2920 2852 2921 if (!store_object(OBJ_COMMIT, &new_data, NULL, &b->oid, next_mark))
+12
gpg-interface.c
··· 144 144 return NULL; 145 145 } 146 146 147 + const char *get_signature_format(const char *buf) 148 + { 149 + struct gpg_format *format = get_format_by_sig(buf); 150 + return format ? format->name : "unknown"; 151 + } 152 + 153 + int valid_signature_format(const char *format) 154 + { 155 + return (!!get_format_by_name(format) || 156 + !strcmp(format, "unknown")); 157 + } 158 + 147 159 void signature_check_clear(struct signature_check *sigc) 148 160 { 149 161 FREE_AND_NULL(sigc->payload);
+12
gpg-interface.h
··· 48 48 void signature_check_clear(struct signature_check *sigc); 49 49 50 50 /* 51 + * Return the format of the signature (like "openpgp", "x509", "ssh" 52 + * or "unknown"). 53 + */ 54 + const char *get_signature_format(const char *buf); 55 + 56 + /* 57 + * Is the signature format valid (like "openpgp", "x509", "ssh" or 58 + * "unknown") 59 + */ 60 + int valid_signature_format(const char *format); 61 + 62 + /* 51 63 * Look at a GPG signed tag object. If such a signature exists, store it in 52 64 * signature and the signed content in payload. Return 1 if a signature was 53 65 * found, and 0 otherwise.
+100 -2
t/t9350-fast-export.sh
··· 314 314 test_expect_success GPG 'signed-commits=verbatim' ' 315 315 316 316 git fast-export --signed-commits=verbatim --reencode=no commit-signing >output && 317 - grep "^gpgsig sha" output && 317 + test_grep -E "^gpgsig $GIT_DEFAULT_HASH openpgp" output && 318 318 grep "encoding ISO-8859-1" output && 319 319 ( 320 320 cd new && ··· 328 328 test_expect_success GPG 'signed-commits=warn-verbatim' ' 329 329 330 330 git fast-export --signed-commits=warn-verbatim --reencode=no commit-signing >output 2>err && 331 - grep "^gpgsig sha" output && 331 + test_grep -E "^gpgsig $GIT_DEFAULT_HASH openpgp" output && 332 332 grep "encoding ISO-8859-1" output && 333 333 test -s err && 334 334 ( ··· 366 366 STRIPPED=$(git rev-parse --verify refs/heads/commit-strip-signing) && 367 367 test $COMMIT_SIGNING != $STRIPPED 368 368 ) 369 + 370 + ' 371 + 372 + test_expect_success GPGSM 'setup X.509 signed commit' ' 373 + 374 + git checkout -b x509-signing main && 375 + test_config gpg.format x509 && 376 + test_config user.signingkey $GIT_COMMITTER_EMAIL && 377 + echo "X.509 content" >file && 378 + git add file && 379 + git commit -S -m "X.509 signed commit" && 380 + X509_COMMIT=$(git rev-parse HEAD) && 381 + git checkout main 382 + 383 + ' 384 + 385 + test_expect_success GPGSM 'round-trip X.509 signed commit' ' 386 + 387 + git fast-export --signed-commits=verbatim x509-signing >output && 388 + test_grep -E "^gpgsig $GIT_DEFAULT_HASH x509" output && 389 + ( 390 + cd new && 391 + git fast-import && 392 + git cat-file commit refs/heads/x509-signing >actual && 393 + grep "^gpgsig" actual && 394 + IMPORTED=$(git rev-parse refs/heads/x509-signing) && 395 + test $X509_COMMIT = $IMPORTED 396 + ) <output 397 + 398 + ' 399 + 400 + test_expect_success GPGSSH 'setup SSH signed commit' ' 401 + 402 + git checkout -b ssh-signing main && 403 + test_config gpg.format ssh && 404 + test_config user.signingkey "${GPGSSH_KEY_PRIMARY}" && 405 + echo "SSH content" >file && 406 + git add file && 407 + git commit -S -m "SSH signed commit" && 408 + SSH_COMMIT=$(git rev-parse HEAD) && 409 + git checkout main 410 + 411 + ' 412 + 413 + test_expect_success GPGSSH 'round-trip SSH signed commit' ' 414 + 415 + git fast-export --signed-commits=verbatim ssh-signing >output && 416 + test_grep -E "^gpgsig $GIT_DEFAULT_HASH ssh" output && 417 + ( 418 + cd new && 419 + git fast-import && 420 + git cat-file commit refs/heads/ssh-signing >actual && 421 + grep "^gpgsig" actual && 422 + IMPORTED=$(git rev-parse refs/heads/ssh-signing) && 423 + test $SSH_COMMIT = $IMPORTED 424 + ) <output 369 425 370 426 ' 371 427 ··· 903 959 # fix up lines which mention the ref for comparison 904 960 sed s/--dashes/nodash/ <actual.raw >actual && 905 961 test_cmp expect actual 962 + ' 963 + 964 + test_expect_success GPG 'setup a commit with dual signatures on its SHA-1 and SHA-256 formats' ' 965 + # Create a signed SHA-256 commit 966 + git init --object-format=sha256 explicit-sha256 && 967 + git -C explicit-sha256 config extensions.compatObjectFormat sha1 && 968 + git -C explicit-sha256 checkout -b dual-signed && 969 + test_commit -C explicit-sha256 A && 970 + echo B >explicit-sha256/B && 971 + git -C explicit-sha256 add B && 972 + test_tick && 973 + git -C explicit-sha256 commit -S -m "signed" B && 974 + SHA256_B=$(git -C explicit-sha256 rev-parse dual-signed) && 975 + 976 + # Create the corresponding SHA-1 commit 977 + SHA1_B=$(git -C explicit-sha256 rev-parse --output-object-format=sha1 dual-signed) && 978 + 979 + # Check that the resulting SHA-1 commit has both signatures 980 + echo $SHA1_B | git -C explicit-sha256 cat-file --batch >out && 981 + test_grep -E "^gpgsig " out && 982 + test_grep -E "^gpgsig-sha256 " out 983 + ' 984 + 985 + test_expect_success GPG 'export and import of doubly signed commit' ' 986 + git -C explicit-sha256 fast-export --signed-commits=verbatim dual-signed >output && 987 + test_grep -E "^gpgsig sha1 openpgp" output && 988 + test_grep -E "^gpgsig sha256 openpgp" output && 989 + 990 + ( 991 + cd new && 992 + git fast-import && 993 + git cat-file commit refs/heads/dual-signed >actual && 994 + test_grep -E "^gpgsig " actual && 995 + test_grep -E "^gpgsig-sha256 " actual && 996 + IMPORTED=$(git rev-parse refs/heads/dual-signed) && 997 + if test "$GIT_DEFAULT_HASH" = "sha1" 998 + then 999 + test $SHA1_B = $IMPORTED 1000 + else 1001 + test $SHA256_B = $IMPORTED 1002 + fi 1003 + ) <output 906 1004 ' 907 1005 908 1006 test_done