freshlybakedca.ke / git
Git fork
fork atom

t/lib-httpd: refactor "one-time-perl" CGI script to not depend on Perl

Our Apache HTTPD setup exposes an "one_time_perl" endpoint to access
repositories. If used, we execute the "apply-one-time-perl.sh" CGI
script that checks whether we have a "one-time-perl" script. If so, that
script gets executed so that it can munge what would be served. Once
done, the script gets removed so that it doesn't execute a second time.

As the name says, this functionality expects the user to pass a Perl
script. This isn't really necessary though: we can just as easily
implement the same thing with arbitrary scripts.

Refactor the code so that we instead expect an arbitrary script to
exist and rename the functionality to "one-time-script". Adapt callers
to use shell utilities instead of Perl so that we can drop the
PERL_TEST_HELPERS prerequisite.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

authored by

Patrick Steinhardt and committed by
Junio C Hamano cee137b7 de9eeabd

+86 -77
unified split
+1 -1
t/lib-httpd.sh
··· 165 install_script broken-smart-http.sh 166 install_script error-smart-http.sh 167 install_script error.sh 168 - install_script apply-one-time-perl.sh 169 install_script nph-custom-auth.sh 170 171 ln -s "$LIB_HTTPD_MODULE_PATH" "$HTTPD_ROOT_PATH/modules"
··· 165 install_script broken-smart-http.sh 166 install_script error-smart-http.sh 167 install_script error.sh 168 + install_script apply-one-time-script.sh 169 install_script nph-custom-auth.sh 170 171 ln -s "$LIB_HTTPD_MODULE_PATH" "$HTTPD_ROOT_PATH/modules"
+3 -3
t/lib-httpd/apache.conf
··· 135 SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} 136 SetEnv GIT_HTTP_EXPORT_ALL 137 </LocationMatch> 138 - <LocationMatch /one_time_perl/> 139 SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} 140 SetEnv GIT_HTTP_EXPORT_ALL 141 </LocationMatch> ··· 159 ScriptAlias /broken_smart/ broken-smart-http.sh/ 160 ScriptAlias /error_smart/ error-smart-http.sh/ 161 ScriptAlias /error/ error.sh/ 162 - ScriptAliasMatch /one_time_perl/(.*) apply-one-time-perl.sh/$1 163 ScriptAliasMatch /custom_auth/(.*) nph-custom-auth.sh/$1 164 <Directory ${GIT_EXEC_PATH}> 165 Options FollowSymlinks ··· 182 <Files error.sh> 183 Options ExecCGI 184 </Files> 185 - <Files apply-one-time-perl.sh> 186 Options ExecCGI 187 </Files> 188 <Files ${GIT_EXEC_PATH}/git-http-backend>
··· 135 SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} 136 SetEnv GIT_HTTP_EXPORT_ALL 137 </LocationMatch> 138 + <LocationMatch /one_time_script/> 139 SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} 140 SetEnv GIT_HTTP_EXPORT_ALL 141 </LocationMatch> ··· 159 ScriptAlias /broken_smart/ broken-smart-http.sh/ 160 ScriptAlias /error_smart/ error-smart-http.sh/ 161 ScriptAlias /error/ error.sh/ 162 + ScriptAliasMatch /one_time_script/(.*) apply-one-time-script.sh/$1 163 ScriptAliasMatch /custom_auth/(.*) nph-custom-auth.sh/$1 164 <Directory ${GIT_EXEC_PATH}> 165 Options FollowSymlinks ··· 182 <Files error.sh> 183 Options ExecCGI 184 </Files> 185 + <Files apply-one-time-script.sh> 186 Options ExecCGI 187 </Files> 188 <Files ${GIT_EXEC_PATH}/git-http-backend>
-27
t/lib-httpd/apply-one-time-perl.sh
··· 1 - #!/bin/sh 2 - 3 - # If "one-time-perl" exists in $HTTPD_ROOT_PATH, run perl on the HTTP response, 4 - # using the contents of "one-time-perl" as the perl command to be run. If the 5 - # response was modified as a result, delete "one-time-perl" so that subsequent 6 - # HTTP responses are no longer modified. 7 - # 8 - # This can be used to simulate the effects of the repository changing in 9 - # between HTTP request-response pairs. 10 - if test -f one-time-perl 11 - then 12 - LC_ALL=C 13 - export LC_ALL 14 - 15 - "$GIT_EXEC_PATH/git-http-backend" >out 16 - "$PERL_PATH" -pe "$(cat one-time-perl)" out >out_modified 17 - 18 - if cmp -s out out_modified 19 - then 20 - cat out 21 - else 22 - cat out_modified 23 - rm one-time-perl 24 - fi 25 - else 26 - "$GIT_EXEC_PATH/git-http-backend" 27 - fi
···
+26
t/lib-httpd/apply-one-time-script.sh
···
··· 1 + #!/bin/sh 2 + 3 + # If "one-time-script" exists in $HTTPD_ROOT_PATH, run the script on the HTTP 4 + # response. If the response was modified as a result, delete "one-time-script" 5 + # so that subsequent HTTP responses are no longer modified. 6 + # 7 + # This can be used to simulate the effects of the repository changing in 8 + # between HTTP request-response pairs. 9 + if test -f one-time-script 10 + then 11 + LC_ALL=C 12 + export LC_ALL 13 + 14 + "$GIT_EXEC_PATH/git-http-backend" >out 15 + ./one-time-script out >out_modified 16 + 17 + if cmp -s out out_modified 18 + then 19 + cat out 20 + else 21 + cat out_modified 22 + rm one-time-script 23 + fi 24 + else 25 + "$GIT_EXEC_PATH/git-http-backend" 26 + fi
+8 -9
t/t5537-fetch-shallow.sh
··· 256 257 REPO="$HTTPD_DOCUMENT_ROOT_PATH/repo" 258 259 - test_expect_success PERL_TEST_HELPERS 'shallow fetches check connectivity before writing shallow file' ' 260 rm -rf "$REPO" client && 261 262 git init "$REPO" && ··· 271 git -C "$REPO" config protocol.version 2 && 272 git -C client config protocol.version 2 && 273 274 - git -C client fetch --depth=2 "$HTTPD_URL/one_time_perl/repo" main:a_branch && 275 276 # Craft a situation in which the server sends back an unshallow request 277 # with an empty packfile. This is done by refetching with a shorter 278 # depth (to ensure that the packfile is empty), and overwriting the 279 # shallow line in the response with the unshallow line we want. 280 - printf "$(test_oid perl)" \ 281 - "$(git -C "$REPO" rev-parse HEAD)" \ 282 - "$(git -C "$REPO" rev-parse HEAD^)" \ 283 - >"$HTTPD_ROOT_PATH/one-time-perl" && 284 test_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \ 285 - fetch --depth=1 "$HTTPD_URL/one_time_perl/repo" \ 286 main:a_branch && 287 288 - # Ensure that the one-time-perl script was used. 289 - ! test -e "$HTTPD_ROOT_PATH/one-time-perl" && 290 291 # Ensure that the resulting repo is consistent, despite our failure to 292 # fetch.
··· 256 257 REPO="$HTTPD_DOCUMENT_ROOT_PATH/repo" 258 259 + test_expect_success 'shallow fetches check connectivity before writing shallow file' ' 260 rm -rf "$REPO" client && 261 262 git init "$REPO" && ··· 271 git -C "$REPO" config protocol.version 2 && 272 git -C client config protocol.version 2 && 273 274 + git -C client fetch --depth=2 "$HTTPD_URL/one_time_script/repo" main:a_branch && 275 276 # Craft a situation in which the server sends back an unshallow request 277 # with an empty packfile. This is done by refetching with a shorter 278 # depth (to ensure that the packfile is empty), and overwriting the 279 # shallow line in the response with the unshallow line we want. 280 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF && 281 + sed "$(printf "$(test_oid perl)" "$(git -C "$REPO" rev-parse HEAD)" "$(git -C "$REPO" rev-parse HEAD^)")" "\$1" 282 + EOF 283 test_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \ 284 + fetch --depth=1 "$HTTPD_URL/one_time_script/repo" \ 285 main:a_branch && 286 287 + # Ensure that the one-time-script script was used. 288 + ! test -e "$HTTPD_ROOT_PATH/one-time-script" && 289 290 # Ensure that the resulting repo is consistent, despite our failure to 291 # fetch.
+26 -22
t/t5616-partial-clone.sh
··· 737 sed 's/\(..\)/'$1'\1/g' 738 } 739 740 - # Create a one-time-perl command to replace the existing packfile with $1. 741 replace_packfile () { 742 - # The protocol requires that the packfile be sent in sideband 1, hence 743 - # the extra \x01 byte at the beginning. 744 - cp $1 "$HTTPD_ROOT_PATH/one-time-pack" && 745 - echo 'if (/packfile/) { 746 - print; 747 - my $length = -s "one-time-pack"; 748 - printf "%04x\x01", $length + 5; 749 - print `cat one-time-pack` . "0000"; 750 - last 751 - }' >"$HTTPD_ROOT_PATH/one-time-perl" 752 } 753 754 - test_expect_success PERL_TEST_HELPERS 'upon cloning, check that all refs point to objects' ' 755 SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" && 756 rm -rf "$SERVER" repo && 757 test_create_repo "$SERVER" && ··· 776 # section header. 777 test_config -C "$SERVER" protocol.version 2 && 778 test_must_fail git -c protocol.version=2 clone \ 779 - --filter=blob:none $HTTPD_URL/one_time_perl/server repo 2>err && 780 781 test_grep "did not send all necessary objects" err && 782 783 - # Ensure that the one-time-perl script was used. 784 - ! test -e "$HTTPD_ROOT_PATH/one-time-perl" 785 ' 786 787 - test_expect_success PERL_TEST_HELPERS 'when partial cloning, tolerate server not sending target of tag' ' 788 SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" && 789 rm -rf "$SERVER" repo && 790 test_create_repo "$SERVER" && ··· 818 819 # Exercise to make sure it works. 820 git -c protocol.version=2 clone \ 821 - --filter=blob:none $HTTPD_URL/one_time_perl/server repo 2> err && 822 ! grep "missing object referenced by" err && 823 824 - # Ensure that the one-time-perl script was used. 825 - ! test -e "$HTTPD_ROOT_PATH/one-time-perl" 826 ' 827 828 test_expect_success PERL_TEST_HELPERS 'tolerate server sending REF_DELTA against missing promisor objects' ' ··· 845 846 # Clone. The client has deltabase_have but not deltabase_missing. 847 git -c protocol.version=2 clone --no-checkout \ 848 - --filter=blob:none $HTTPD_URL/one_time_perl/server repo && 849 git -C repo hash-object -w -- "$SERVER/have.txt" && 850 851 # Sanity check to ensure that the client does not have ··· 899 grep "want $(cat deltabase_missing)" trace && 900 ! grep "want $(cat deltabase_have)" trace && 901 902 - # Ensure that the one-time-perl script was used. 903 - ! test -e "$HTTPD_ROOT_PATH/one-time-perl" 904 ' 905 906 # DO NOT add non-httpd-specific tests here, because the last part of this
··· 737 sed 's/\(..\)/'$1'\1/g' 738 } 739 740 + # Create a one-time-script command to replace the existing packfile with $1. 741 replace_packfile () { 742 + cp "$1" one-time-pack && 743 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF 744 + if grep packfile "\$1" >/dev/null 745 + then 746 + sed '/packfile/q' "\$1" && 747 + # The protocol requires that the packfile be sent in sideband 748 + # 1, hence the extra \001 byte at the beginning. 749 + printf "%04x\001" \$((\$(wc -c <"$PWD/one-time-pack") + 5)) && 750 + cat "$PWD/one-time-pack" && 751 + printf "0000" 752 + else 753 + cat "\$1" 754 + fi 755 + EOF 756 } 757 758 + test_expect_success 'upon cloning, check that all refs point to objects' ' 759 SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" && 760 rm -rf "$SERVER" repo && 761 test_create_repo "$SERVER" && ··· 780 # section header. 781 test_config -C "$SERVER" protocol.version 2 && 782 test_must_fail git -c protocol.version=2 clone \ 783 + --filter=blob:none $HTTPD_URL/one_time_script/server repo 2>err && 784 785 test_grep "did not send all necessary objects" err && 786 787 + # Ensure that the one-time-script script was used. 788 + ! test -e "$HTTPD_ROOT_PATH/one-time-script" 789 ' 790 791 + test_expect_success 'when partial cloning, tolerate server not sending target of tag' ' 792 SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" && 793 rm -rf "$SERVER" repo && 794 test_create_repo "$SERVER" && ··· 822 823 # Exercise to make sure it works. 824 git -c protocol.version=2 clone \ 825 + --filter=blob:none $HTTPD_URL/one_time_script/server repo 2> err && 826 ! grep "missing object referenced by" err && 827 828 + # Ensure that the one-time-script script was used. 829 + ! test -e "$HTTPD_ROOT_PATH/one-time-script" 830 ' 831 832 test_expect_success PERL_TEST_HELPERS 'tolerate server sending REF_DELTA against missing promisor objects' ' ··· 849 850 # Clone. The client has deltabase_have but not deltabase_missing. 851 git -c protocol.version=2 clone --no-checkout \ 852 + --filter=blob:none $HTTPD_URL/one_time_script/server repo && 853 git -C repo hash-object -w -- "$SERVER/have.txt" && 854 855 # Sanity check to ensure that the client does not have ··· 903 grep "want $(cat deltabase_missing)" trace && 904 ! grep "want $(cat deltabase_have)" trace && 905 906 + # Ensure that the one-time-script script was used. 907 + ! test -e "$HTTPD_ROOT_PATH/one-time-script" 908 ' 909 910 # DO NOT add non-httpd-specific tests here, because the last part of this
+15 -12
t/t5702-protocol-v2.sh
··· 1120 ! grep "git< version 2" log 1121 ' 1122 1123 - test_expect_success PERL_TEST_HELPERS 'when server sends "ready", expect DELIM' ' 1124 rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" http_child && 1125 1126 git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && ··· 1132 1133 # After "ready" in the acknowledgments section, pretend that a FLUSH 1134 # (0000) was sent instead of a DELIM (0001). 1135 - printf "\$ready = 1 if /ready/; \$ready && s/0001/0000/" \ 1136 - >"$HTTPD_ROOT_PATH/one-time-perl" && 1137 1138 test_must_fail git -C http_child -c protocol.version=2 \ 1139 - fetch "$HTTPD_URL/one_time_perl/http_parent" 2> err && 1140 test_grep "expected packfile to be sent after .ready." err 1141 ' 1142 1143 - test_expect_success PERL_TEST_HELPERS 'when server does not send "ready", expect FLUSH' ' 1144 rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" http_child log && 1145 1146 git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && ··· 1157 1158 # After the acknowledgments section, pretend that a DELIM 1159 # (0001) was sent instead of a FLUSH (0000). 1160 - printf "\$ack = 1 if /acknowledgments/; \$ack && s/0000/0001/" \ 1161 - >"$HTTPD_ROOT_PATH/one-time-perl" && 1162 1163 test_must_fail env GIT_TRACE_PACKET="$(pwd)/log" git -C http_child \ 1164 -c protocol.version=2 \ 1165 - fetch "$HTTPD_URL/one_time_perl/http_parent" 2> err && 1166 grep "fetch< .*acknowledgments" log && 1167 ! grep "fetch< .*ready" log && 1168 test_grep "expected no other sections to be sent after no .ready." err ··· 1446 grep "$COMMON" out 1447 ' 1448 1449 - test_expect_success PERL_TEST_HELPERS 'http:// --negotiate-only without wait-for-done support' ' 1450 SERVER="server" && 1451 - URI="$HTTPD_URL/one_time_perl/server" && 1452 1453 setup_negotiate_only "$SERVER" "$URI" && 1454 1455 - echo "s/ wait-for-done/ xxxx-xxx-xxxx/" \ 1456 - >"$HTTPD_ROOT_PATH/one-time-perl" && 1457 1458 test_must_fail git -c protocol.version=2 -C client fetch \ 1459 --no-tags \
··· 1120 ! grep "git< version 2" log 1121 ' 1122 1123 + test_expect_success 'when server sends "ready", expect DELIM' ' 1124 rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" http_child && 1125 1126 git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && ··· 1132 1133 # After "ready" in the acknowledgments section, pretend that a FLUSH 1134 # (0000) was sent instead of a DELIM (0001). 1135 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && 1136 + sed "/ready/{n;s/0001/0000/;}" "$1" 1137 + EOF 1138 1139 test_must_fail git -C http_child -c protocol.version=2 \ 1140 + fetch "$HTTPD_URL/one_time_script/http_parent" 2> err && 1141 test_grep "expected packfile to be sent after .ready." err 1142 ' 1143 1144 + test_expect_success 'when server does not send "ready", expect FLUSH' ' 1145 rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" http_child log && 1146 1147 git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && ··· 1158 1159 # After the acknowledgments section, pretend that a DELIM 1160 # (0001) was sent instead of a FLUSH (0000). 1161 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && 1162 + sed "/acknowledgments/,//{s/0000/0001/;}" "$1" 1163 + EOF 1164 1165 test_must_fail env GIT_TRACE_PACKET="$(pwd)/log" git -C http_child \ 1166 -c protocol.version=2 \ 1167 + fetch "$HTTPD_URL/one_time_script/http_parent" 2> err && 1168 grep "fetch< .*acknowledgments" log && 1169 ! grep "fetch< .*ready" log && 1170 test_grep "expected no other sections to be sent after no .ready." err ··· 1448 grep "$COMMON" out 1449 ' 1450 1451 + test_expect_success 'http:// --negotiate-only without wait-for-done support' ' 1452 SERVER="server" && 1453 + URI="$HTTPD_URL/one_time_script/server" && 1454 1455 setup_negotiate_only "$SERVER" "$URI" && 1456 1457 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && 1458 + sed "s/ wait-for-done/ xxxx-xxx-xxxx/" "$1" 1459 + EOF 1460 1461 test_must_fail git -c protocol.version=2 -C client fetch \ 1462 --no-tags \
+7 -3
t/t5703-upload-pack-ref-in-want.sh
··· 468 test_commit m3 && 469 git tag -d m2 m3 470 ) && 471 - git -C "$LOCAL_PRISTINE" remote set-url origin "http://127.0.0.1:$LIB_HTTPD_PORT/one_time_perl/repo" && 472 git -C "$LOCAL_PRISTINE" config protocol.version 2 473 ' 474 ··· 481 # RPCs during a single negotiation. 482 oid1=$(git -C "$REPO" rev-parse $1) && 483 oid2=$(git -C "$REPO" rev-parse $2) && 484 - echo "s/$oid1/$oid2/" >"$HTTPD_ROOT_PATH/one-time-perl" 485 } 486 487 test_expect_success 'server is initially ahead - no ref in want' ' ··· 533 git -C "$REPO" config uploadpack.allowRefInWant true && 534 rm -rf local && 535 cp -r "$LOCAL_PRISTINE" local && 536 - echo "s/main/rain/" >"$HTTPD_ROOT_PATH/one-time-perl" && 537 test_must_fail git -C local fetch 2>err && 538 539 test_grep "fatal: remote error: unknown ref refs/heads/rain" err
··· 468 test_commit m3 && 469 git tag -d m2 m3 470 ) && 471 + git -C "$LOCAL_PRISTINE" remote set-url origin "http://127.0.0.1:$LIB_HTTPD_PORT/one_time_script/repo" && 472 git -C "$LOCAL_PRISTINE" config protocol.version 2 473 ' 474 ··· 481 # RPCs during a single negotiation. 482 oid1=$(git -C "$REPO" rev-parse $1) && 483 oid2=$(git -C "$REPO" rev-parse $2) && 484 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF 485 + sed "s/$oid1/$oid2/" "\$1" 486 + EOF 487 } 488 489 test_expect_success 'server is initially ahead - no ref in want' ' ··· 535 git -C "$REPO" config uploadpack.allowRefInWant true && 536 rm -rf local && 537 cp -r "$LOCAL_PRISTINE" local && 538 + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && 539 + sed "s/main/rain/" "$1" 540 + EOF 541 test_must_fail git -C local fetch 2>err && 542 543 test_grep "fatal: remote error: unknown ref refs/heads/rain" err