Skip to content

Conversation

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Nov 30, 2024

registry: deprecate APIEndpoint.TrimHostname

This field was added in 19515a7, but looks
to be always set for endpoints used, so we can trim remote names unconditionally.

This option was added for possible future expansion, allowing registry-
mirrors to get the full reference of the image (including domain-name),
for them to host a mirror for multiple upstreams on the same registry.

That approach will unlikely be implemented, and containerd has a different
approach for this, where the reference to the original registry is passed
through a query parameter instead.

The field is unlikely used outside of our codebased, but deprecating it
before removal just in case.

- Description for the changelog

registry: deprecate APIEndpoint.TrimHostName; hostname is now trimmed unconditionally for remote names. This field will be removed in the next release.

- A picture of a cute animal (not mandatory but encouraged)

@thaJeztah thaJeztah added this to the 28.0.0 milestone Nov 30, 2024
@thaJeztah thaJeztah self-assigned this Nov 30, 2024
@thaJeztah thaJeztah changed the title registry: deprecate APIEndpoint.TrimName registry: deprecate APIEndpoint.TrimHostname Nov 30, 2024
@thaJeztah thaJeztah force-pushed the registry_deprecate_trimname branch from 18505e1 to 10212e3 Compare November 30, 2024 12:46
@thaJeztah
Copy link
Member Author

Ahm, right, so this was a bit of a gamble; the field was not documented, and it was pretty unclear in which situations trimming was needed. Possibly "pull" and/or "auth" endpoints should not trim, and push endpoints should, but I need to dig a bit deeper. If we cannot remove this field, at least we should document it, and outline under what situations the field should be set (or not).

For future reference; this test is currently failing (I need to look if the test just needs to be adjusted, or if the failure is legit);

=== Failed
=== FAIL: distribution TestPullSchema2Config/success_first_time (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:311: download failed after attempts=1: unknown blob

=== FAIL: distribution TestPullSchema2Config/500_status (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:311: download failed after attempts=1: unknown blob

=== FAIL: distribution TestPullSchema2Config/EOF (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:311: download failed after attempts=1: unknown blob

=== FAIL: distribution TestPullSchema2Config/unauthorized (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:323: expected error="download failed after attempts=1: unknown blob" to contain "unauthorized: you need to be authenticated"

=== FAIL: distribution TestPullSchema2Config/unauthorized_JSON (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:323: expected error="download failed after attempts=1: unknown blob" to contain "unauthorized: you need to be authenticated"

=== FAIL: distribution TestPullSchema2Config/unauthorized_JSON_no_body (0.00s)
    pull_v2_test.go:293: HTTP GET /v2/
    pull_v2_test.go:293: HTTP GET /v2/library/testremotename/blobs/sha256:66ad98165d38f53ee73868f82bd4eed60556ddfee824810a4062c4f777b20a5b
    pull_v2_test.go:323: expected error="download failed after attempts=1: unknown blob" to contain "unauthorized: authentication required"

@thaJeztah thaJeztah force-pushed the registry_deprecate_trimname branch from 10212e3 to bb5ec1f Compare November 30, 2024 13:22
@Zavalynech

This comment was marked as spam.

1 similar comment
@Zavalynech

This comment was marked as spam.

@thaJeztah thaJeztah force-pushed the registry_deprecate_trimname branch from bb5ec1f to a92342a Compare December 2, 2024 11:12
@thaJeztah thaJeztah force-pushed the registry_deprecate_trimname branch 2 times, most recently from ee2938f to 2a63cc8 Compare December 3, 2024 13:13
Comment on lines -298 to +302
case r.Method == "GET" && r.URL.Path == "/v2":
w.WriteHeader(http.StatusOK)
case r.Method == "GET" && r.URL.Path == "/v2/docker.io/library/testremotename/blobs/"+expectedDigest.String():
case r.Method == "GET" && r.URL.Path == "/v2/library/testremotename/blobs/"+expectedDigest.String():
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I could tell, this docker.io/ in the path was purely to match the URLs we constructed in the test (which didn't set TrimHostpath, but in non-test code, it would be set?

@thaJeztah
Copy link
Member Author

Possibly flaky test;

=== RUN   TestDirectRoutingOpenPorts
    port_mapping_linux_test.go:648: ip netns add test-routed-open-ports
    port_mapping_linux_test.go:648: ip netns exec test-routed-open-ports ip link add br0 type bridge
    port_mapping_linux_test.go:648: ip netns exec test-routed-open-ports ip addr add 192.168.124.1/24 dev br0 nodad
    port_mapping_linux_test.go:648: ip netns exec test-routed-open-ports ip link set br0 up
    port_mapping_linux_test.go:648: ip netns exec test-routed-open-ports ip addr add fdc0:36dc:a4dd::1/64 dev br0 nodad
    port_mapping_linux_test.go:648: ip netns exec test-routed-open-ports ip link set br0 up
    port_mapping_linux_test.go:654: ip link add docker netns test-routed-open-ports type veth peer name eth-test
    port_mapping_linux_test.go:654: ip netns exec test-routed-open-ports ip link set docker up master br0
    port_mapping_linux_test.go:654: ip link set eth-test up
    port_mapping_linux_test.go:654: ip addr add 192.168.124.2/24 dev eth-test nodad
    port_mapping_linux_test.go:654: ip addr add fdc0:36dc:a4dd::2/64 dev eth-test nodad
    port_mapping_linux_test.go:658: ip netns add test-remote-host
    port_mapping_linux_test.go:658: ip netns exec test-remote-host ip link add remote netns test-routed-open-ports type veth peer name eth0
    port_mapping_linux_test.go:658: ip netns exec test-routed-open-ports ip link set remote up master br0
    port_mapping_linux_test.go:658: ip netns exec test-remote-host ip link set eth0 up
    port_mapping_linux_test.go:658: ip netns exec test-remote-host ip addr add 192.168.124.3/24 dev eth0 nodad
    port_mapping_linux_test.go:658: ip netns exec test-remote-host ip addr add fdc0:36dc:a4dd::3/64 dev eth0 nodad
    port_mapping_linux_test.go:662: ip netns exec test-remote-host ip route add default via 192.168.124.2
    port_mapping_linux_test.go:663: ip netns exec test-remote-host ip -6 route add default via fdc0:36dc:a4dd::2
    l3_segment_linux.go:91: ip link delete eth-test
    l3_segment_linux.go:91: Cannot find device "eth-test"
        
    l3_segment_linux.go:91: Error: exit status 1
--- FAIL: TestDirectRoutingOpenPorts (27.02s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT
    --- PASS: TestDirectRoutingOpenPorts/ACCEPT (0.00s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v4/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v4/ping (11.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v6/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v6/ping (11.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v4/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/80 (1.02s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat/v6/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/ping (0.00s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/ping (1.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v4/http/81 (0.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/80 (2.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/nat-unprotected/v6/http/81 (2.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v4/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v4/ping (0.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v6/ping
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v6/ping (2.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/80 (0.04s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v4/http/81 (3.05s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/80 (2.08s)
=== RUN   TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/ACCEPT/routed/v6/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP
    --- PASS: TestDirectRoutingOpenPorts/DROP (0.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v4/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v4/ping (11.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v6/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v6/ping (11.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v4/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v4/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v6/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/nat/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat/v6/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/ping (0.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/ping (0.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v4/http/81 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/80 (0.02s)
=== RUN   TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/nat-unprotected/v6/http/81 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v4/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v4/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v4/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v4/ping (0.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v6/ping
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v6/ping
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v6/ping
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v6/ping (0.00s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v4/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v4/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v4/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v4/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v4/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v4/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v4/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v4/http/81 (3.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v6/http/80
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v6/http/80
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v6/http/80
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v6/http/80 (0.01s)
=== RUN   TestDirectRoutingOpenPorts/DROP/routed/v6/http/81
=== PAUSE TestDirectRoutingOpenPorts/DROP/routed/v6/http/81
=== CONT  TestDirectRoutingOpenPorts/DROP/routed/v6/http/81
        --- PASS: TestDirectRoutingOpenPorts/DROP/routed/v6/http/81 (3.01s)

@thaJeztah
Copy link
Member Author

Discussed with @dmcgowan to get more clarity on the purpose of this field; This option was added for possible future expansion, allowing registry-mirrors to get the full reference of the image (including domain-name), for them to host a mirror for multiple upstreams on the same registry.

That approach will unlikely be implemented, and containerd has a different approach for this, where the reference to the original registry is passed through a query parameter instead.

From this, it looks indeed OK to remove this, although he mentioned Notary using some of this, so we need to double-check if anything in docker/cli would depend on it (and/or break when removing this).

This field was added in 19515a7, but looks
to be always set for endpoints used, so we can trim remote names unconditionally.

This option was added for possible future expansion, allowing registry-
mirrors to get the full reference of the image (including domain-name),
for them to host a mirror for multiple upstreams on the same registry.

That approach will unlikely be implemented, and containerd has a different
approach for this, where the reference to the original registry is passed
through a query parameter instead.

The field is unlikely used outside of our codebased, but deprecating it
before removal just in case.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah merged commit d313bb5 into moby:master Dec 9, 2024
140 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants