Skip to content

hosted-e2e red on every PR: patch.socket.dev 403s bun; new transitive gem patches #1945

Closed

Description

hosted-e2e is required through ci-ok. Since about 2026-11-01 17:12 UTC it has failed on every non-draft PR run. The cause is production drift, not code: main has had no commits since 2026-10-30, and the last green hosted-e2e run was at 15:10 UTC on job 124887673673. Until this is fixed or bypassed, no PR can pass ci-ok or merge.

Evidence

  • Run 41330638545 (agent/fix-bundler-path-tier-resolver) failed on attempts 1 and 2. In both, all 3 internal retries were red.
  • Run 41330161127 (arch-refactor/1941-apply-fixture) failed on attempts 1 and 2. In both, all 3 internal retries were red.
  • bun was 1.4.4 in both the passing 15:10 run and the failing runs, so a bun upgrade is not the cause.

1. bun_hosted_install_proof: patch.socket.dev returns 403 to bun (every try)

bun install v1.4.4 (38afee6c8)
error: GET https://patch.socket.dev/patch/npm/minimist/1.2.2/***/***/minimist-1.2.2.tgz - 403

npm (package-lock and shrinkwrap), pnpm, yarn classic, yarn berry and vlt download the same hosted artifact in the same job and pass. Only bun's request is refused. That points to a CDN or WAF rule on patch.socket.dev keyed on bun's request, such as its User-Agent or headers. Real users of bun install against hosted mode would hit the same 403.

2. gem_bundler_hosted_install_proof: new transitive gem patches (some tries)

Production now publishes patches for actionpack 6.0.3 (f86d5caa-9eb0-4984-9ec7-87d5b0c83ed6) and activesupport 6.0.3 (f6e9cc97-51e4-4d84-b646-4635a781765b). In the failing tries, the rewritten Gemfile wrapped those two transitive gems but not activestorage:

source "https://rubygems.org"
gem "activestorage", "6.0.3"

source "https://patch.socket.dev/patch-registry/gem/78b877c4-…/f86d5caa-…/" do
  gem "actionpack", "6.0.3"
end

source "https://patch.socket.dev/patch-registry/gem/05b1b81c-…/f6e9cc97-…/" do
  gem "activesupport", "6.0.3"
end

In other tries the leg passed. The leg's wired_gem_registry_uuid reads the first patch-registry block in the Gemfile. When activestorage is not wired, it picks up actionpack's UUID instead and fails with a misleading "not in GEM_UUIDS" message.

Unblock now

Set the repository variable HOSTED_E2E_DISABLED=true (Settings -> Secrets and variables -> Actions -> Variables), then re-run the failed jobs. Delete the variable once patch.socket.dev serves bun again.

Follow-ups

  • Production: find out why patch.socket.dev returns 403 to bun for hosted npm artifacts.
  • Production or CLI: find out why a hosted scan sometimes leaves the direct activestorage unwired once its transitive dependencies have patches.
  • Test: match the gem leg's wired UUID to activestorage's own block. A ci-janitor PR follows.

Generated by Claude Code

Activity

  1. mikolalysenko commented on Nov 1, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Triage: CI-only, so priority:p3. Open fix PR: #1946 (gem leg reads activestorage's own registry block). That PR's status says the bun minimist-1.2.2.tgz 403 from patch.socket.dev is still unfixed, so hosted-e2e stays red until the server returns the tarball again or HOSTED_E2E_DISABLED is set. The same red check currently shows on agent PR #1943. Leaving this open for #1946 and the server-side half.


    Generated by Claude Code

  2. mikolalysenko commented on Nov 1, 2026

    @mikolalysenko
    CollaboratorAuthor

    Update: production recovered. A re-run of hosted-e2e on #1946 (run 41333148666, attempt 2, finished 19:12 UTC) ran for real, not bypassed, and passed: 16 passed, 0 failed. Both bun_hosted_install_proof and gem_bundler_hosted_install_proof are green, so HOSTED_E2E_DISABLED is no longer needed. PRs that went red between 17:12 and about 19:00 UTC need their failed jobs re-run.

    Leaving this open so the cause can be recorded: the patch.socket.dev 403 to bun, and activestorage being left unredirected while its transitive gems got patches. Close it if both were a one-off production blip.


    Generated by Claude Code

  3. mikolalysenko commented on Nov 4, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Janitor: closing as completed. The gem half was fixed by #1946 (merged 2026-11-02, 2e3e30ec; the gem leg reads activestorage's own block). The bun 403 from patch.socket.dev was a production blip: hosted-e2e has run for real (HOSTED_E2E_DISABLED unset) and passed on every PR since, most recently on #2021 at 2026-11-04 18:05Z (16 passed, including bun_hosted_install_proof and gem_bundler_hosted_install_proof). No one has commented in the 72h since the owner's last update. Reopen if it comes back.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions