Skip to content

FIX: treat mixed-case URL schemes as remote URLs in backend media persistence - #4183

Merged
Roman Lutz (romanlutz) merged 4 commits into
microsoft:mainfrom
LuckVd:fix/mixed-case-scheme-media-url
Nov 11, 2026
Merged

Roman Lutz (romanlutz) merged 4 commits into
microsoft:mainfrom
LuckVd:fix/mixed-case-scheme-media-url

Conversation

@LuckVd

@LuckVd LuckVd commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Description

Backend media persistence classified a path-typed media value with a mixed-case URL scheme (e.g. HTTPS://example.com/img.png) as raw base64 instead of a remote URL, because the prefix check in persist_media_value_async was case-sensitive. Downstream, base64.b64decode then either silently replaced the URL with a garbage image file (URL length % 4 == 0) or raised binascii.Error (% 4 == 2), reachable from attack/chat ingestion and converter preview.

Tests

  • Extended the parametrized test_existing_references_are_not_persisted_async with HTTPS:// and Http:// rows, asserting REMOTE_URL origin, no persistence, and original spelling preserved.
  • tests/unit/backend/ full suite: 4023 passed, 4 skipped; the 3 pre-existing failures (uvicorn lifespan/auth, reproduced on a clean tree before this change) are unrelated to this PR.
  • Verified the end-to-end classification with the repo's own fixture pattern (SQLiteMemory(":memory:") + patched CentralMemory): mixed-case URL now returns origin=remote_url, persisted=False and survives persist_message_pieces_async unchanged.

Disclosure: this PR was prepared with AI assistance; the reproducer, root-cause analysis and tests are included for direct verification.

…sistence

A path-typed media value like 'HTTPS://example.com/img.png' fell through
the case-sensitive http(s) prefix check in persist_media_value_async and
was classified as raw base64: the URL was then silently replaced by a
garbage image file (or raised binascii.Error, depending on the URL
length modulo 4). The prefix check now lowercases the scheme, and the
original spelling is preserved, consistent with microsoft#3861 / PR microsoft#3918.
Copilot AI balanced review requested due to automatic review settings November 11, 2026 02:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the mixed-case HTTP(S) media URL handling change.

No open issues 🎉

What changed in this PR

Fixes #4182 by recognizing mixed-case HTTP(S) schemes in backend media persistence, consistent with the earlier serializer fix in #3918. Remote URLs retain their original spelling instead of being treated as raw base64.

Changes:

  • Makes the HTTP(S) prefix check case-insensitive.
  • Adds regression cases for HTTPS:// and Http://, checking remote-URL classification, no persistence, and unchanged spelling.
File Description
tests/​unit/​backend/​test_media_persistence.py Adds mixed-case URL regression cases.
pyrit/​backend/​services/​media_persistence.py Recognizes HTTP(S) schemes case-insensitively.

Review effort: Balanced

@LuckVd

LuckVd commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree

@LuckVd

LuckVd commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Updated branch with upstream main (merge of #4120, #4178 and related commits) to satisfy the up-to-date branch requirement — CI will re-run on the refreshed head. No functional changes to this PR.

RFC 2397's data scheme is case-insensitive like http(s). Previously a
DATA:image/png;base64,... value fell through to the raw-base64 branch and
the whole URI (header included) was passed to b64decode, corrupting the
persisted file or raising binascii.Error - the same failure class this PR
fixes for http(s). Also lowercase the scheme pass-through check in
_resolve_media_url for consistency with the rest of the backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@its-amann

Copy link
Copy Markdown
Contributor

since HTTPS:// urls now pass through untouched, build_filename in pyrit/backend/models/_media.py still checks the scheme case sensitively, so a blob url like HTTPS://acct.blob.core.windows.net/c/image.png?sv=1&sig=x gets a download filename ending in .png?sv=1&sig=x (sas signature included) instead of .png

-    if value and not value.startswith("data:"):
-        source = urlparse(value).path if value.startswith("http") else value
+    if value and not value.lower().startswith("data:"):
+        source = urlparse(value).path if value.lower().startswith(("http://", "https://")) else value

fixed this plus a test in this commit, you can just cherry pick it instead of redoing it: its-amann@56481e2

@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Nov 11, 2026
Merged via the queue into microsoft:main with commit 6c87f81 Nov 11, 2026
61 of 62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: backend media persistence treats mixed-case URL schemes (HTTPS://...) as raw base64 and silently replaces the URL with a garbage image file

4 participants