fix(sessions): keep inlineData mimeType when encoding or decoding content - #1607
Open
hermanrous wants to merge 1 commit into
Open
hermanrous wants to merge 1 commit into
hermanrous wants to merge 1 commit into
Conversation
…tent SessionUtils.encodeContent and SessionUtils.decodeContent rebuilt each inline blob with Blob.builder().data(...), which produces a Blob carrying only `data` and silently drops `mimeType` and `displayName`. `mimeType` is documented as required on com.google.genai.types.Blob, and this is the persistence path used by VertexAiSessionService (appendEvent -> SessionJsonConverter.convertEventToJson -> encodeContent, and back through fromApiEvent -> decodeContent), so every multimodal event stored to Vertex AI lost its MIME type permanently. The round trip decodeContent(encodeContent(content)) did not return the input either. Reuse the existing blob's builder so the remaining fields are preserved, and add tests covering encode, decode, the round trip and the JSON that is persisted.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
SessionUtils.encodeContentandSessionUtils.decodeContentrebuilt every inline blob from scratch:Blob.builder().data(...)produces aBlobcarrying onlydata, somimeTypeanddisplayNameare silently discarded.com.google.genai.types.BlobdocumentsmimeTypeas "Required. The IANA standard MIME type of the source data.", and this is the live persistence path for Vertex AI sessions:VertexAiSessionService.appendEvent→SessionJsonConverter.convertEventToJson→SessionUtils.encodeContentand back on reload via
fromApiEvent→SessionUtils.decodeContent.So every multimodal event (image / audio / PDF
inlineData) stored to Vertex AI loses its MIME type permanently, anddecodeContent(encodeContent(content))does not return the input either.adk-pythonpreservesinline_data.mime_typeon the same path (model_dump(mode='json'), and_drop_vertex_unsupported_part_fieldsonly popspart_metadata), andCONTRIBUTING.mdnames adk-python as the source of truth for validation.How
Reuse the existing blob's builder instead of constructing a fresh
Blob:The fields that are not being rewritten (
mimeType,displayName) are preserved;displayNamecomes back for free, which is the same class of loss.Backward compatible: the stored JSON simply gains a
mimeTypefield that adk-python already writes, and sessions written before this change still read fine (mimeTypestays absent, exactly as today).Tests
New
SessionUtilsTestcovers encode, decode, the round trip, and the JSON that is actually persisted:encodeContent_inlineData_preservesMimeTypedecodeContent_inlineData_preservesMimeTypedecodeContent_encodeContent_roundTripsInlineDataconvertEventToJson_inlineData_persistsMimeTypeOn unmodified
mainall four fail:With the change applied:
Verified with
mvn -B -pl core -am -Dtest=SessionUtilsTest teston Java 21.Note
I used an AI assistant to draft this change. I reviewed it, ran the build and the tests locally, and verified each claim in this description myself.