Image-metadata label is not merged into resolved config (remoteUser/containerUser dropped, lifecycle hooks ignored, etc.) #20

Summary

The devcontainer.metadata image label is parsed but never merged into the resolved config. As a result, fields the spec says must propagate from base-image / feature metadata into runtime behavior are silently dropped. The user-visible symptom that surfaced this is Engine.Exec running as root when the base image's metadata declares remoteUser=vscode, but the underlying gap is broader: the entire metadata-merge subsystem implemented by the reference tooling (@devcontainers/cli, devpod) is missing.

This causes silent divergence from devcontainer exec for the same image + same devcontainer.json.

Reproduction (RemoteUser case)

devcontainer.json with no remoteUser:

{
  "dockerComposeFile": ["docker-compose.yml"],
  "service": "app",
  "workspaceFolder": "/workspaces/dap"
}

Image carries LABEL devcontainer.metadata='[{\"id\":\"ghcr.io/devcontainers/features/common-utils\",\"version\":\"2.x.x\"},{\"remoteUser\":\"vscode\",\"containerUser\":\"vscode\"}]' and Config.User=root.

ws, _ := eng.Up(ctx, args)
res, _ := eng.Exec(ctx, ws, dc.ExecOptions{Cmd: []string{\"whoami\"}})
// Expected: \"vscode\"
// Actual:   \"root\"

devcontainer exec --workspace-folder . whoami against the same image returns vscode.

Root cause — RemoteUser/ContainerUser specifically

1. Parser drops the fields. feature/metadata.go:108-122 (ParseMetadataLabel) declares RemoteUser/ContainerUser on metadataEntry (lines 29-30) but does not copy them into the returned config.FeatureMetadata.
2. Destination has no slot. config.FeatureMetadata (config/resolved.go:176-204) has no RemoteUser/ContainerUser fields — even fixing DAP cutover (M5): replace @devcontainers/cli shell-out in apps/runtime #1 has nowhere to land.
3. No merge into cfg. up.go:614-687 (layerFeatures) parses baseMeta but only uses it for markAlreadyInstalled and re-emit. It never merges RemoteUser/ContainerUser (or anything else) back into cfg.
4. Final no-ID entry is filtered out. feature/metadata.go:109-111 skips entries with empty ID. The example label's second entry (the resolved-config entry) has no ID, so the parser discards exactly the entry that carries remoteUser/containerUser — even after fixing DAP cutover (M5): replace @devcontainers/cli shell-out in apps/runtime #1, this would still drop the canonical resolved-config entry.
5. Engine.Exec ignores effectiveUser. exec.go:63,71-80 uses opts.User verbatim with no fallback to effectiveUser(ws.Config). Lifecycle (lifecycle.go:166) and userEnvProbe (userenvprobe.go:44,55) both fall back; Exec is the inconsistent caller. (Secondary — once DAP cutover (M5): replace @devcontainers/cli shell-out in apps/runtime #1-Add CHANGELOG.md and start tracking from v0.1.0 #4 are fixed effectiveUser returns the right thing, but the inconsistency should still be removed.)

Broader gap — metadata merge subsystem is missing

devpod's MergeConfiguration (pkg/devcontainer/config/merge.go:35-58) reduces the full metadata chain (base-image label entries → each feature → user devcontainer.json) into the resolved config with last-write-wins semantics. The equivalent step does not exist in this library. Consequences:

• metadataEntry (feature/metadata.go:19-32) only declares 11 fields. JSON keys not declared are silently discarded on parse. Spec-mergeable fields currently dropped at parse time:
  • userEnvProbe, remoteEnv, updateRemoteUserUID
  • waitFor, shutdownAction, overrideCommand
  • hostRequirements
  • forwardPorts, portsAttributes, otherPortsAttributes
  • mounts
  • lifecycle hooks: onCreateCommand, updateContentCommand, postCreateCommand, postStartCommand, postAttachCommand
  • name
• FeatureMetadata (config/resolved.go:176-204) lacks corresponding fields for most of the above. (It does have lifecycle-hook fields, but ParseMetadataLabel doesn't read them and up.go doesn't merge them into cfg.Lifecycle.)
• No reducer step. Even features the user did declare can't contribute their own containerEnv / init / privileged / capAdd / securityOpt / mounts / lifecycle hooks into the final cfg via metadata, only via the ad-hoc paths already wired up.
• Re-emit asymmetry (feature/metadata.go:47-57). Base-image carry-through copies only 8 fields and drops Customizations, RemoteUser, ContainerUser, ResolvedRef, etc. Each rebuild progressively erases metadata even if a future parser learns to read it.
• Final resolved-config entry only writes RemoteUser/ContainerUser (feature/metadata.go:79-82). Per spec it should reflect all user-overrides for downstream consumers reading the label.
• Label is read only when features are layered (up.go:615-617). A compose service pointing at a fully pre-baked image with all features installed plus remoteUser/lifecycle hooks declared in its label gets zero of that metadata applied — layerFeatures returns at line 615-617 before InspectImage. devpod inspects the label unconditionally.

Reference: devpod's merge

// pkg/devcontainer/config/merge.go:47-48
mergedConfig.RemoteUser    = firstString(reversed, func(e *ImageMetadata) string { return e.RemoteUser })
mergedConfig.ContainerUser = firstString(reversed, func(e *ImageMetadata) string { return e.ContainerUser })

reversed + firstString = last-write-wins (user devcontainer.json beats features beats base image label, but if the user doesn't set the field, it falls through). merge.go:35-58 shows the full set of fields that need this treatment.

Suggested fix

Treat as one subsystem rather than one bug:

1. Expand metadataEntry to the full mergeable spec surface.
2. Expand config.FeatureMetadata (or introduce a sibling MetadataLayer type) so parsed entries have somewhere to land.
3. Stop filtering no-ID entries in ParseMetadataLabel — the resolved-config entry has no ID by design.
4. Introduce a MergeMetadata(cfg, baseLabel, featureMetadata...) reducer, run unconditionally during Engine.Up (move the InspectImage call out of layerFeatures's feature-count guard so it runs for image/build/compose paths regardless of whether features are being layered).
5. Expand buildMetadataLabel carry-through and final-entry to cover the same surface, so re-emit is lossless.
6. Make Engine.Exec fall back to effectiveUser(ws.Config) when opts.User == \"\" for parity with lifecycle/userEnvProbe.

Workaround

Set remoteUser explicitly in the user-side devcontainer.json. Wrong if image authors update the canonical user.

Environment

• github.com/crunchloop/devcontainer@v0.0.0-20260509041931-b14345269602
• Reference behavior: @devcontainers/cli@0.83.x, devpod pkg/devcontainer/config/merge.go