Skip to content

feat(runtime): unlock the installed fastled.exe while running (zccache-style rename trick) so pip can upgrade/uninstall mid-session #163

Closed

Description

@zackees

Goal

On Windows, pip uninstall fastled, pip install -U fastled, and rm -rf .venv fail while a fastled session is still running. The session holds OS file locks on at least three binaries inside the install location: Scripts\fastled.exe (the pip console-script launcher, blocked-waiting on python), python.exe (executing fastled.cli:main), and site-packages\fastled\bin\fastled.exe (the bundled Rust CLI — kept alive because, on Windows, runtime.rs::launch_replacement uses command.status() rather than exec, so the original install exe blocks waiting for the managed child).

The realistic pain case is a global install (e.g. C:\tools\python13\...) being upgraded, not a venv being deleted. There, the only file that must be overwritable is site-packages\fastled\bin\fastled.exe — python.exe and the console-script launcher are not owned by the fastled package and pip install -U fastled does not touch them, so unlocking those is out of scope.

Prior art

zccache crates/zccache/src/daemon/trampoline.rs::unlock_exe (verbatim port of clud crates/clud-bin/src/trampoline.rs) solves the identical problem for zccache.exe / clud.exe:

  1. On startup, if running from the canonical install path, rename own image foo.exe -> foo.exe.old.<pid^nanos> (legal while running on Windows: same-directory rename), then fs::copy the still-running image back to the canonical name. The process keeps executing from the renamed file; the canonical name is now an unlocked plain file that pip can overwrite or delete.
  2. A sweeper try-deletes *.old.* siblings, ignoring failures — the OS lock itself is the liveness check; no PID tracking.
  3. Skip if already under the managed runtime dir (canonicalized compare).
  4. Env opt-out (ZCCACHE_NO_UNLOCK=1).

soldr uses the directory variant (release_worktree.rs, forge-ci/soldr#710): remove_dir_all -> on sharing violation, rename into per-volume ~/.soldr/trash-<volume>/<nanos>-<pid>/ -> sweep on every invocation behind a 5-minute throttle marker.

Gotchas inherited from their issue history: (a) cross-volume rename silently degrades to copy+delete — keep .old in the same directory as the original; (b) inherited stdio handles can keep parents waiting forever (zccache#276, clud#37/#38); (c) a process's cwd handle pins the launch dir; (d) Defender briefly holds locks — sweeps must tolerate per-entry failure and retry on later launches.

Design decisions

  • Where: new unlock_install_exe(&current_exe) called at the top of maybe_reexec_from_managed_runtime (crates/fastled-cli/src/runtime.rs:99), before ensure_runtime_copy. Skip when FASTLED_MANAGED_RUNTIME is set (already the child), when path_is_under(current_exe, run_root) (never trash the managed copy), or when FASTLED_NO_UNLOCK=1. Windows-only via #[cfg(windows)]; no-op on Unix (which exec()s in place and allows unlink of running binaries anyway).
  • Copy-back, not rename-only: keeps the canonical name working for the next invocation and keeps pip semantics (it sees a plain file). Matches zccache.
  • .old.<pid^nanos> stays in the install directory — renaming out to ~/.fastled/trash/ is unsafe on multi-volume setups and adds nothing since we sweep on every launch. After a venv deletion the corpse goes with the tree.
  • Sweep on every launch, synchronous, best-effort — not piggybacked on run_periodic_gc (24h-gated, flock-guarded, too cold).
  • Flock interaction: the dance runs before lock_runtime_root and only touches the install dir, so no contention with ~/.fastled/run/.lock.
  • FASTLED_ORIGINAL_EXE: confirmed write-only today (set at runtime.rs:383, no consumer) — replacing the original exe breaks nothing. If self-update write-back is added later, it must capture the path before the unlock rename.
  • Leave launch_replacement's command.status() as-is — the waiting parent is harmless once its image is unlocked, and detached spawn would change process-tree lifetime semantics the viewer relies on (JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE).
  • Stem derivation: derive .old names from current_exe.file_stem(), not a hardcoded "fastled".

Implementation outline

  1. New module crates/fastled-cli/src/install_unlock.rs: pub fn unlock_install_exe(current_exe: &Path); empty body on non-Windows.
  2. Windows body: check FASTLED_NO_UNLOCK; sweep_old_siblings(parent) (remove every <stem>.exe.old.*, ignore errors); fs::rename(exe, exe.old.<pid^nanos>) — on failure log-and-return, never fatal; fs::copy(old, exe) — on failure attempt rollback rename, return.
  3. Wire into maybe_reexec_from_managed_runtime after the FASTLED_MANAGED_RUNTIME early-return, guarded by not-under-run_root/update_root.
  4. Register module; add FASTLED_NO_UNLOCK const next to the other env constants in runtime.rs.

Test plan

Unit (tempdir-based, runnable in Windows CI):

  • unlock creates .old.* sibling and an unlocked canonical copy with identical bytes
  • sweep removes pre-existing .old.* files
  • FASTLED_NO_UNLOCK=1 is a no-op
  • paths under the simulated run_root are skipped
  • read-only parent dir: returns without panic, canonical exe intact

Manual on Windows:

  1. pip install . then start a long-running fastled session in window A.
  2. Window B: ren <site-packages>\fastled\bin\fastled.exe foo.exe must succeed (fails today).
  3. pip install --force-reinstall fastled must succeed; window A keeps running.
  4. Next fastled launch sweeps the leftover .old.*.

Acceptance criteria

  • While a fastled session is running, site-packages\fastled\bin\fastled.exe can be renamed, deleted, or overwritten (pip reinstall succeeds).
  • The running session is unaffected by the swap and exits with its normal code.
  • <stem>.exe.old.* corpses are swept on subsequent launches; failures to sweep are silent and retried next launch.
  • FASTLED_NO_UNLOCK=1 disables the dance.
  • Unit tests above pass in soldr cargo test --workspace on Windows CI.

Decisions

  • Priority: P2 — workaround exists (close the session first); main value is upgrade-while-running ergonomics.
  • Scope: Rust-side only. Unlocking python.exe / pip's Scripts\fastled.exe and full rm -rf .venv-while-running are explicit non-goals (not package-owned; python.exe pins the venv regardless).
  • Known accepted edge: if Defender holds the freshly copied exe, the first launch after install may fail to unlock; the rename is rolled back and the next launch retries.

Non-goals

  • Unlocking Scripts\fastled.exe or python.exe (not owned by this package).
  • rm -rf .venv during a live session (python.exe still pins the tree).
  • Changing launch_replacement to detached spawn.
  • Self-update write-back via FASTLED_ORIGINAL_EXE (currently unused).

Related

Activity

  1. added a commit that references this issue on Jun 12, 2026
    c69997e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions