Repository navigation
Conversation
…reliably The 0.2.0 release run (43928665884) failed on all three platforms for two distinct reasons: * Linux built fine but packaging could not find LibTorch: current cargo stores build script output under target/release/build/torch-sys/<hash>/ (nested per crate), so the torch-sys-*/output glob matched nothing and the step failed with "found []". LibTorch had only ever been downloaded inside the torch-sys OUT_DIR, where packaging cannot reach it. * macOS arm64 could not build at all: tch 0.24 -> torch-sys 0.24 has no pre-built LibTorch for Apple Silicon, so its fallback tries to fetch a CPython wheel from PyPI and aborts with "Failed to retrieve torch from pypi. Pre-built version of libtorch for apple silicon are not available." Windows would have hit the same class of failure. Prefetch LibTorch once per platform into target/store/libtorch/<version>/ <device>/libtorch (cpu on Linux/macOS, cu130 on Windows), verify libtorch/build-version against the pinned version, and export LIBTORCH plus LD_LIBRARY_PATH / DYLD_LIBRARY_PATH / PATH so torch-sys never downloads anything itself and build time and packaging see the same libraries. Packaging now resolves the library directory from LIBTORCH first, falls back to the torch-sys build output (both cargo build-dir layouts), and fails with an explicit message when nothing is found. It bundles the LibTorch shared libraries, keeps the patchelf/install_name_tool rpath rewrite, verifies the CUDA runtime is bundled for the cu130 build, checks the loader can resolve the bundled libraries, and rejects archives over the 2 GiB release asset limit. koharu-torch is not restored; packaging still ships the direct tch/koharu libraries only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Release run 43928665884 (tag
0.2.0) failed on all three platforms. Two separate root causes, both LibTorch related.Linux — packaging could not find LibTorch (build succeeded)
5fe0b71dropped the prefetch step and relied ontch's own download. Two problems with that:OUT_DIR(.../build/torch-sys/<hash>/out/libtorch), which the packaging step never looked at.build/torch-sys-<hash>/output; it nests build dirs per crate asbuild/torch-sys/<hash>/output. The same run shows the macOS job using that layout (target/release/build/torch-sys/08dcea37e4771c48/out/build_script_build), so thetorch-sys-*/outputglob matched zero files — hencefound []instead of a per-platform message.macOS arm64 — could not build at all
tch 0.24→torch-sys 0.24publishes no Apple Silicon LibTorch; its fallback path asks PyPI for a CPython wheel and there is none. OnlyLIBTORCHpointing at an extractedlibtorch-macos-arm64-2.11.0.zipavoids that branch.Windows
Still running in the failing run, but the same mechanism applies: the build depended on tch downloading LibTorch itself into its
OUT_DIR, so the packaging step would not have found the DLLs either.What changed (
.github/workflows/release.ymlonly)Prefetch LibTorch on all three platforms. Downloads into
target/store/libtorch/<version>/<device>/libtorch:libtorch-shared-with-deps-2.11.0+cpu.ziplibtorch-macos-arm64-2.11.0.zip(MPS included)libtorch-win-shared-with-deps-2.11.0+cu130.zipLIBTORCH_VERSIONis pinned to 2.11.0, which isTORCH_VERSIONin torch-sys 0.24 — torch-sys compares it againstlibtorch/build-versionand aborts on a mismatch, so the step verifies the archive'sbuild-versionitself and fails with a clear message instead of a build-time surprise. The step is idempotent (re-uses an already extracted tree) and retries transient CDN failures.Exported for the following steps:
LIBTORCH,LD_LIBRARY_PATH(Linux) /DYLD_LIBRARY_PATH(macOS) /PATH(Windows). torch-sys takesLIBTORCHfirst and skips every download path, so build time and packaging use the same libraries.Packaging (
Package runner) resolves LibTorch properly.$LIBTORCH/libfirst — authoritative, same tree the build used.cargo:rustc-link-search=native=,cargo:rustc-link-search=,cargo:libtorch_lib=, scanning bothbuild/torch-sys-*/output(legacy) andbuild/torch-sys/*/outputplus a recursive glob.LIBTORCH, the build outputs scanned, whether the build dir exists, and every candidate checked.It then copies all LibTorch shared libraries (
.so*/.dylib/.dll), keeps thepatchelf --set-rpath $ORIGINandinstall_name_tool -add_rpath @executable_pathhandling (macOS now skips the rpath insert when it is already present), and adds guards: the cu130 build must bundle acudart64_*.dll, the loader must resolve the bundled LibTorch libraries (Linuxldd, macOSotool -Lrejecting leaked/Users/build paths), and the archive is rejected if it reaches GitHub's 2 GiB release-asset limit.macOS runner label is unchanged (
macos-latest), andkoharu-torchis not restored.Verification done here
Local only — no three-platform build was run, so this does not claim native CI success.
actionlintclean;yaml.safe_loadparses; bothshell: pythonblocks compile.build-versionpreserved,LIBTORCH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/GITHUB_PATHwritten, idempotent re-run with zero downloads, mismatchedbuild-versionrejected).download.pytorch.org: 814 MB extracted,build-version = 2.11.0+cpu, all required libraries present.torch-sys 0.24.0+tch 0.24.0build withLIBTORCHpointing at that tree: compiles and links with no PyPI ordownload.pytorch.orgaccess in the log, and the emitted build output starts withcargo:rustc-link-search=native=<prefetched>/lib— the exact prefix the packaging step parses.patchelf, real torch-linked binary) produces an archive whose binary runs from a clean environment with$ORIGINrpath resolvinglibtorch_cpu.so,libc10.soandlibgomp-98b21ff3.so.1out of the package; fallback resolution for both cargo build-dir layouts; explicit failure when nothing is discoverable; macOS and Windows paths (dylib/DLL bundling,@executable_pathrpath, CUDA DLLs next toflint.exe); CUDA bundle withoutcudart64_*.dllrejected; every required library name confirmed to exist in the published archives.Known risk for the next run
The Windows cu130 package carries ~2.66 GB of DLLs uncompressed (≈1.87 GB zipped) because PyTorch's CUDA LibTorch bundles the whole CUDA runtime (cuBLAS, cuDNN, cuFFT, cuSOLVER, NVRTC…). Together with
flint.exeand the koharu libraries it is close to GitHub's 2 GiB per-asset limit — the step now fails with an explicit size message rather than an opaque upload error. If the real binary pushes it over, the follow-up is to split the Windows asset (CUDA runtime as a second archive) rather than to drop CUDA.