Repository navigation
feat(skills): fetch skill bundles from Orchestrator - #1996
dianagrecu-uipath wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Archive downloads are fully buffered before size limits are checked, risking worker memory exhaustion, and corrupt ZIP entries are misclassified as source_unavailable instead of invalid.
2 open issues
What changed in this PR
Implements Orchestrator-backed skill bundle fetching behind the existing SDK source interface, enabling coded and low-code agents to resolve remote bundles without changing how they consume skills.
Changes:
- Makes Orchestrator the default source, with bundle lookup, version resolution, archive extraction and error mapping.
- Shares the local copy logic and adds resolved digests to bundle models and telemetry.
- Updates documentation and tests for source selection, remote fetching and archive safety.
| File | Description |
|---|---|
packages/uipath/docs/core/skills.md |
Documents remote fetching, pinned versions and digests. |
packages/uipath/docs/core/environment_variables.md |
Documents Orchestrator as the default source. |
packages/uipath-platform/tests/services/skills/test_telemetry.py |
Tests digest attributes in events and spans. |
packages/uipath-platform/tests/services/skills/test_sources_orchestrator.py |
Tests lookup, versions, failures and archive safety. |
packages/uipath-platform/tests/services/skills/test_get_bundle_source.py |
Updates default-source and fallback tests. |
packages/uipath-platform/src/uipath/platform/skills/skills.py |
Adds an optional resolved bundle digest. |
packages/uipath-platform/src/uipath/platform/skills/_telemetry.py |
Records digests on resolution events and load spans. |
packages/uipath-platform/src/uipath/platform/skills/_sources.py |
Implements Orchestrator fetching and shares copy logic. |
packages/uipath-platform/src/uipath/platform/skills/_skills_service.py |
Updates service docstrings for the new default. |
packages/uipath-platform/src/uipath/platform/skills/_constants.py |
Defines endpoints, lookup limits and archive limits. |
Review effort: Balanced
| {**headers, "Accept": "application/zip"}, | ||
| step="version archive", | ||
| ) | ||
| if len(response.content) > MAX_ARCHIVE_BYTES: |
| except OSError as e: | ||
| raise BundleSourceError(f"cannot write skill bundle '{bundle}': {e}") from e |
| def _unpack_and_copy( | ||
| bundle: str, archive: bytes, dest: Path, stop: threading.Event | ||
| ) -> list[Skill]: | ||
| """Unpack ``archive`` into a temporary folder, then copy its skills into ``dest``.""" | ||
| try: | ||
| with tempfile.TemporaryDirectory(prefix="uipath-skill-bundle-") as tmp: | ||
| staging = Path(tmp) | ||
| _unpack(bundle, archive, staging, stop) | ||
| dest.mkdir(parents=True, exist_ok=True) | ||
| return _copy_bundle_skills(bundle, staging, dest, stop) | ||
| except OSError as e: | ||
| raise BundleSourceError(f"cannot write skill bundle '{bundle}': {e}") from e | ||
|
|
||
|
|
||
| def _unpack(bundle: str, archive: bytes, staging: Path, stop: threading.Event) -> None: | ||
| """Write the archive's files under ``staging``, refusing anything outside it. | ||
|
|
||
| An entry that is not a relative path inside the bundle makes the archive | ||
| ``invalid``. Hidden files and symbolic links are left out. Each file is cut | ||
| at one byte over ``MAX_FILE_BYTES``, so the copy step still sees it as too | ||
| large and skips it with its usual warning; the bytes written in all stay | ||
| within ``MAX_ARCHIVE_BYTES``. | ||
| """ | ||
| try: | ||
| zf = zipfile.ZipFile(io.BytesIO(archive)) | ||
| except zipfile.BadZipFile as e: | ||
| raise BundleSourceError( | ||
| f"skill bundle '{bundle}' archive is not a zip file", reason="invalid" | ||
| ) from e | ||
| with zf: | ||
| infos = zf.infolist() | ||
| if len(infos) > MAX_ARCHIVE_ENTRIES: | ||
| raise BundleSourceError( | ||
| f"skill bundle '{bundle}' has over {MAX_ARCHIVE_ENTRIES} files", | ||
| reason="too_large", | ||
| ) | ||
| root = staging.resolve() | ||
| written = 0 | ||
| for info in infos: | ||
| _check(stop) | ||
| if info.is_dir(): | ||
| continue | ||
| name = info.filename | ||
| parts = name.split("/") | ||
| if "\\" in name or any(part in ("", ".", "..") for part in parts): | ||
| raise BundleSourceError( | ||
| f"skill bundle '{bundle}' archive entry {name!r} is not a " | ||
| "relative path inside the bundle", | ||
| reason="invalid", | ||
| ) | ||
| if not all(is_plain_name(part) for part in parts): | ||
| continue | ||
| if (info.external_attr >> 16) & _FILE_TYPE_MASK == _SYMLINK: | ||
| logger.warning("Skipping symbolic link %s in bundle '%s'", name, bundle) | ||
| continue | ||
| target = (staging / name).resolve() | ||
| if root not in target.parents: | ||
| raise BundleSourceError( | ||
| f"skill bundle '{bundle}' archive entry {name!r} leaves the bundle", | ||
| reason="invalid", | ||
| ) | ||
| target.parent.mkdir(parents=True, exist_ok=True) | ||
| written += _write_capped(zf, info, target, MAX_FILE_BYTES + 1, stop) | ||
| if written > MAX_ARCHIVE_BYTES: | ||
| raise BundleSourceError( | ||
| f"skill bundle '{bundle}' unpacks to over {MAX_ARCHIVE_BYTES} bytes", | ||
| reason="too_large", | ||
| ) |
There was a problem hiding this comment.
A damaged archive is reported as source_unavailable, but the PR's error table says a malformed archive is 'invalid'. Only a non-zip file (at ZipFile()) and OSError are handled. Errors raised while reading an entry (a CRC mismatch gives zipfile.BadZipFile, a broken deflate stream gives zlib.error) are not caught, so _failure_for's catch-all reports them as source_unavailable with the raw error text. An archive with both a file 'a' and a file 'a/SKILL.md' fails at target.parent.mkdir with an OSError and is reported as source_unavailable with 'cannot write skill bundle'. Copilot raised the CRC case too.
| def _reason_for(status: int, error_code: str | None, step: str) -> FailureReason: | ||
| """The ``BundleFailure.reason`` for an HTTP error status.""" | ||
| if status in (401, 403): | ||
| return "forbidden" | ||
| if status == 404: | ||
| return "not_found" if step == "bundle lookup" else "version_not_found" | ||
| if status == 400 and error_code == _FOLDER_NOT_FOUND: | ||
| return "not_found" | ||
| if 400 <= status < 500: | ||
| return "invalid" | ||
| return "source_unavailable" |
There was a problem hiding this comment.
429 (rate limited) and 408 (timeout) are reported as 'invalid' after the SDK's retries run out. BaseService.request_async retries 408/429/502/503/504/524 up to 5 attempts. If they still fail, _reason_for puts every remaining 4xx under 'invalid', which means a broken bundle; a temporary server problem fits 'source_unavailable' better. This follows the PR's own 'Other 4xx → invalid' row; the row is the issue.
| async def archive( | ||
| self, key: uuid.UUID, digest: str, headers: dict[str, str] | ||
| ) -> bytes: | ||
| """The zip of the version with ``digest``; its entries are the manifest paths.""" | ||
| response = await self._get( | ||
| f"{BUNDLE_ENDPOINT.format(key=key)}/versions/{quote(digest, safe='')}/archive", | ||
| {**headers, "Accept": "application/zip"}, | ||
| step="version archive", | ||
| ) | ||
| if len(response.content) > MAX_ARCHIVE_BYTES: | ||
| raise BundleSourceError( | ||
| f"version archive is over {MAX_ARCHIVE_BYTES} bytes", reason="too_large" | ||
| ) | ||
| return response.content |
There was a problem hiding this comment.
The archive is read fully into memory before its size is checked. request_async reads the whole body, and only then does len(response.content) > MAX_ARCHIVE_BYTES run. A body much larger than the manifest says is refused only after it is all in memory. The risk is low: the manifest limit runs first and the server is Orchestrator. Copilot raised this too.
| response = await self._get( | ||
| f"{BUNDLE_ENDPOINT.format(key=key)}/versions/{quote(digest, safe='')}/archive", | ||
| {**headers, "Accept": "application/zip"}, | ||
| step="version archive", | ||
| ) | ||
| if len(response.content) > MAX_ARCHIVE_BYTES: |
There was a problem hiding this comment.
The download limit is checked after the whole archive is buffered. A response larger than 48 MiB can exhaust worker memory before it is rejected.
| with zf.open(info) as source, target.open("wb") as sink: | ||
| while written < limit: | ||
| _check(stop) | ||
| chunk = source.read(min(_READ_CHUNK, limit - written)) |
There was a problem hiding this comment.
Corrupt ZIP entries are reported as source_unavailable instead of invalid. Only errors opening the ZIP are translated; errors while reading its entries escape.
c8701c3 to
3769349
Compare
4662e12 to
46c75df
Compare
🚨 Heads up:
|


Stacked on #1962
The base is
feat/skills-sdk(#1962). Merge #1962 first; this branch is then rebased ontomain.Summary
OrchestratorBundleSourcereplaces the placeholder from #1962, andorchestratorbecomes the defaultUIPATH_SKILLS_SOURCE. Coded and low-code agents now fetch their bound skill bundles from Orchestrator. Nothing above theSkillBundleSourceseam changes:SkillSet, the overrides, the spans, UiPath/uipath-langchain-python#1179 and UiPath/uipath-agents-python#829.How a bundle is fetched:
GET /api/skill-bundles?search=<name>in the bundle's folder (folderPath), or in the job's folder when the reference names none.searchmatches part of a name, so the client keeps the exact match, or a match that differs only in case when there is no exact one. Two matches that differ only in case areinvalid.GET /api/bundles/{key}/versions/{version}.pinnedsends the version as given (a version number, a tag or asha256:digest);latestsendslatest. The version is checked against those forms and URL-encoded before it reaches the path. The manifest gives the version number, the digest and the file sizes, and a version over the download limit is refused here, before any download.GET /api/bundles/{key}/versions/{digest}/archive, by the manifest's digest, so the files match the manifest even whenlatestmoves in between.Archive safety:
.., an absolute path, backslashes, empty segments) makes the bundleinvalid; nothing is written outside the temporary folder.too_largeinstead of filling the disk.Errors, as
BundleFailure.reason:forbiddennot_foundversion_not_foundinvalidsource_unavailableFailure details carry Orchestrator's error message, never request headers.
Other changes:
SkillBundle.digest(optional) holds the resolved version's digest. Theskill.bundle.resolvedevent and theSkill loadspan record it asskill.digest.localstays available for local development and tests. An unknownUIPATH_SKILLS_SOURCEnow falls back to the default (orchestrator) with a warning.skills.mdandenvironment_variables.mddescribe the new default and the forms a pinned version can take.Test plan
packages/uipath-platform: full suite (2305 passed, 12 skipped),ruff check,ruff format --check,mypy --config-file pyproject.toml ..test_sources_orchestrator.py(33 tests, pytest-httpx): lookup and paging, case-only matches, pinned versions, unsafe versions refused before any request, the job-folder fallback, error mapping, the download limit, the digest check, path traversal, symbolic links, hidden and oversized files, the unpack limit, a file that is not a zip.check_version_uniqueness.pyandcheck_dependency_version_bumps.pypass against this PR's range.load_all_asyncresolved a version pinned by number,latestand a version pinned by digest, and reportednot_foundandversion_not_foundfor a missing bundle and a missing version.