Repository navigation
Conversation
Reading prefix.> also returns keys written by sources with a nested prefix, which may use another serializer. Skip and log an undecodable multi-token key instead of failing the whole snapshot; such keys were never read before multi-token IDs were supported. Undecodable single-token keys can only belong to this source and still raise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit da0f9c5. Undecodable payloads under the prefix raise again. Sources sharing a bucket with nested prefixes need compatible serializers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I'm withdrawing this implementation because it introduces a compatibility regression for sources sharing a bucket with nested prefixes. Changing prefix.* to prefix.> makes the parent read entries belonging to nested sources. Those entries are decoded with the parent's serializer before the ownership check, so a nested source using an incompatible serializer can now make the parent's get_schedules() fail. The scheduler then substitutes an empty list for that source, dropping its otherwise valid schedules from that refresh. This configuration worked for single-token IDs before the patch. The dotted-ID omission reported in #34 is still a real issue. I'm leaving #34 open for a solution that addresses key ownership and serializer compatibility without breaking existing configurations. Closing this PR without merging; this does not mark #34 as fixed. |
A schedule ID such as
daily.reportis accepted byScheduledTask, stored successfully in NATS KV, and then omitted byget_schedules(). Its currentprefix.*subject filter matches only one token after the prefix, so a scheduler never sees those jobs in its source snapshot.Use
prefix.>to include multi-token IDs. Check each decoded schedule's ID against its stored KV key so a source with a nested configured prefix does not contribute its schedules to the parent source. The stored key format, schedule ID, serializer, and delete/post-send paths are unchanged. This preserves prefix ownership for valid schedules with distinct stored keys; it does not add an atomic snapshot or a new key encoding scheme.Fixes #34. Assignment was requested in the issue; no assignee has been set.
Validation actually run
Environment: Windows / Python 3.14.7, repository lock (Taskiq 0.13.0, nats-py 2.12.0), and a real native NATS Server 2.15.3 with JetStream over a private loopback port. The official server archive was checked against its GitHub release SHA-256 digest. Each test service used its own temporary store and was terminated after validation.
4c15b00d794d85b3dbb0719fd46ab3028eb1352bworktree: 4 failed, 2 passed. The four failures demonstrate missing dotted IDs for cron/time schedules with both single-token and dotted prefixes. The two passing controls cover nested-prefix isolation and malformed-payload error/recovery.pytest -q -n 2: 23 passed, the complete repository suite on the patch. Real connections, KV keys, and JetStream history requests are used; no NATS/Taskiq component is mocked by the new tests.post_send, and sources sharing a bucket with nested prefixes. Malformed own-prefix payloads still raise and recover after deletion.prek run --all-fileshooks (including zizmor), and diff whitespace checks passed. Black emits its existing Python 3.15-target warning under Python 3.14 but exits successfully; nats-py emitsasyncio.iscoroutinefunctiondeprecation warnings.Not run locally: a complete scheduler daemon/worker-process end-to-end test, clustered NATS/failover, older NATS server versions, or the full OS/Python CI matrix. The broader wildcard does not resolve pre-existing key collisions when two prefix/ID combinations encode to the same stored key, and this patch does not change custom bucket history semantics.
Update
A follow-up commit briefly made undecodable multi-token keys be skipped with a warning; it has been reverted. The tree at
49ad0389c97ba928d32e7ec9ef5a598a22d294c5is identical to040343c7edd9a6072f28d1c448163d12aa2f2c97, so the validation above still applies, and undecodable payloads under the prefix raise as described.Known limitation: nested prefixes in one bucket
prefix.>also matches keys written by a source with a nested prefix (for examplescheduleandschedule.tenantsharing a bucket). The parent decodes those entries with its own serializer before the ownership check. If the nested source uses an incompatible serializer, or stores a malformed payload, the parent'sget_schedules()raises. With the previousprefix.*filter, the same configuration worked for single-token IDs.A key such as
schedule.tenant.jobcannot be attributed from its name alone: it may be IDtenant.jobinschedule, or IDjobinschedule.tenant. This patch therefore keeps decoding errors visible instead of skipping them. Sources that share a bucket with nested prefixes need compatible serializers; separate buckets avoid the overlap. I'm happy to switch to skipping and logging such keys if you prefer that trade-off.A probe against a real NATS Server 2.15.3 (Pickle parent, JSON nested source) shows the parent snapshot raising
UnpicklingErroron this branch and returning its own schedule on the base.The follow-up commit and its revert were prepared with Claude (Anthropic) assistance. The PR remains a Draft; no completed human review is claimed.
Review status
AI-assisted investigation, implementation and testing. Please keep this PR in Draft pending independent and account-owner review. No completed human review is claimed.