Schedule events hours late since Sep 30, missing Oct 2; manual dispatch works

[laporanhariankla-droid]

🏷️ Discussion Type

Bug

💬 Feature/Topic Area

Schedule & Cron Jobs

Discussion Details

Main question

Why did scheduled workflow runs that previously supported delivery around 08:30 WIB (Asia/Jakarta, UTC+07:00) become hours late after September 29, 2026, and stop appearing on October 2, while manual dispatch still works?

Private repository, GitHub Free, main default branch. Repository name, run URLs/IDs, commit hashes, credentials and business data are omitted from this public report. Relevant identifiers can be supplied through a private GitHub channel if needed.

We understand schedule events have no exact-time SLA and can be delayed/dropped under load. This is about the change from a few minutes to several hours or no run creation, not second-perfect cron timing.

Timeline

All timestamps UTC; add seven hours for WIB.

Date	Observed run creation
Sep 29	Sender created 01:34:14; successful delivery 01:34:29–01:34:36 (08:34 WIB).
Sep 30	First preparation run created 03:02:16; configured slots 00:00, 00:30, 01:00. Sender created 06:52:10; configured sender slots 01:25, 01:35. Its hosted job started 06:52:13, three seconds later.
Oct 1	Preparation created 03:08:51. Watchdog created 07:50:11 instead of configured 01:50.
Oct 2	Zero repository-wide schedule run objects as of 08:36 UTC (15:36 WIB), confirmed via REST API. Manual preparation and delivery succeeded today.

Sep 29 and Sep 30 used the same commit, without an intervening code change. The delay is in run creation, not hours waiting for a runner after a run already exists.

Checks

• Workflow files on the default branch match pushed source.
• Private repository, not archived/disabled; Actions enabled; workflows active.
• No queued/in-progress runs remained when checked.
• Manual dispatch starts hosted runners in seconds and succeeds.
• Account is below included Actions-minute allowance in both September and October; no evidence of minute-quota exhaustion.
• API requests return HTTP 200 with ample rate limit.
• Source changes pushed by the repository owner.
• All three production workflows disabled/re-enabled and verified active October 2. This is an attempted recovery, not proof tomorrow is fixed.

Independent diagnostic

A temporary workflow was pushed before today's 02:23, 02:28 and 02:33 UTC slots. One ubuntu-latest job only prints trigger/cron/commit/time. No checkout, dependencies, secrets, external APIs, cache, concurrency or job-level condition.

Manual dispatch started within three seconds and completed in about three seconds. No scheduled run objects appeared by 02:35 UTC; the diagnostic was then disabled. It was newly added, so this is a bounded observation with a short registration window, not proof delayed events could never arrive.

Diagnostic cron: '23,28,33 2 2 10 *'.

Current production UTC schedules:

• Preparation: '7,17,27,37,47,57 0 * * *' and '7,17 1 * * *'.
• Sender: '16,21,26,31,36,41 1 * * *'.
• Watchdog: '50 1 * * *'.

A one-minute sender gap introduced October 1 was removed; current sender slots are five minutes apart. That mistake cannot explain September 30 (old sender slots were ten minutes apart), or the independently delayed preparation/watchdog runs. Documenting this so a configuration issue is not hidden.

The published Actions Job Delays incident October 1 was 14:47–17:56 UTC, outside these morning windows. No causal connection established.

Questions for GitHub / community

Support received the report but closed it because this account has self-service support only. No technical diagnosis supplied.

1. Was schedule-event delivery/registration degraded for some private repositories between Sep 30 and Oct 2?
2. Which checks distinguish a backlog, dropped events, stale registration, or account restriction when no run object exists?
3. Is there a supported way to restore/re-register a previously working schedule, or a private channel where GitHub staff can inspect workflow identifiers?

Related report today: https://github.com/orgs/community/discussions/209331 . Similar symptom on another private repository; context only, not proof of the same root cause.

Our goal is to explain and resolve why the existing morning workflow stopped triggering near its usual time. Thank you.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[Jyotir2004]

GitHub explicitly says schedule events can be delayed during high Actions load, and sufficiently high load can cause queued scheduled jobs to be dropped.
Your key observation is stronger than ordinary runner delay: there is no workflow-run object at all for the missed schedules. That means the failure occurred before a hosted runner could start.
Manual workflow_dispatch working in seconds makes a general Actions execution/account problem less likely.
The Sep 30 sender run eventually appearing at 06:52 UTC after its expected ~01:25/01:35 UTC window is consistent with delayed schedule processing.
The Oct 1 watchdog appearing at 07:50 UTC instead of 01:50 UTC independently shows the issue isn't specific to one production workflow.
The minimal diagnostic workflow also failing to produce scheduled runs is particularly useful evidence because it removes your application code, secrets, dependencies, APIs, checkout, cache, and concurrency from the equation.
One important distinction

I would not describe this publicly as “GitHub dropped my cron jobs” as a confirmed fact.

GitHub's documentation says scheduled events may be delayed or dropped under high load, but your evidence doesn't reveal whether the underlying cause was:

Actions scheduling backlog/load,
delayed schedule-event delivery,
stale/broken schedule registration,
an account/repository-level scheduling restriction,
or another GitHub-side scheduling problem.

So your questions to GitHub are appropriately framed as diagnostic questions rather than accusations.

I would add one more check

There is a potentially relevant GitHub change immediately around your incident: GitHub announced that workflow runs, checks, and statuses began following Actions retention settings from October 1, 2026.

However, that should not explain your primary symptom. Retention affects removal of existing historical run data; it doesn't normally explain why a brand-new scheduled event fails to create a run. I'd mention it only as something GitHub support should rule out, not as the suspected cause.

Also, the current GitHub Actions changelog doesn't show an obvious Oct. 1–2 announcement identifying a broad scheduled-workflow outage; the Actions changelog currently lists an Oct. 1 Actions Runner Controller release and September changes.

Your strongest technical argument

I would emphasize this sentence to GitHub:

The distinguishing symptom is absence of a workflow_run/run object for scheduled invocations, while the identical workflow starts immediately through workflow_dispatch; therefore the failure appears to occur before runner assignment and before workflow-job execution.

That's much stronger than simply saying “cron is late.”

What I'd do next

For the next scheduled window, don't change the cron expressions again. Keep one minimal diagnostic workflow active alongside production.

[Jyotir2004]

One thing I would change: your diagnostic cron

23,28,33 2 2 10 *

is a very narrow registration window. If it is already disabled, don't draw a strong conclusion from its failure. A continuously active diagnostic for 24–48 hours would produce much stronger evidence.

GitHub itself recommends moving scheduled jobs away from common high-load times such as the beginning of an hour. Your production schedules are already mostly offset from :00, which makes the repeated multi-hour/no-run behavior more interesting than a simple “you scheduled exactly on the hour” explanation.

Bottom line: your report is technically well-constructed. The evidence currently supports “scheduled-event creation/delivery problem” much more strongly than “runner problem” or “application-code problem,” but it doesn't yet identify the exact GitHub-side component responsible.