Skip to content

Optional algorithm experiments (off by default): guard + runway, integral RC, glucose-based application factor - #10

Merged
ps2 merged 3 commits into
LoopKit:mainfrom
loopkitdev:experiments/guard-runway
Oct 24, 2026
Merged

ps2 merged 3 commits into
LoopKit:mainfrom
loopkitdev:experiments/guard-runway

Conversation

@ps2

@ps2 ps2 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Adds optional, off-by-default algorithm experiments, AlgorithmExperiment, carried on AlgorithmInput.experiments (a Set, with a protocol-extension default of [], so existing conformers are unchanged). With the empty set, every experiment code path is skipped and the forecast and dose are standard LoopAlgorithm.

Each experiment is an on/off switch with no tunable parameters, so that people who choose the same switches pool into one group whose results can be compared. Experiments combine; each combination is its own group.

Experiment What it switches on
guardAndRunway LoopEval's guard + runway (below)
integralRetrospectiveCorrection Integral retrospective correction. The same as useIntegralRetrospectiveCorrection; either one enables it.
glucoseBasedApplicationFactor Loop's glucose-based partial application for automatic-bolus dosing: the factor rises with current glucose from 0.2 at the lower correction target + 10 mg/dL to 0.8 at 200 mg/dL, replacing automaticBolusApplicationFactor (and, through it, the automatic-bolus cap). Temp-basal and manual recommendations are unchanged.

With integralRetrospectiveCorrection and guardAndRunway together, the post-low rise-cut applies to integral RC's rise side.

guardAndRunway

What it does

It aims to improve time in range for people who enter fewer meals, paired with a lows guard. It was evaluated in LoopEval: closed-loop counterfactual replay of real CGM and insulin data, plus a simulated cohort, scored against the standard algorithm swept over insulin needs.

  • Post-low rise-cut. For 12 h after a reading below 70 mg/dL, and while glucose is below 180, the rising side of retrospective correction is weighted 0.3×. The rebound after a low (rescue carbs, counter-regulation) is not treated as a lasting need for insulin.
  • Volatility band. With no carbs on board, the first two hours of the forecast are lowered in proportion to recent glucose volatility (σ5), so the low guard sees it. Eventual glucose is unchanged.
    • The band's depth is keyed to the person's median σ5. The app can pass it as volatilityBaseline (GlucoseVolatility.baseline(_:); four weeks recommended). Otherwise it is computed from a day or more of glucose history.
    • Without either (a new user, under a day of glucose), the band runs at full depth.
  • Runway reserve. At 150 mg/dL or above with no carbs on board, the algorithm finds the most extra insulin that suspending scheduled basal could still take back, if glucose then fell along the forecast less one σ5 band, without reaching the suspend threshold within 3 h.
    • 70% of that amount (at most 60 mg/dL's worth) is added to automatic dosing, applied as an internal lowering of the correction aim.
    • Calm glucose leaves room for it; volatile glucose leaves little or none.
    • The user's target, the manual bolus recommendation and the suspend threshold are unchanged.

Evaluation

Two LoopEval tools, both scored against standard LoopAlgorithm. Lift is the TIR gained beyond what insulin-needs tuning reaches at the same time below 54. That tuning is the standard algorithm with basal, ISF and carb ratio scaled together, ×0.90 to ×1.10. The 95% CI is bootstrapped over people.

People ΔTIR Δ time < 54 Δ time < 70 Lift [95% CI]
Simulated cohort, 14 days each 896 +0.61 −0.011 −0.06 +0.86 [+0.34, +1.23]
Replay of real CGM and insulin data, 28 days each 189 +0.77 −0.023 −0.12 +0.99 [+0.44, +1.54]
  • Settings: each simulated person and each replayed donor keeps their own deployed Loop configuration: settings, dosing strategy and RC mode.
  • Where the gain comes from: most of it is the runway. On the simulated cohort, the guard alone cuts lows (−0.03) but costs TIR (−0.39).
  • Insulin: daily insulin rises about 0.2 U in the simulation and is unchanged in replay.

Combinations (lift; simulated / replay):

Lift
guardAndRunway +0.86 / +0.99
guardAndRunway + glucoseBasedApplicationFactor +0.83 / +0.79
guardAndRunway + integralRetrospectiveCorrection +0.73 / −1.55
glucoseBasedApplicationFactor alone +0.09 / +0.27
integralRetrospectiveCorrection alone +0.19 / −1.53

In replay, integral RC without the clamp adds time below 54 (+0.39 with guardAndRunway) and cancels the guard's reduction.

Limits:

  • These are simulation and replay results. They have not yet been checked against people running the experiment.
  • Replay ranks TIR changes well, but its time-below-54 changes are not yet validated against field outcomes.
  • The reduced-meal-entry gain this experiment aims at has not been re-measured on the current simulated cohort.

Implementation notes

  • GlucoseVolatility.sigma5 is the causal volatility estimate: an EWMA of 5-minute glucose increments, restarted after gaps. Glucose is thinned to 5-minute spacing so 1-minute CGM works.

  • The runway reads the forecast before the band lowers it.

  • Gaps in a sparse forecast (Loop drops points once nothing changes) are filled every 5 min before the reserve is scanned. A forecast shorter than 3 h gives no runway.

  • Retrospective correction (standard and integral) gains an optional rise-side weight, used only by the post-low rise-cut.

  • The app must pass volatilityBaseline. Loop hands the algorithm 12 h of glucose, too little for the in-package fallback (at least a day), so without it every Loop user would get the band at full depth, never lowered for volatile glucose. Compute it with GlucoseVolatility.baseline(_:) from up to four weeks of stored glucose. It moves slowly, so once a day is often enough.

  • New users. Until there is a day of glucose there is no baseline, and the band runs at full depth. The baseline only ever lowers it, for people whose glucose is volatile (depth min(1, 6.7 / baseline)). Full depth errs toward fewer lows at a little time in range. The app should pass a baseline from whatever history exists once there is a day of it, and the full four weeks after that.

    • Evaluated as full depth for everyone over four weeks, the worst case: simulated cohort lift +0.82 TIR, Δt<54 −0.009 (vs +0.86 / −0.011 with each person's baseline); donor replay lift +0.98, ΔTIR +0.70, Δt<54 −0.028 (vs +0.99 / +0.77 / −0.023). It changes the band only for people whose baseline is above 6.7: 8% of the simulated cohort, 23% of donors.
    • With the band off instead, lows stay at the standard algorithm's level: simulated lift +0.55, Δt<54 +0.004.

Testing

  • Fixtures: existing fixtures decode with no experiment.
  • Closed gates: reproduce the standard output.
  • Unit tests per mechanism, including:
    • the runway's gates, its volatility response and cap;
    • sparse and short forecasts;
    • the manual recommendation staying unchanged.
  • Implementation cross-check against LoopEval's engine, where the candidate was evaluated. On one donor-week (1,792 Loop decisions), each decision was recomputed from its recorded inputs with and without the experiment, in both this package and LoopEval.
    • The experiment-minus-standard dose differences agree: summed over the week, +32.8 U here vs +33.0 U in LoopEval, per-decision correlation 0.95.
    • This checks that the two implementations match. It is not an estimate of extra insulin delivered: decisions are replayed independently, so the added insulin never feeds back into later cycles, which in real use would back off.
    • In the closed-loop simulation, where that feedback happens, daily insulin rises by roughly 0.1–0.3 U.

Adds AlgorithmExperiment, an optional input on AlgorithmInput (protocol-extension
default nil, so existing conformers are unchanged). With nil every experiment code
path is skipped and the forecast and dose are standard LoopAlgorithm.

There is one experiment, with no tunable parameters, so that people who choose to
try it pool into one group. It is LoopEval's current best candidate for time in
range with fewer meal entries, paired with its lows guard:

- Post-low rise-cut (C20): for 12 h after a reading below 70 mg/dL, and while
  glucose is below 180, the rising side of retrospective correction is weighted
  0.3x.
- Volatility band (C29): with no carbs on board, the first two hours of the
  forecast are lowered by recent glucose volatility (sigma5), so the low guard sees
  it. Depth is keyed to the person's median sigma5 (volatilityBaseline, or computed
  from a day or more of glucose history; without it the band is off).
- Runway reserve (C40): at 150 mg/dL or above with no carbs on board, find the
  most extra insulin that suspending scheduled basal could still take back if
  glucose fell along the forecast less one sigma5 band, without reaching the
  suspend threshold within 3 h. 70% of it (at most 60 mg/dL's worth) is added to
  automatic dosing, as an internal lowering of the correction aim. The user's
  target, the manual bolus recommendation and the suspend threshold are unchanged.
  Gaps in a sparse forecast are filled every 5 min before the reserve is scanned.

GlucoseVolatility.sigma5 is the causal volatility estimate (glucose thinned to
5-minute spacing so 1-minute CGM works); GlucoseVolatility.baseline the person's
median.

Tests: existing fixtures decode with no experiment; closed gates reproduce the
standard output; unit tests per mechanism, including runway gates, its volatility
response and cap, sparse forecasts, and that the manual recommendation is
unaffected. Implementation cross-check against LoopEval's engine: on one donor-week
(1792 decisions, each recomputed from its recorded inputs with and without the
experiment) the per-decision dose differences agree, summing to +32.8 U vs +33.0 U
with correlation 0.95. Decisions are replayed independently, so this checks the
implementations match; it is not an estimate of extra insulin delivered.
@loopkitdev
loopkitdev force-pushed the experiments/guard-runway branch from 7bb9ac3 to e5f73da Compare October 18, 2026 06:10
@ps2
ps2 marked this pull request as ready for review October 23, 2026 16:40
…se-based application factor

AlgorithmInput.experiment (one optional value) becomes experiments, a Set<AlgorithmExperiment> that
defaults to empty, so experiments combine: each acts on its own part of the algorithm, and each
combination is its own group when results are pooled. Fixtures encode it as a sorted array of names.

Two experiments join guardAndRunway, both as on/off switches with no parameters:

- integralRetrospectiveCorrection: the same as useIntegralRetrospectiveCorrection; either one enables
  integral RC. With guardAndRunway, the post-low rise-cut applies to integral RC's rise side as before.
- glucoseBasedApplicationFactor: Loop's glucose-based partial application for automatic-bolus dosing.
  The factor rises with current glucose from 0.2 at the lower correction target + 10 mg/dL to 0.8 at
  200 mg/dL (Loop's GlucoseBasedApplicationFactorStrategy, keyed on the target in effect now), and
  replaces automaticBolusApplicationFactor and, through it, the automatic-bolus cap. Temp-basal and
  manual recommendations are unchanged.

Tests: the IRC experiment reproduces the input flag on two fixtures; the factor's ramp; the factor
replaces the constant for automatic bolus only; a combination round-trips through a fixture.
@ps2 ps2 changed the title Optional algorithm experiment (off by default): guard + runway Optional algorithm experiments (off by default): guard + runway, integral RC, glucose-based application factor Oct 23, 2026
A new user has under a day of glucose, so there is no baseline and the band was off: the
experiment added time in range without its lows reduction. The band now runs at full depth
(gain 1) until a baseline exists; the baseline only ever lowers it, for people whose glucose
is volatile. Evaluated in LoopEval as full depth for everyone over four weeks (the worst
case): lift +0.82 vs +0.86 on the simulated cohort and +0.98 vs +0.99 on donor replay, with
lows no higher, against +0.55 / +0.99 with the band off.
@ps2
ps2 merged commit 28e9c95 into LoopKit:main Oct 24, 2026
AndEliRich pushed a commit to AndEliRich/LoopWorkspace that referenced this pull request Oct 24, 2026
…t, welcome screen width

- LoopAlgorithm: optional algorithm experiments (LoopKit/LoopAlgorithm#10), type renamed to ExperimentOption (LoopKit/LoopAlgorithm#11)
- Loop: use LoopAlgorithm's experiments, and add Guard + Runway (LoopKit/Loop#2608)
- LoopOnboarding: limit the welcome screen to a readable width (LoopKit/LoopOnboarding#39)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant