Skip to content

Optional algorithm experiments (off by default): asymmetric integral RC, and the LoopEval stack - #1

Closed
loopkitdev wants to merge 2 commits into
mainfrom
experiments/frontier-candidates
Closed

loopkitdev wants to merge 2 commits into
mainfrom
experiments/frontier-candidates

Conversation

@loopkitdev

@loopkitdev loopkitdev commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Adds two optional, off-by-default algorithm experiments, so people who choose to can run the two leading candidates from LoopEval and report results. With no experiment set (the default everywhere), every experiment code path is skipped and LoopAlgorithm produces exactly the standard forecast and dose.

There are deliberately only two choices, with no tunable parameters, so the people who try one pool into large, comparable groups.

The two experiments

AlgorithmInput.experiment: AlgorithmExperiment? (default nil):

.asymmetricIntegralRC (LoopEval C01). Integral retrospective correction, switched on by the experiment whatever the integral-RC setting. The falling side of the correction is weighted 1.5× and the rising side 0.5×, and the correction rate is capped at 4 mg/dL per minute. It reacts faster when glucose runs below the forecast and more slowly when it runs above.

.stack (LoopEval C30). Three parts that act in different situations:

  • Post-low rise-cut. For 12 h after a reading below 70 mg/dL, and while glucose is below 180, the rising side of retrospective correction is weighted 0.3×. The rebound after a low is not treated as a lasting need for insulin.
  • Volatility band. While no carbs are on board, the first two hours of the forecast are lowered in proportion to recent glucose volatility, so the low guard sees it. Eventual glucose is unchanged.
  • Calm-high licence. At 180 mg/dL or above, with volatility at or below the person's own median and no carbs on board, the automatic-bolus application factor is doubled, capped at 1. Automatic-bolus dosing only.

The band depth and the licence threshold are keyed to the person's median volatility, AlgorithmInput.volatilityBaseline. GlucoseVolatility.baseline(_:) computes it from a glucose history; four weeks gave a stable value in LoopEval. If no baseline is supplied, the stack computes one from the input's glucose history when that holds at least a day of readings. Otherwise it runs the post-low rise-cut alone.

Glucose is thinned to 5-minute spacing before the volatility estimate. With 1-minute CGM the estimator would otherwise see only increments it ignores and never leave its floor. On 5-minute data this is identical to what was evaluated.

How much to trust them

Both were scored in LoopEval by closed-loop counterfactual replay of real data, against the stock algorithm swept over insulin needs. A validation of that replay on 102 real settings edits and 41 dosing-strategy switches found:

  • It predicts changes in time in range (need-adjusted coefficient 0.66–0.98) and the direction of changes in mean glucose.
  • It does not predict changes in time below 70 or below 54. Predicted hypo changes are about ten times what the field shows and have the right sign about half the time.

Most of the evaluated benefit of both experiments is on the hypo side: both reduced time below 70 and below 54 in replay, while their time in range was flat (stack) or slightly lower (asymmetric IRC). That benefit is therefore unvalidated. Real-world results on time below 70 and 54 are exactly what these experiments can provide; please treat them as experiments, not as improvements.

Cross-check against LoopEval

One week of one person's real cycles (1,792 decisions) was run through this package and through LoopEval's decision-time replay, each with and without the experiment, on identical inputs. LoopEval runs a fork of LoopAlgorithm with deployed-Loop emulation switches, so the two stock algorithms differ slightly (eventual glucose within 1.2 mg/dL at the median). What is compared is each experiment's change from stock.

effect on eventual glucose, per-cycle correlation same sign insulin change over the week, package vs LoopEval
plain integral RC (reference) 0.997 99.7% +178 U vs +171 U
.asymmetricIntegralRC 0.991 96.4% −7.5 U vs −20.9 U
.stack 0.756 95.5% −33.3 U vs −29.1 U

On cycles where the two stock algorithms agree within 2 mg/dL, the asymmetric-IRC change is −5.5 U vs −6.3 U and −5.5 vs −6.0 mg/dL. Most of the all-cycle difference comes from cycles where the stock algorithms already decide differently near target, which changes how much room a candidate has to cut. The stack's per-cycle eventual-glucose correlation is lower because its band does not touch eventual glucose; its dose changes correlate 0.89.

Testing

  • swift test: 125 tests pass (110 existing, 15 new).
  • Default behaviour: existing fixtures decode with no experiment, and the stack changes nothing when its gates are closed or when it has no baseline and no low.
  • Unit tests: RC gains, IRC rate cap, C01 forcing integral RC, the stack's parameters, post-low gate and its effect on the forecast, σ5 on 5- and 1-minute CGM, the baseline, band shape, licence, fixture round trip.

Not covered

  • The LoopPredictionInput convenience overload of generatePrediction stays standard.
  • The volatility band is applied in run(). Callers that use only generatePrediction(…, experiment:) get the retrospective-correction parts but not the band.

ps2 added 2 commits October 15, 2026 12:35
Adds AlgorithmExperiments, an options struct carried on AlgorithmInput (with a
protocol-extension default of .none, so existing conformers are unchanged) and
passed to generatePrediction. With .none every experiment code path is skipped
and the forecast and dose are standard LoopAlgorithm.

The experiments are the leading candidates from LoopEval's closed-loop
counterfactual evaluation:

- retrospectiveCorrectionDropScale / RiseScale: asymmetric RC gains, standard
  and integral (C01, asymmetric IRC, with integral RC on)
- integralRetrospectiveCorrectionMaxVelocity: optional cap on the IRC
  correction rate (part of C01 as scored)
- postLowRiseCut: within 12 h of a reading below 70 and while glucose is
  below 180, scale the rising side of RC (C20)
- volatilityBand: lower the near-term forecast by recent glucose volatility,
  tapering to zero by 2 h, so the low guard sees it (C29)
- calmHighLicence: a larger automatic-bolus application factor on a calm,
  carb-free high (C23/C25)
- negativeInsulinDamper: the Loop and Learn customization (C37)

AlgorithmExperiments.stack(medianVolatility:) builds the LoopEval stack (C30)
and .asymmetricIntegralRC the C01 configuration. GlucoseVolatility.sigma5 is
the causal volatility estimate the band and the licence key on.

Tests: existing fixtures decode with no experiments; explicit .none, unity
gains and experiments whose gates are closed reproduce the standard output;
one unit test per mechanism.
…nd the stack

The experiments will be offered to people who choose to run them and report
results; fewer choices pool more people per experiment. The public API is now
AlgorithmExperiment, an enum with two cases and no tunable parameters:

- asymmetricIntegralRC (LoopEval C01): integral RC switched on by the
  experiment, falling side x1.5, rising side x0.5, rate capped at 4 mg/dL/min
- stack (LoopEval C30): post-low RC rise-cut + volatility band + calm-high
  licence, keyed to the person's volatility baseline

AlgorithmInput gains experiment and volatilityBaseline (both default nil). The
component parameters are internal so they cannot be mixed. The negative insulin
damper is removed.

GlucoseVolatility.baseline(_:) computes the person's median sigma5 from a
glucose history; when no baseline is supplied the stack computes it from the
input history (needs a day of readings) or runs its post-low cut alone. Glucose
is thinned to 5-minute spacing before sigma5: with 1-minute CGM the estimator
otherwise sees only increments it ignores and never leaves its floor.
Identical to the evaluated estimator on 5-minute data.
@loopkitdev loopkitdev changed the title Optional algorithm experiments (off by default): asymmetric RC, post-low rise-cut, volatility band, calm-high licence, negative insulin damper Optional algorithm experiments (off by default): asymmetric integral RC, and the LoopEval stack Oct 15, 2026
@ps2

ps2 commented Oct 18, 2026

Copy link
Copy Markdown

Superseded by LoopKit#10, which replaces these two experiments with one: guard + runway.

1 similar comment
@loopkitdev

Copy link
Copy Markdown
Owner Author

Superseded by LoopKit#10, which replaces these two experiments with one: guard + runway.

@loopkitdev loopkitdev closed this Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants