Repository navigation
Optional algorithm experiments (off by default): asymmetric integral RC, and the LoopEval stack - #1
Closed
loopkitdev wants to merge 2 commits into
Closed
loopkitdev wants to merge 2 commits into
loopkitdev wants to merge 2 commits into
Conversation
Adds AlgorithmExperiments, an options struct carried on AlgorithmInput (with a protocol-extension default of .none, so existing conformers are unchanged) and passed to generatePrediction. With .none every experiment code path is skipped and the forecast and dose are standard LoopAlgorithm. The experiments are the leading candidates from LoopEval's closed-loop counterfactual evaluation: - retrospectiveCorrectionDropScale / RiseScale: asymmetric RC gains, standard and integral (C01, asymmetric IRC, with integral RC on) - integralRetrospectiveCorrectionMaxVelocity: optional cap on the IRC correction rate (part of C01 as scored) - postLowRiseCut: within 12 h of a reading below 70 and while glucose is below 180, scale the rising side of RC (C20) - volatilityBand: lower the near-term forecast by recent glucose volatility, tapering to zero by 2 h, so the low guard sees it (C29) - calmHighLicence: a larger automatic-bolus application factor on a calm, carb-free high (C23/C25) - negativeInsulinDamper: the Loop and Learn customization (C37) AlgorithmExperiments.stack(medianVolatility:) builds the LoopEval stack (C30) and .asymmetricIntegralRC the C01 configuration. GlucoseVolatility.sigma5 is the causal volatility estimate the band and the licence key on. Tests: existing fixtures decode with no experiments; explicit .none, unity gains and experiments whose gates are closed reproduce the standard output; one unit test per mechanism.
…nd the stack The experiments will be offered to people who choose to run them and report results; fewer choices pool more people per experiment. The public API is now AlgorithmExperiment, an enum with two cases and no tunable parameters: - asymmetricIntegralRC (LoopEval C01): integral RC switched on by the experiment, falling side x1.5, rising side x0.5, rate capped at 4 mg/dL/min - stack (LoopEval C30): post-low RC rise-cut + volatility band + calm-high licence, keyed to the person's volatility baseline AlgorithmInput gains experiment and volatilityBaseline (both default nil). The component parameters are internal so they cannot be mixed. The negative insulin damper is removed. GlucoseVolatility.baseline(_:) computes the person's median sigma5 from a glucose history; when no baseline is supplied the stack computes it from the input history (needs a day of readings) or runs its post-low cut alone. Glucose is thinned to 5-minute spacing before sigma5: with 1-minute CGM the estimator otherwise sees only increments it ignores and never leaves its floor. Identical to the evaluated estimator on 5-minute data.
|
Superseded by LoopKit#10, which replaces these two experiments with one: guard + runway. |
1 similar comment
Owner
Author
|
Superseded by LoopKit#10, which replaces these two experiments with one: guard + runway. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two optional, off-by-default algorithm experiments, so people who choose to can run the two leading candidates from LoopEval and report results. With no experiment set (the default everywhere), every experiment code path is skipped and LoopAlgorithm produces exactly the standard forecast and dose.
There are deliberately only two choices, with no tunable parameters, so the people who try one pool into large, comparable groups.
The two experiments
AlgorithmInput.experiment: AlgorithmExperiment?(default nil):.asymmetricIntegralRC(LoopEval C01). Integral retrospective correction, switched on by the experiment whatever the integral-RC setting. The falling side of the correction is weighted 1.5× and the rising side 0.5×, and the correction rate is capped at 4 mg/dL per minute. It reacts faster when glucose runs below the forecast and more slowly when it runs above..stack(LoopEval C30). Three parts that act in different situations:The band depth and the licence threshold are keyed to the person's median volatility,
AlgorithmInput.volatilityBaseline.GlucoseVolatility.baseline(_:)computes it from a glucose history; four weeks gave a stable value in LoopEval. If no baseline is supplied, the stack computes one from the input's glucose history when that holds at least a day of readings. Otherwise it runs the post-low rise-cut alone.Glucose is thinned to 5-minute spacing before the volatility estimate. With 1-minute CGM the estimator would otherwise see only increments it ignores and never leave its floor. On 5-minute data this is identical to what was evaluated.
How much to trust them
Both were scored in LoopEval by closed-loop counterfactual replay of real data, against the stock algorithm swept over insulin needs. A validation of that replay on 102 real settings edits and 41 dosing-strategy switches found:
Most of the evaluated benefit of both experiments is on the hypo side: both reduced time below 70 and below 54 in replay, while their time in range was flat (stack) or slightly lower (asymmetric IRC). That benefit is therefore unvalidated. Real-world results on time below 70 and 54 are exactly what these experiments can provide; please treat them as experiments, not as improvements.
Cross-check against LoopEval
One week of one person's real cycles (1,792 decisions) was run through this package and through LoopEval's decision-time replay, each with and without the experiment, on identical inputs. LoopEval runs a fork of LoopAlgorithm with deployed-Loop emulation switches, so the two stock algorithms differ slightly (eventual glucose within 1.2 mg/dL at the median). What is compared is each experiment's change from stock.
.asymmetricIntegralRC.stackOn cycles where the two stock algorithms agree within 2 mg/dL, the asymmetric-IRC change is −5.5 U vs −6.3 U and −5.5 vs −6.0 mg/dL. Most of the all-cycle difference comes from cycles where the stock algorithms already decide differently near target, which changes how much room a candidate has to cut. The stack's per-cycle eventual-glucose correlation is lower because its band does not touch eventual glucose; its dose changes correlate 0.89.
Testing
swift test: 125 tests pass (110 existing, 15 new).Not covered
LoopPredictionInputconvenience overload ofgeneratePredictionstays standard.run(). Callers that use onlygeneratePrediction(…, experiment:)get the retrospective-correction parts but not the band.