Repository navigation
[experiment] fully order independent type checking - #65064
Draft
Jake Bailey (jakebailey) wants to merge 127 commits into
Draft
Jake Bailey (jakebailey) wants to merge 127 commits into
Jake Bailey (jakebailey) wants to merge 127 commits into
Conversation
Checker results should not depend on which files were checked first, or on which types an editor happened to ask for before diagnostics were requested. Tests had no way to show such dependence, so fixes for it could only be described, not demonstrated. The harness-only checkOrder option queries the checker in a different order: files in reverse, each file in its own program, or every expression's type and symbol before any diagnostics. Listing several orders produces a baseline for each, so order dependence shows up as differing baselines.
The checkOrder option only tries a few fixed orders over whole files, but results can depend on which single query a checker answers first, as when an editor asks for the type of one node before anything else is checked. Such dependence is hard to find by hand and easy to reintroduce. With TS_CHECK_ORDER_FUZZ=n, each test is checked again n times in fresh programs that first answer a seeded random selection of type, symbol, and diagnostics queries. Any change in diagnostics or in the type or symbol of a node fails the test with the seed and a minimized list of queries that reproduces it. The fuzzer is off by default, so it does not slow the normal suite.
Asking the checker for types and symbols before requesting diagnostics changes the diagnostics and types reported for these tests. Recording them in that order too shows which later changes make the results independent of what was asked first.
These tests exercise checker results that can depend on the order in which declarations, files, and relations are checked: variance measurement cycles, relation caching under recursion and depth limits, instantiation limits, contextual re-checks, and deferred type reference cycles. Recording the current baselines makes later changes to make checking order independent show up as reviewable baseline diffs.
Variance measured through a cycle of recursive references used to depend on which type in the cycle was measured first, so reordering declarations or assignments changed which assignments were errors. These cases cover cycles through interfaces, type aliases, callbacks, and nested compositions, alongside annotated and structurally measured types whose results should not change.
The errors for a getter and setter that disagree on abstractness or accessibility were reported on both names by whichever accessor was checked first, guarded by a flag on the getter. When that happened during a query rather than a full check, the flag was set without the errors being reported. Since the errors only depend on the modifiers, each accessor now reports the error on its own name with no state, which yields the same set of errors in any order.
The type parameters of a type alias are resolved along with its declared type, but some readers took them from the links without resolving the declared type first, relying on it having been resolved earlier. Reading them through an accessor that resolves the declared type makes the dependency explicit, so the result does not depend on what was resolved before.
checkAssertion runs in several check modes, such as when a call is first checked without its context sensitive arguments, and each run overwrote the operand type kept for the deferred comparability check. That check used whichever was stored last, so its result depended on the order in which the assertion was checked. The deferred check now uses the cached normal type of the operand. The assertion itself only needs the operand type for `as const`, and the operand of a non-const assertion is contextually typed by the asserted type alone, so in any check mode the assertion computes the cached type the deferred check will read, with inference for an enclosing call hidden from the operand. This is the result the deferred check would produce on its own, and checks each operand once rather than again with fresh flow analysis in the deferred check.
Nothing reads or writes it.
The type of a destructured parameter's binding elements depended on
whether the parameter's type happened to be cached already. For
example, in non-strict mode
(({ [a()?.d]: c = "" }) => {})();
reported TS2537 when 'c' was queried before the IIFE call resolved
the parameter (its declared type is the undefined widening type),
but not otherwise. Conversely, in
f7({ a: {} });
function f7({ a: { x = 0, y = 0 } } = { a: {} }) {}
calling f7 first made 'x' and 'y' errors of type any.
Make the choice depend only on the declaration. Parameters of
context-sensitive functions take their type from contextual
assignment, so ask for the parameter's type. Everything else uses
the type derived from the declaration, as it did when nothing was
cached. Asking for the parameter's type unconditionally instead
introduces circularities in patterns whose initializers reference
sibling elements.
Asking for the type of a context-sensitive function's parameter can
resolve the enclosing call, which assigns the parameter its
contextual type from the inference in progress. The first query
nevertheless returned the type it computed itself from the final
signature, while every later query returned the assigned type. In
declare function fun<T>(g: (x: T) => T, h: (y: T) => T, x: T): T;
fun((Math.random() < 0.5 ? (x => x) : (x => undefined)), x => x, 10);
the last 'x' was number when the call was checked first, but any
when 'x' was queried first and its result was cached.
Return the assigned type whenever one exists, so that the first
query agrees with the rest, as the original comment intended.
When resolving a function expression's members reaches the function again through its enclosing call, the contextual check sees an empty signature list and gives up, yet it left the node marked as context checked. Whichever query got there first therefore decided whether the function's parameters were ever contextually typed, and diagnostics such as implicit any errors could be lost depending on check order. Leave the node unchecked in that case so a later request performs the check, and have the deferred function check request it, so that the check is performed at least once whichever query reached it first.
Generic calls nested in the arguments of other generic calls, or in context sensitive functions passed to them, depend on the enclosing call for their contextual type and parameter types, and the enclosing call infers from them in turn. When the inner call was queried first, it was already marked as resolving by the time its contextual type resolved the enclosing call, so the enclosing inference saw a placeholder and both calls cached results that a check in source order never produces. Computing the contextual type of a call nested in an argument of an enclosing call before resolving it resolves the enclosing call first, as source order does. Other contexts are not resolved ahead of time, since computing them may evaluate expressions that checking does not, such as the target of a destructuring assignment.
Asking for the symbol of a property access that hits an index signature stored the index symbol as the access's resolved symbol. Checking a delete expression reads that link to decide whether the operand must be optional, so a prior symbol query turned `delete a.x` on an index signature into an error that a plain check does not report. The type of the index symbol at such an access, which hovers show, is found by comparing it to the symbol the access resolved to, so that comparison now looks the index symbol up again rather than reading it from links.
A context-sensitive function's signature takes the type parameters of
a generic contextual signature, and its parameters their types, when
the function is contextually checked. Anything reading the signature
before then saw it incomplete and cached what it derived from it. Type
aliases in the body read the type parameters as outer type parameters,
so querying such an alias first resolved its conditional types eagerly.
In
declare function f(fun: <T>(t: T) => void): void
f(function self(t) {
const s = self;
type P = Parameters<typeof s>;
});
P was [t: unknown] when the call was checked first, but [t: T] when P
was asked for first, since s then had the uncontextual signature.
Split out assigning the contextual parameter types, which completes
the signature, from the rest of the contextual check, and have every
way to obtain the signature request it: the type of the function's
symbol, before anything resolves its members, and the signature of its
declaration. The rest of the check, inferring the return type and
checking the signature, remains for checking the function itself.
Contextual typing can reach the enclosing call while the signature is
being created, so keep the signature created there rather than replace
the one that was contextually typed. This also makes it unnecessary to
leave a function unchecked when its members are found being resolved.
A single signature type produced by higher order inference is instantiated with the full mapper combined in, since its signature can reference type parameters other than the outer type parameters of its declaration. The instantiation cache was keyed only by the mapped outer type parameters, so two instantiations differing in another type parameter (here the this type) shared whichever result was computed first. Which property access was checked first therefore decided the type of both, and checking `outer.log` gave it the type of `inner.log` contrary to the test's expectation.
Explaining why a name failed to resolve looks up other symbols, for example to find a spelling suggestion, and that can resolve aliases. When the failed name was the target of an alias, the suggestion lookup re-entered the alias being resolved, or skipped it, depending on which aliases were already in progress. In declarationEmitUnknownImport that decided whether two "Circular definition of import alias" errors were reported: they appeared when the import was checked first, and not when the export specifier was resolved first. Defer the explanation until no resolution is in progress, so it only observes finished results. The circularity errors that were artifacts of the suggestion lookup are no longer reported.
When one reference hit the flow analysis depth limit, flow analysis was disabled for every later reference in the same function or module body. The type of a reference then depended on whether another reference in the body had been analyzed before it. Each reference now gives up only when its own analysis hits the limit, so its type is the one it would get if it were analyzed first.
A computed property name stored a placeholder type while its
expression was checked, and returned the placeholder when re-entered.
That kept the cycle off the resolution stack, so the computations in
between did not fail, and the placeholder `{}` escaped into messages
when checking entered through the property name, while `any` did when
it entered through the variable it references. In
blockScopedBindingUsedBeforeDef, the error read "Type '{}' cannot be
used as an index type" or "Type 'any' ..." depending on what was
checked first.
Resolve computed property names on the resolution stack, failing with
the error type on re-entry like other circular resolutions.
Shared flow node results are cached for the duration of one flow
analysis, including incomplete results computed under a loop back
edge. Those incomplete results reflect the partial types of the loop
being analyzed; once that loop's back edge has been exited and the
loop label has a complete type, they are stale. Reusing them made a
reference's type depend on which reference was analyzed first: in
exhaustiveSwitchCheckCircularity, querying the reference inside the
loop before the switch expression produced `never` instead of "bbb".
Remember which flow loop was innermost when a shared result was
computed, and only reuse an incomplete result while that loop is still
being analyzed.
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Compiler 2.270 ± 7% 2.240 ± 5% ~ (p=0.782 n=15)
│ before.txt │ after.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 365.9Mi ± 3% 364.4Mi ± 6% ~ (p=0.683 n=15)
Resolving a call's signature can be requested while checking one of its arguments, through contextual typing, as well as while checking the call itself. Errors that resolution reports at the current node, such as an instantiation depth error, were located at whichever node asked first, so checking an argument ahead of its call moved the error onto the argument. Locating resolution at the call makes the error independent of which was checked first.
isDeeplyNestedType only counts occurrences of a recursion identity that are newer than the previous one, so that nested type arguments, which were created before the types containing them, are not mistaken for expanding recursion. Type ids stand in for "newer", but they follow the order in which types happened to be created, so whether a comparison gives up as deeply nested depended on what had been checked before. A type's generation is one more than that of the types it was built from, which captures the same ordering wherever it follows from how the types are built, and nothing else. With it, both assignments in relationDeeplyNestedCheckOrder report their errors regardless of which is checked first.
Comparisons of types referencing constrained type parameters are considered related while the same comparison with unconstrained type parameters is in progress, since constraints only make more types related. The same goes for a completed comparison that was found to succeed, but its cached result was not consulted, so siblings that differ only in constraints were each compared from scratch. That only went unnoticed because completed comparisons stay in progress until the outermost one completes. Caching comparisons as soon as they are known to hold regardless of their context makes them leave that set early, and without this, relating structurally identical generic interfaces creates ten times the types.
A relation that succeeds while an outer comparison is still in progress
has only been shown to hold under the outer comparison's assumption.
Caching it globally made later lookups depend on which comparison
happened to run first. The same was true for results cut short by depth
limits measured against outer resolutions.
Track the lowest resolution each result depends on and whether it was
truncated. Results are cached globally only when the comparison that
introduced their assumptions completes. Truncated results are cached
only at their own head. Failures remain cached immediately, since
assumptions only ever make a relation more likely to succeed.
│ /tmp/bench-rx-H.txt │ /tmp/bench-rx-GC.txt │
│ sec/op │ sec/op vs base │
Compiler 2.135 ± 8% 2.145 ± 9% ~ (p=0.577 n=10)
│ /tmp/bench-rx-H.txt │ /tmp/bench-rx-GC.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 366.5Mi ± 3% 367.5Mi ± 5% ~ (p=0.853 n=10)
A comparison that only relates by assuming comparisons in progress within it is cached as related once it completes, since nothing it assumed is in progress anymore. But the comparison itself still reported that it might be related, so callers that reject results depending on assumptions accepted it when it was read from the cache, and rejected it when it was computed.
Type parameters that share a symbol (tuple elements, clones made for
signature and mapped type instantiation, distributive and restrictive
copies) all fell through CompareTypes to the type id tie-break. Type
ids reflect creation order, so union ordering and everything derived
from it depended on which code happened to run first. These ties were
nearly all of the id fallbacks seen across the test suite.
Order them by how they were derived instead: the kind of copy, then the
original and the instantiation it was made for. Do the same for fresh
and regular literal types. What remains on the id fallback is mostly
object literal types re-created by repeated checks of one expression,
which need stable ids to order.
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Compiler 2.070 ± 2% 2.045 ± 3% ~ (p=0.670 n=10)
│ before.txt │ after.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 354.0Mi ± 4% 366.7Mi ± 4% ~ (p=0.075 n=10)
Object literal types created from the same expression, or derived
from one by spreading, share a symbol and have no mapper or target,
so CompareTypes fell through to the type id. Which of two such types
was created first depends on check order, so union order (and thus
printed types) depended on it too.
Their members are fixed at creation, so comparing member symbols,
optionality, and member types orders them by content instead. The
baseline changes are unions whose constituents now appear in that
content order.
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Compiler 2.045 ± 3% 2.075 ± 2% ~ (p=0.304 n=10)
│ before.txt │ after.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 366.7Mi ± 4% 365.5Mi ± 5% ~ (p=0.796 n=10)
When an inferred property assignment's widened expression type is identical to the type being printed, the pseudo-type printer reuses the expression and prints the declared property's type instead. That type is widened without the context of its containing object literal, so it can differ from the expression's real widened type: sibling normalization is lost, and an empty array literal prints as undefined[] where the real type is any[]. Only treat the expression as equivalent when the declared property's type matches, so the printed type always agrees with the checker.
Checking the same object literal again (inference passes, overload
resolution, contextual re-checks) creates a new type each time, even
when every member has the same type as before. These copies are
indistinguishable except by type ID, so anything ordered by them
depends on how often and in what order the literal was checked.
Return the existing type when an object literal produces the same
members, so its identity depends only on its content. The changed
baselines print one level less of a recursive this type, and drop a
union member that was a duplicate copy of another.
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Compiler 2.050 ± 4% 2.070 ± 7% +0.98% (p=0.040 n=10)
│ before.txt │ after.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 365.2Mi ± 6% 362.9Mi ± 6% ~ (p=0.481 n=10)
Spreading creates a fresh type and fresh property symbols on every
call, so re-checking an object literal with a spread yields copies
that differ only by ID. Spreads of different inputs also frequently
produce identical members, e.g. when the right side overrides every
property of the left, and unions of such results kept every copy.
Memoize spread results by their inputs, memoize the property symbols
a spread synthesizes by the properties they come from, and intern the
resulting types by member identity. Unions of spread results now
contain each distinct member once.
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Compiler 2.070 ± 7% 2.080 ± 5% ~ (p=0.302 n=10)
│ before.txt │ after.txt │
│ peak-RSS-bytes │ peak-RSS-bytes vs base │
Compiler 362.9Mi ± 6% 368.2Mi ± 5% ~ (p=0.063 n=10)
Every effect a check produces only adds to what the store collects: diagnostics are deduplicated, and reference marks, deferred nodes and emit helper flags are flags or sets. Applying one directly gives the same store whichever check reaches it first, so a check only needs to record effects while results it computes may still be discarded, which is while a cycle or a discard mark is in progress. Running every outermost computation started by a check as a query recorded the effects of all of them, and made every read of a cached result look for effects to account for. Instead, computations in a check keep their frames but apply effects as they happen, and reads account only for work. When a cycle or a discard mark begins inside one, the outermost computation becomes a query from then on, as before, so that effects of discarded results still apply only when a kept result observed them. The computations in progress then are kept, so the effects they applied earlier are right. PERF-PLACEHOLDER
getLateMergedSymbol runs for nearly every symbol lookup, and for most members the answer is that nothing can merge into them. Finding that out took a merged-symbol lookup of the parent and a lookup in the lateBindableKinds memo map on each call. Whether a member's parent can late-bind members depends only on syntax and on the merges finished before checking, so cache it per member in a dense symbol store and only fall back to the full lookup when the parent can late-bind. PERF-PLACEHOLDER
Counting instantiations hashes every instantiated type and its mapper, and the hash caches were maps keyed by pointer, which made the lookups a visible part of checking time. Key type hashes by type ID in a paged store, falling back to a map for types from another store, and symbol hashes in a dense symbol store. Mapper hashes go in a small direct-mapped cache, since a mapper is usually reused right away and a miss only recomputes its hash from the cached type hashes. Also check the type hash cache before hashing the type's symbol, which only the uncached path needs. The hashes themselves are unchanged. PERF-PLACEHOLDER
Entering the computation of a memo map entry looked up the entry's memo state in the side map up to three times. Look it up once, again only after marking a cycle, which can give the entry a state, and skip scanning the keys in progress when there are none. PERF-PLACEHOLDER
A comparison computed the relation key of its source and target in isTypeRelatedTo, again in checkTypeRelatedToEx to begin the comparison's own computation, and again in recursiveTypeRelatedTo for the outermost pair. Hashing the key is not cheap, so pass the key down, and remember the outermost pair's key in the Relater. PERF-PLACEHOLDER
getMergedSymbol is among the most called functions in the checker, and every call looked the symbol up in a map keyed by pointer. The merges done before checking never change afterwards, so keep them in a paged store indexed by symbol ID like other symbol links; looking up a symbol that merged into nothing does not allocate. PERF-PLACEHOLDER
Reading a cached comparison result looks up its memo state as well as the result. Only about one in ten comparisons gets a memo state, but a map lookup that misses costs about as much as one that hits. Relation keys are already hashes, so give memo maps an optional function returning a key's bits, and keep a bit filter of the keys that have a memo state. The relation maps use it to skip looking up states that cannot exist. PERF-PLACEHOLDER
Reading a cached result first asks whether the read must be observed. That test read the checking flag, two store counts, and the current resolution's start on every read, which kept memo.get and other hot readers from inlining. The answer only changes when one of those inputs does, so keep it as a single floor on the instantiation stack length, recomputed wherever an input changes. PERF-PLACEHOLDER
A comparison reads the result cached for its pair before starting, and read it again on starting and once more on first visiting its own pair, all before anything could cache one. Pass the result read first along, and use it for the first visit. PERF-PLACEHOLDER
Pushing a resolution built the whole record and copied it onto the stack, once per computation and once per checked node. Popping already zeroes the record, so extend the stack over the zeroed space and set only the fields needed. PERF-PLACEHOLDER
A memo map held its entries, its memo states, the keys of computations in progress, and the key filter inline: 72 bytes where the map it replaced took 8. Memo maps are embedded in every object type for its instantiations and in several links, and most never have anything but entries, so the extra bytes were most of the growth of object types. Move everything but the entries into a side table allocated on first use, so a memo map takes 16 bytes. Reading an entry checks the one pointer. PERF-PLACEHOLDER
Every result computed or set within a resolution that instantiated anything keeps an instantiation count, 80 bytes for its 64 registers. Most counts are tiny: on VS Code, half of them have at most two nonzero registers and nearly three quarters at most eight. Keep up to eight nonzero registers as a list ordered by index in a 32-byte count, and switch to all 64 registers only past that. The registers and so the estimate are the same as before whatever the order of additions. PERF-PLACEHOLDER
The limit on the number of instantiations was made independent of check order by keeping, for every computation and every cached result, a HyperLogLog estimate of the distinct instantiations needed to compute it from scratch, merged into whatever read it. That bookkeeping was a large share of the extra memory and time over main: an estimate stored on memo states for results set inside other computations, structural hashes for every type instantiated, and a merge on most cached reads. The limit only exists to stop runaway instantiations, which are broken whatever the order, so it is not worth that cost. Count instantiations since the start of the current statement or expression, as before, and accept that whether the limit is reached depends on what was cached before, like the much larger limit on total instantiation depth. The error is reported once, at the node that reached it, since computing the result elsewhere would not reliably reach the limit again. The depth limit stays per computation and independent of order. Without counts, reading a cached result while checking has nothing to account for unless effects or errors are pending, so the check of whether to observe a read no longer depends on the instantiation stack. The instantiation limits test drops the case that showed the count limit reached regardless of shared work. PERF-PLACEHOLDER
Each checked statement, expression and deferred node pushed a resolution of its own, only so that it had its own instantiation count. Now that the count is a plain counter reset at the start of each of them, the node's resolution only added a push and a pop to every check of a node, so reset the counter there instead. The resolution for the current node, which delimits the results that errors located at the node depend on, is unchanged. PERF-PLACEHOLDER
Every comparison began a memo computation keyed by the relation key of its outermost pair, so that results it caches carry the effects of the whole comparison. That meant computing the key and pushing a frame for each one, yet most comparisons settle on quick checks and never reach recursiveTypeRelatedTo, the only place results are cached. Begin the computation, computing the key if the caller did not, on the first call to recursiveTypeRelatedTo instead. Work done before then cannot affect a cached result other than through the types compared, so it belongs with the caller like any other uncached call. The cached entry for the outermost pair is read when the computation begins, since a nested comparison may have cached it in the meantime. PERF-PLACEHOLDER
putRelater reset a relater by assigning a whole new Relater, which the compiler does by building one on the stack and copying it over, for every comparison. The struct has grown with the fields that track the outermost comparison and the depths of the stacks, so the copy is now a noticeable part of each comparison. Reset the fields one at a time, and clear the set of maybe keys only when it has entries. PERF-PLACEHOLDER
TryGet on the node and symbol link stores assigned an id to an entity just to find that it had no links. A node or symbol without an id has no links in any store, so a lookup can stop there instead. This also means that reading links no longer changes anything, which keeps id assignment tied to the writes that need it. PERF-PLACEHOLDER
objectFlagsCell is too large to inline, so every read of a cached object flag, such as whether a type could contain type variables, paid for a call with seven arguments and a closure just to test one bit. Callers now test the bit first, and only call objectFlagsCell when the flag is not yet computed or a query needs to observe the read. PERF-PLACEHOLDER
Every checked node pushes a type resolution, so the chain that resolvingAnywhereIndex and findResolutionCycleStartIndex walk was as deep as the syntax being checked, though neither ever matches the resolution of a current node. A call nested in a few expressions thus walked every enclosing node to find the calls in progress, where main only walked a stack of calls. The searches now walk a second chain that leaves out the resolutions of current nodes. Its link takes four more bytes per resolution, which reordering the fields to drop padding more than pays for. PERF-PLACEHOLDER
resolveStructuredTypeMembers is called for nearly every property, signature or index lookup, and almost always finds the members already resolved. Its fast path computed a memo key before testing the flags, and the function was too large to inline. The fast path is now only the flag test and a cast, which inlines, and everything else moves out of line. PERF-PLACEHOLDER
While checking, nearly every cached computation takes the same path through beginMemo and endMemo: no pending effects to replay, no cycle or discard mark in progress, and on exit no effects recorded, no source to pop and no query begun from a check. That path went through beginCheckedMemo, startEffects, stopEffects and endResolution, each a call that does not inline, around a few field stores. Do those stores directly in beginMemo and endMemo when that case holds, and fall back to the general path otherwise. PERF-PLACEHOLDER
Every resolution carried the current node saved by a query made outside of checking, though only the computation that begins such a query uses it, and queries do not nest: once one begins, its node is being checked until it ends. Keep it in one field of the checker instead, which makes every resolution 16 bytes smaller to touch on entry and zero on exit. PERF-PLACEHOLDER
Every checked expression, statement and deferred node enters and exits a resolution for the current node, some seventeen million times in a large project. Each did so through beginTypeResolution and popTypeResolution, calls that do not inline, though a node's resolution is never searched for cycles and so needs less than either does. Do the work of both directly in enterCurrentNode and exitCurrentNode. PERF-PLACEHOLDER
Each memoized computation looked up its key among the computations in progress and then, finding none, probed the table again to insert it. Probe once for the slot that holds the key or would, and use it for either case. PERF-PLACEHOLDER
Computations whose results are entries of a memo map, rather than cells, still began and ended through the general path, since the fast path in endMemo excluded resolutions with a memo state source and memoMap.begin had none. Give both the same fast path, popping the entry's state from its source in endMemo. PERF-PLACEHOLDER
The keys that intern object literal types and binding pattern types sorted the names of their members, allocating a slice and looking up each member again by name. Both keys include the node the type is created for, whose members are always added in source order, so the order needs no normalizing; key the members as the table lists them. Spread types still sort, since spreads of different inputs can produce the same members in different orders, and those are the same type. PERF-PLACEHOLDER
Memo cells kept a pointer to their memo state inline so that recording effects in checks, which looked up the state of nearly every result read, avoided a map lookup per read. Checks no longer look up the state of what they read unless effects are pending, and the memo state map gained a filter that answers most lookups for results without a state, which is nearly all of them: checking a large project gives about 150 thousand cells a state. The pointer only doubled the size of every cell. Keep the states of cells in the map, behind the filter, as for the entries of memo maps. PERF-PLACEHOLDER
The grammar check of a regular expression literal ran once because of a TypeChecked flag on the node, set by a plain write. Discarding the results of a cycle does not undo such a write, so if the check ran while results were later discarded, it would not run again. Use a memo set instead, like the other checks whose only results are diagnostics, and drop the memo field that guarded nothing.
Its last use was in reportNonexistentProperty, which kept both a set of (name, containing type, JS) keys and the flag on the name's node. The flag already stopped a second report for the node on any containing type, so the keyed set never caught anything the flag did not. Keep a single set of reported name nodes instead. Reports only run while checking, so this stays checking-only state, as the keyed set was.
Jake Bailey (jakebailey)
force-pushed
the
checker-db
branch
from
October 31, 2026 00:13
7eda73a to
1846356
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There's too much here to explain, but at a high level, this is a massive change which aims to make the checker entirely order independent, that is, it doesn't matter when you call the checker, which nodes you walk first, if you do or do not pull something, check some other file, helicopter into the wrong part of a cycle, etc, the behavior, diagnostics, everything, will be the same.
Looking at individual commits is my suggested way to look inside, not the entire diff, which will surely crash your browser.
This is all AI driven, so it's a bit hard to unbox (I plan to break this apart since some stuff can be done early), but some interesting tidbits (handwritten by me!)
There's a new
checkOrdertest option, which makes certain tests variant on interesting orderings, like "what if we visit every single node before checking" or "what if we reverse the order of the files" or even "what if we use PRNG to randomly walk nodes and then check". This up front found on the order of 500 failing tests we already had that were secretly order dependent. These were all fuzzed out and kept.There are a significant number of new tests which all show various ordering bugs never before discovered.
The first chunk of the PR are fixing places where code peeked at caches that "happened to be filled" instead of calling the actual function to pull on them.
Then, all of the ID-using code gets removed. At the end everything now has an ID independent ordering based on content, flags, the works. This caught a bunch of places where we were not caching things and ending up with types that were by all rights identical but had different IDs and then were unstable.
The big and scary change is the introduction of a new state management type dubbed "memo". Every single piece of checker state was moved into memos on a sidecar object on the checker (maybe for future sharing??). This memo system keeps track of the current computation, what depends on what else, eliminating basically all flags/bools that track "computed" or "resolved"-ness. This tracking also records side effects, including diagnostics, "referenced" marks, deferred checks, emit helper checks. The result is that if you query via the API and that calculates some error, those same errors will appear later if the checker runs.
With those data structures in place, now all of the cycles can be handled. Once a cycle is discovered, the checker starts logging what it's doing, and throws all of that info away assuming it's been poisoned.
All of the different recursion stacks get pulled into a single memo stack, which is a cool simplification too.
Variance is now a fixpoint algorithm, proceeding in rounds until stability. In practice, it doesn't iterate much.
Instantiation limits also stop being global numbers (since of course, that is order dependent side effect stuff). Instead, things get limited by the amount of work that's been done, and if you then pull on caches related to work, it tallies that work up to error consistently when needed. (I don't think this is fully done yet, though, since there are depth limits that are really hard to get right. Still thinking about that.)
In some cases, cycles still can't be fixed up, and so they're made consistent by doing what we've done previously for these cases, which is to pick a canonical place to restart to at least get it consistent.
Unfortunately, this all has a cost; the end of the PR tries to claw back perf, but at the end of the day, if we do more work tracking, we spend more CPU and memory tracking. It's still an open question whether this can be cheapened any more. But, my hope is that this idea is minimal enough that we could then optimize TS as a whole assuming this order-independence, e.g. work stealing, checker recycling, shared state, maybe even using LRU caches, or dumping data to disk.
But, it could also well be that this is slop and impossible to manage. But I don't want to sit on it anymore and want to see some real world effects.
Also, this is really meant to be rebased, so expect this all to be nasty nasty force pushes all the time as this gets tweaked. Unfortunately right now that's like 125+ commits, so that sucks.
However, I think that this will fix countless issues on the tracker. I have no idea how to evaluate all 5k issues to see if this PR fixes them. But it's gotta be a lot.