Repository navigation
1.5.0: tests by the mcpp test spec, test sessions, tests beside each other - #11
Merged
Merged
Conversation
…each other - The backend knows no test framework (mcpp test spec, docs/specs/test.md): a program's cases are what it lists (B3), one listing of the project kept until it changes; cases run by identifier (P4) whatever framework wrote them, and a program of no protocol runs whole. spec/test-frameworks.json, the framework table, its static scanning and report readers, the setting mcpp.test.macros and the dependency on mtest's static manifest are removed; scan reads where a source starts and nothing else. - A case's outcome carries its checks (held, failed), each failure's source and fatal (O3) and its own output (O8); VS Code shows one message per failed check, a diff where it states both sides, the statement and whether the case went on, the case's own output, and the failed checks on the case's line. - A test runs at once in a lane of its own: two tests of a project, or a test beside a build, do not wait for each other, and cancelling one leaves the other (mcpp releases its build directory before tests run, mcpp#915). mcpp.runtime.concurrency holds only clean and pack. - One set of arguments for every test command (profile, target, mcpp.task.testArgs), so a listing and a run plan one configuration; discovery after events is debounced. - Debugging some cases of a protocol program states the protocol's variables and the selection.
… O9) - Where a project's test framework states a test session (test-session, B6) and its mcpp gives launches (test-launch-records, B5), test/runCases runs the cases through it: the session is started once per project and spoken to in JSON-RPC on its standard streams; a run is a job of its own whose cases arrive as test/caseEvent with their checks, failures and output, and whose end is job/finished. Cancelling the job cancels the run. - A run is known before the session's first notification of it: its answer is taken on the session's reading thread. - The session ends with its input, so it ends with the backend; a lifetime bound by the system is not asked, as on Linux it follows the thread that started it. - The setting mcpp.test.session (auto, off) runs every case through mcpp test when off. - docs/architecture.md states a test run by the mcpp test spec.
… test spec - test/cases answers the project's last listing without `list`, and lists again with it only when the project changed since: the cases shown stay until a listing replaces them, and writing a file builds nothing. - CLion runs some cases of a test by their identifiers whatever framework wrote them: the run configuration names a test and its cases (its framework field is removed), and the gutter places a case's run on the first element of the line its program lists it at, in the file it names. Cases are listed when first shown and after a build or a test, one listing at a time per program. - Neovim lists the cases when a file is read, when the session starts and after a run; writing a file shows the last listing. - protocol.schema.json states test/cases, test/runCases, test/caseResult and test/caseEvent by the mcpp test spec: a case's checks (O3), its output (O8), a failure's source and fatal; source/entries and launch parameters name no framework. - mcpp.lock no longer holds mcpplibs.mtest.manifest, which no target depends on.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mcpp-ide 1.5.0 reads tests by the mcpp test spec (mcpp docs/specs/test.md, SPEC-017 v2.0) and knows no test framework by name. Every component is 1.5.0.
Changes
mcpp test --casesper project. The listing is kept: it is listed again only when the project has changed and an editor asks (when cases are first shown, and after a build or a test). Writing a file shows the last listing and builds nothing. Cases run by identifier (--case, P4). The framework table, the static reading of cases with mtest's manifest, andmcpp.test.macrosare removed.test-sessionand its mcpp statestest-launch-records, a run of some cases goes through the session. The session starts once per project and speaks JSON-RPC 2.0 over its standard streams. A run is registered when its answer is read, before any of its notifications. The session ends when its input ends, so it ends with the backend. The new settingmcpp.test.session(auto,off) chooses between the session andmcpp test.mcpp.runtime.concurrencynow applies only to clean and pack.mcpp.task.testArgs). Discovery is debounced. Debugging selected cases sets the protocol's variables.protocol.schema.jsondescribes the test methods by the spec.Compatibility
Each view depends on a feature mcpp states. With an mcpp that lacks
test-protocol, a program is one case. Withouttest-launch-records, or without a framework that states a session, cases run throughmcpp test. mcpp 2026.11.10.1 releases its build directory before tests run, so tests and builds proceed side by side.Verification
mcpp test, 14 programs and 25 cases. e2e/protocol covers a session over the mock mcpp (a run, its case events with checks and output, the job's end) and the kept listing.mtest serve) on a sample project. The first run took 0.35 s; later runs took 0.07 s, with begin and end events, checks, output and failures. Killing the backend with SIGKILL leaves no session process behind.