The extension has no automated tests, and CI only runs when a version tag is pushed (publish.yml). A change to the webview, the client, or the language server can break other features without anyone noticing until after release.
Goal: every push to any branch runs the most common checks, so regressions in the main features (verification diagnostics, the webview views, server startup, packaging) show up while the branch is being worked on, not after a merge. main only accepts PRs whose checks pass.
Approach, cheapest first:
- Tooling and CI plumbing: working lint, a PR workflow, required checks.
- Fast unit tests (seconds, no VS Code): webview render functions, webview script, server utilities.
- Integration tests in a real VS Code (minutes): the extension activates, the server starts, and Verify reports the expected diagnostics.
- Wider coverage: oldest supported VS Code version, lifecycle, Windows and macOS.
Full webview UI automation (WebdriverIO / vscode-extension-tester) is left out for now. The unit tests of the webview script cover most of that, at a fraction of the cost and flakiness.
References:
The sub-issues are ordered from easiest to hardest.
The extension has no automated tests, and CI only runs when a version tag is pushed (
publish.yml). A change to the webview, the client, or the language server can break other features without anyone noticing until after release.Goal: every push to any branch runs the most common checks, so regressions in the main features (verification diagnostics, the webview views, server startup, packaging) show up while the branch is being worked on, not after a merge.
mainonly accepts PRs whose checks pass.Approach, cheapest first:
Full webview UI automation (WebdriverIO / vscode-extension-tester) is left out for now. The unit tests of the webview script cover most of that, at a fraction of the cost and flakiness.
References:
The sub-issues are ordered from easiest to hardest.