Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/conpty-compat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -297,3 +297,64 @@ jobs:
retention-days: 14
if-no-files-found: warn
path: startup-latency.json

windows-2025-startup-under-load:
# Bounded follow-up for #81 / #144 / #40: does local CPU contention on the
# runner inflate PowerShell start-up enough to reach the fixture deadlines.
runs-on: windows-2025
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: '3.12'
- name: Measure start-up latency under CPU load
shell: pwsh
run: |
@'
import json, multiprocessing, platform, subprocess, time
def burn(deadline):
while time.monotonic() < deadline:
sum(i * i for i in range(20000))
def samples(command, count, timeout):
rows = []
for _ in range(count):
started = time.monotonic()
try:
result = subprocess.run(command, capture_output=True, timeout=timeout)
rows.append({"elapsed": round(time.monotonic() - started, 4), "returncode": result.returncode, "timeout": False})
except subprocess.TimeoutExpired:
rows.append({"elapsed": round(time.monotonic() - started, 4), "returncode": None, "timeout": True})
return rows
def summary(rows):
times = sorted(row["elapsed"] for row in rows)
def at(fraction):
return times[min(len(times) - 1, int(len(times) * fraction))]
return {"count": len(rows), "min": times[0], "p50": at(0.5), "p90": at(0.9), "max": times[-1],
"over_1s": sum(1 for t in times if t >= 1.0),
"over_5s": sum(1 for t in times if t >= 5.0),
"timeouts": sum(1 for row in rows if row["timeout"])}
command = ["powershell.exe", "-NoLogo", "-NoProfile", "-NonInteractive", "-Command", "exit 0"]
deadline = time.monotonic() + 120
workers = [multiprocessing.Process(target=burn, args=(deadline,)) for _ in range(8)]
for worker in workers:
worker.start()
Comment on lines +339 to +341

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard multiprocessing startup before collecting loaded samples

On the configured Windows runner, multiprocessing uses spawn, so each child imports this script and executes the unguarded worker.start() loop again. All eight children consequently exit with a bootstrapping RuntimeError before reaching burn, while the parent still collects samples and successfully writes a report claiming eight active workers. Reproducing the extracted script locally with the spawn start method produced eight such errors and a zero parent exit status. The reported measurements therefore do not establish startup latency under CPU load. Put orchestration behind an if __name__ == "__main__": guard, verify worker readiness/liveness during sampling, and rerun the evidence before retaining the loaded-run conclusions.

AGENTS.md reference: AGENTS.md:L12-L12

Useful? React with 👍 / 👎.

time.sleep(2)
loaded = samples(command, 20, 15)
for worker in workers:
worker.terminate()
report = {"schema": "termforge-startup-under-load-v1", "platform": platform.platform(),
"workers": 8, "loaded": summary(loaded), "samples": loaded}
print(json.dumps({k: v for k, v in report.items() if k != "samples"}))
with open("startup-under-load.json", "w", encoding="utf-8") as stream:
json.dump(report, stream, indent=2)
'@ | Out-File -Encoding utf8 startup-under-load.py
python startup-under-load.py
- name: Preserve latency evidence
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: startup-under-load-r2
retention-days: 14
if-no-files-found: warn
path: startup-under-load.json
12 changes: 11 additions & 1 deletion research/ui/results/2026-10-30-hosted-startup-latency/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,19 @@ dispatch-only 作业 `windows-2025-startup-latency`([工作流](../../../../.g
- 因此 `ps51` 在 CI 里出现 **10219 ms** 的 `timed_out`(#81)不能归因于启动成本;Node 夹具的 10 秒超时(#144)同理;
- 三处(ps51 10.2 s、纯 PowerShell 12 s、Node 10 s)都属于"纯启动/纯脚本、几乎无输出、刚好越过期限",与本次测得的正常分布相差 1–2 个数量级,指向**宿主级停顿**(安全扫描、VM 调度、存储阻塞)。

## 附加:本地 CPU 负载下的启动耗时

为区分「启动成本」与「宿主停顿」,另跑一次 `windows-2025-startup-under-load`(8 个 CPU 忙循环同时运行,测 20 次 PowerShell 5.1 启动):

| 项 | 次数 | min | p50 | p90 | max | ≥1s | ≥5s | 超时 |
|---|---|---|---|---|---|---|---|---|
| PowerShell 5.1(8 路 CPU 负载) | 20 | 0.156 s | 0.172 s | 0.172 s | 1.921 s | 1 | 0 | 0 |

对照空闲时的 p50 0.110 s:负载只把中位数抬到 0.172 s,最慢 1.921 s,**仍与 10 秒期限相差一个数量级以上**。

## 边界

- 只测空闲镜像;未在完整 CTest 同时段、安全软件扫描中或并行负载下测量;
- 未修改任何期限、夹具或准备流程;
- 不能据此断定停顿的具体来源,只排除"启动成本"这一解释;
- 与 #144 的 Node 夹具计时(warm 63 ms、冷启动 1.47 s)互相印证。
- 与 #144 的 Node 夹具计时(warm 63 ms、冷启动 1.47 s)互相印证。
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,27 @@
"timeouts": 0
}
},
"interpretation": "Pure start-up cost is 1-2 orders of magnitude below the fixed fixture deadlines, so the observed 10-12 s timeouts cannot be explained by start-up cost.",
"interpretation": "Pure start-up cost is 1-2 orders of magnitude below the fixed fixture deadlines, so the observed 10-12 s timeouts cannot be explained by start-up cost. Local CPU contention raises the median only to 0.172 s, so it does not explain the 10 s stalls either.",
"limitations": [
"idle image only; not under full CTest concurrency or AV scanning",
"no deadline or fixture change",
"does not identify the stall source, only excludes start-up cost"
"does not identify the stall source, only excludes start-up cost",
"loaded run uses eight busy loops, not the full CTest or an AV scan"
],
"performance_claim": false
"performance_claim": false,
"loaded_run": {
"run": 40945977134,
"job": "windows-2025-startup-under-load",
"workers": 8,
"powershell": {
"count": 20,
"min": 0.156,
"p50": 0.172,
"p90": 0.172,
"max": 1.921,
"over_1s": 1,
"over_5s": 0,
"timeouts": 0
}
}
}