Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/r54-cpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Bounded R54 hosted CPU diagnosis

on:
push:
branches: ['perf/grapheme-cpu-hosted-r54', 'perf/grapheme-cpu-hosted-r54-setup2', 'perf/grapheme-cpu-hosted-r54-entry3']

permissions:
contents: read

concurrency:
group: r54-cpu-${{ github.ref }}
cancel-in-progress: false

jobs:
cpu-boundary:
# Evidence or exporter fixes must not silently create replacement observations.
if: github.event.created == true
runs-on: windows-2025
timeout-minutes: 30
env:
PYTHONUTF8: '1'
CARGO_HOME: ${{ github.workspace }}/local/r54-cargo-home
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: '3.12'
- name: Install fixed CPU observer
run: python -m pip install -r benchmarks/requirements.txt
- name: Select fixed compiler
run: rustup default 1.92.0
- name: Verify build, accounting and publication boundaries
run: python -m unittest experiments.measure.test_r54_contract.CostPlanTests experiments.measure.test_r54_batch experiments.measure.test_r54_hosted experiments.measure.test_r54_cpu_diagnostic experiments.measure.test_r54_cpu_hosted -v
- name: Build audited pair and execute one six-observation diagnosis
id: diagnose
run: python -m experiments.measure.r54_cpu_hosted --out local/r54-cpu-raw
- name: Prepare the synthetic artifact whitelist
if: always() && steps.diagnose.outcome != 'skipped'
run: python -m experiments.measure.r54_publish --experiment cpu-boundary --raw local/r54-cpu-raw --out local/r54-cpu-public
- name: Preserve complete or failed synthetic evidence
if: always() && steps.diagnose.outcome != 'skipped'
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: r54-cpu-${{ github.run_id }}-${{ github.run_attempt }}
retention-days: 30
if-no-files-found: error
path: local/r54-cpu-public/
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ if(BUILD_TESTING)
endif()
termforge_add_python_test(core -m unittest experiments.core.test_core experiments.core.test_unicode -v)
termforge_add_python_test(recovery -m unittest experiments.core.test_recovery -v)
termforge_add_python_test(measure -m unittest experiments.measure.test_measure experiments.measure.test_r54_contract.CostPlanTests experiments.measure.test_r54_batch experiments.measure.test_r54_hosted experiments.measure.test_r54_cpu_diagnostic -v)
termforge_add_python_test(measure -m unittest experiments.measure.test_measure experiments.measure.test_r54_contract.CostPlanTests experiments.measure.test_r54_batch experiments.measure.test_r54_hosted experiments.measure.test_r54_cpu_diagnostic experiments.measure.test_r54_cpu_hosted -v)
termforge_add_python_test(binary_paths -m unittest experiments.test_build_paths -v)
termforge_add_python_test(conpty_margin_contract -m unittest experiments.conpty.test_margin_contract -v)
termforge_add_python_test(benchmark_tools -m unittest discover -s benchmarks/tests -v)
Expand Down
1 change: 1 addition & 0 deletions docs/EVIDENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,5 +165,6 @@ python -m unittest experiments.shell.test_workflow_sessions -v
| 固定字素成本的批次协议与执行 | [R54 v1 合同和构建/环境记录](../experiments/measure/R54-COST.md)、[执行器](../experiments/measure/r54_batch.py)、[合成进程回归](../experiments/measure/test_r54_batch.py)、[实际预检及限定验证](../experiments/measure/results/2026-10-27-r54-runner/validation.json) | 固定两个种子、各 112 个位置,字节原样保留、首错停止、完整批次后逐组配对统计。共享工作站十区间活动预检未通过,未启动成本样本;测量相关 26 项测试通过,首次 Linux 候选 CI 的采样依赖导入错误及修正分别留证。5% 是环境诊断护栏,并非产品性能容差;实际批次、扰动归因及用户预算确认由 #54 接续。 |
| 首次 hosted 字素成本批次 | [来源、停止位置与归档](../experiments/measure/results/2026-10-27-r54-hosted/README.md)、[机器可读记录](../experiments/measure/results/2026-10-27-r54-hosted/validation.json) | 两份固定构建与4次sanity通过,空闲预检通过;第24位置的未归属CPU活动估计5.410%触发原5%护栏,保留23个有效记录和1个失败记录,第二批未启动。全部24次输出正确,但批次不完整,不生成配对成本摘要;临时VM逐进程归因、完整受控批次和预算仍未完成。 |
| CPU计量边界诊断(#54) | [固定六位置及归档](../experiments/measure/results/2026-10-27-r54-cpu-boundary/README.md)、[独立复核](../experiments/measure/results/2026-10-27-r54-cpu-boundary/verify.py) | 同一共享Windows宿主、原已审计构建的六次输出和来源核验通过;分别保留完整子进程、内层区间、runner和全机计数。区间外CPU为0.078125–0.109375s,仍有明显未归属活动,不算空闲成本批次,不反推旧hosted原因或修改原5%护栏。 |
| 新hosted VM的CPU分量(#54) | [预声明](../experiments/measure/R54-CPU-HOSTED.md)、[六位置/来源/完整归档与只读复核](../experiments/measure/results/2026-10-27-r54-cpu-hosted/README.md) | 新构建和六个输出均通过;完整扣除自身/runner后余额仍为3.333%–5.263%,不归因旧事件、不作成本验收。准备失败与未创建作业的入口分别保留,未重跑或补样。 |

目前没有足够证据判断 TermForge 距离 WindTerm 快或慢多少。正式结论必须同时提供负载/配置/环境身份、正确性、完成端点、观测开销、重复样本与限制。未知字段保持未知;历史校准未完成不转成要求用户反复手动运行的前置。执行规则见 [ADR-0002](adr/0002-automation-first-investigation.md),核心选型重审见 [ADR-0003](adr/0003-core-reference-and-reuse.md)。
2 changes: 1 addition & 1 deletion docs/FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
| 自动正确性与构建 | **限定验收**:Windows shell/ConPTY/呈现及 Windows/Linux 核心 CI、差分/恢复/队列/生命周期场景 | 产品打包、干净机器使用、依赖升级、长期回归和更广平台矩阵;#40 的 CI 纯夹具超时、窄窗口选择 #73、四 shell 原事件 #80、hosted 准备 #81 和[复制夹具 OpenClipboard 5 #85](https://github.com/JavaZeroo/termforge/issues/85) 等独立失败仍未闭合;#72 保存判据与 #80 同步条件的限定修正不代替其他配置或未知根因验收 | R2 / R4 |
| Windows 产品交付 | **实现中,未验收**:首个 [v0.1.0-alpha.1 预发布](releases/v0.1.0-alpha.1.md)提供 Windows x64、静态 MSVC 运行库、附资源许可和构建身份的免安装 ZIP;未签名,已验环境限 Windows 11 本机与 Server 2025 CI,布局仍写入 LOCALAPPDATA | 这是 R1 体验包,尚非 R2 日常开发版;干净机器与更广平台矩阵、签名/安装、配置升级及持续运行仍待完成。Windows 10 未验收,Server 2022 已知资源门槛未过 | R2 / R3 |
| Linux/macOS 与其他架构 | **未实现**客户端;Linux 核心 CI 已有。[官方平台范围台账](../research/windterm-2.7.0-platform-scope.md)已固定 2.7.0 的 Windows x86/x64、macOS x64、Linux x64 资产及声明下限;这不是 TermForge 支持或实测结论 | 最终需真实 Linux/macOS 客户端;平台调查可提前进行,整合及完整验收排在 Windows 11 x64 完整版本之后。其他系统版本、最低版本和架构由用户后续明确;旧系统及 ARM 等保持调查记录,不自动承诺或排除,也不能由核心 CI 推断产品支持 | 持续调查;R5 全平台出口 |
| 吞吐、响应、内存与稳定性 | **实现中,未验收**:已有机制实验、连续 CPU 区间校准与合成回归;候选字素改动的受限诊断中 project/interactive 吞吐中位数约下降 16%/9%,由 [#54](https://github.com/JavaZeroo/termforge/issues/54) 跟踪。[冻结批次执行器](../experiments/measure/R54-COST.md#冻结批次执行器windows)核对构建来源、原始输出、停止规则与逐轮配对统计;[独立 hosted 构建与实测](../experiments/measure/results/2026-10-27-r54-hosted/README.md)已验证来源和失败保留,但首批因环境护栏停止,尚无完整成本批次;[独立CPU边界诊断](../experiments/measure/results/2026-10-27-r54-cpu-boundary/README.md)另记完整子进程/执行器分量,不改原护栏或判成本通过 | 旧诊断的共享宿主、allocator 包装与缺少真实呈现/输入时延端点使其不构成正式预算;新的不完整批次也不作吞吐结论。仍需同负载/同正确性、无插桩对照、观测开销和长期/多会话公平性;可靠原版基线后提出逐项预算及允许波动范围,**由用户明确确认**。与 WindTerm 的实际差距**待取证** | 当前优先;贯穿 R1–R5 |
| 吞吐、响应、内存与稳定性 | **实现中,未验收**:已有机制实验、连续 CPU 区间校准与合成回归;候选字素改动的受限诊断中 project/interactive 吞吐中位数约下降 16%/9%,由 [#54](https://github.com/JavaZeroo/termforge/issues/54) 跟踪。[冻结批次执行器](../experiments/measure/R54-COST.md#冻结批次执行器windows)核对构建来源、原始输出、停止规则与逐轮配对统计;[独立 hosted 构建与实测](../experiments/measure/results/2026-10-27-r54-hosted/README.md)已验证来源和失败保留,但首批因环境护栏停止,尚无完整成本批次;[共享宿主CPU诊断](../experiments/measure/results/2026-10-27-r54-cpu-boundary/README.md)及[新hosted六位置](../experiments/measure/results/2026-10-27-r54-cpu-hosted/README.md)分别记录完整子进程/执行器分量;后者仍有3.333%–5.263%未归属余额,不改原护栏或判成本通过 | 旧诊断的共享宿主、allocator 包装与缺少真实呈现/输入时延端点使其不构成正式预算;新的不完整批次也不作吞吐结论。仍需同负载/同正确性、无插桩对照、观测开销和长期/多会话公平性;可靠原版基线后提出逐项预算及允许波动范围,**由用户明确确认**。与 WindTerm 的实际差距**待取证** | 当前优先;贯穿 R1–R5 |
| 大历史与动态内存压缩 | **未实现**产品能力;WindTerm 压缩属于官方性能声明,内部机制尚未证实 | 1/10/100 万行等明确规模下的内存/滚动/搜索/淘汰实验;根据瓶颈决定独立存储/压缩实现 | R4 |
| 完整目标验收 | **待取证**:已有固定版本和局部观察,没有完整行为/性能对照矩阵 | 先在 Windows 11 x64 完整验收功能/行为/UI/性能及独立构建,再完成 Linux/macOS 全平台出口;重要差异和性能容差须有用户确认,Windows 通过不代表全平台完成;不依赖 WindTerm 二进制或专有资源 | R5 分平台出口 |

Expand Down
29 changes: 29 additions & 0 deletions experiments/measure/R54-CPU-HOSTED.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# R54:独立 hosted CPU 分量诊断

本次延续[六位置CPU边界合同](R54-CPU-DIAGNOSTIC.md),回答在一个新GitHub-hosted Windows VM中,自身区间外CPU、runner CPU与全机余额分别多大。共享工作站的[一次实际诊断](results/2026-10-27-r54-cpu-boundary/README.md)已显示这些分量可以分开记录,但其活动水平不支持成本验收;旧hosted第24位置的原因仍未知。

## 固定执行

以 PR #106 的 `beeb64496b2e2898e8619fa5e9c39f490a3891c4` 为基点。入口为[工作流](../../.github/workflows/r54-cpu.yml),只接受下述三个明确分支的创建push,拒绝attempt2;后续代码/导出器/证据修正不自动触发替代观察。原 `r54-cost.yml`、两批各112位置、原5%护栏及其失败记录不变。

首个`perf/grapheme-cpu-hosted-r54`创建作业 [40509157581](https://github.com/JavaZeroo/termforge/actions/runs/40509157581) 来自`eed0ab7bae5fcc5d05a25cea5161dabc4f935878`,attempt1,在31项准备测试中的发布夹具路径边界比较处失败。夹具根路径未规范化,被测发布器则将输入路径resolve后比较;旧job没有记录该临时目录的实际路径形式。构建、诊断和导出均未启动,实际观察为0,旧run不重跑。本地使用真实8.3别名复现了该缺陷:旧比较为false、规范化后为true;这不是旧hosted路径形式的直接证据。修正仅将夹具根目录规范化,不放宽发布器路径边界。

修正与审查后,`perf/grapheme-cpu-hosted-r54-setup2`于2026-10-27 13:22:55 UTC在`c14adb9a7fa446476ad5c95c1f8c1b139ceb5ac4`创建。提供者列表只出现无路径过滤的[默认回归40511533857](https://github.com/JavaZeroo/termforge/actions/runs/40511533857),没有该分支的CPU或候选工作流作业,实际诊断观察仍为0;原实验分支同源码更新对应的CPU作业40511437233为skipped。不能将默认回归计作诊断执行。

GitHub的[路径过滤规则](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#git-diff-comparisons)依赖push文件差异,空差异不运行工作流。上述缺席与新分支在已发布commit上创建、两个工作流均有路径过滤相符,但事件实际文件列表未留存。为消除入口对此差异计算的依赖,本诊断移除paths过滤,仍保留明确分支白名单、创建push条件和attempt1检查。

第三入口预声明为`perf/grapheme-cpu-hosted-r54-entry3`的新创建push,使用修订后新source/run身份,一次完成原构建及六位置诊断。前三个入口的实际状态分别保留,不重建旧分支、不重跑旧run、不增加或替换任何已存在的测量样本;后续修订push仍不会执行诊断。该入口变化不改测量代码、oracle、停止条件或产品容差。[实际run40512310499及归档复核](results/2026-10-27-r54-cpu-hosted/README.md)已完成:六个位置输出正确、计量完整,扣除自身完整工作后仍有3.333%–5.263%的余额;没有成本或性能验收结论。

一个新的`windows-2025` VM依次准备Python3.12、psutil7.2.2、Rust1.92.0,执行合同测试,然后[复建并审计](r54_build.py)原固定 `.3/.4` 及四次短正确性。仍是MSVC x64、Release、locked/offline、no-default-features、jobs1、独立target及两个共同harness覆盖;80/84来源文件、原锁文件、真实编译/链接日志和EXE哈希保持可追溯。它是新构建,不借用本地或上次hosted二进制身份。

构建/测试结束后只执行一次 `.3,.4,.4,.3,.3,.4`,全部为原interactive 22,000B、chunk1、observe0、内部CPU/墙钟至少2秒。使用未经修改的`r54_cpu_diagnostic`完整子进程/runner/全机计量,原完整输出oracle和来源前后审计继续执行。单子进程90秒、清理5秒;来源、输出、计数或退出不成立时立即停止,不补样。构建期限600秒、job30分钟;构建失败不进入诊断。

本次不跑空闲资格预检,不计算A/B成本比或吞吐回归,不接受或调整产品性能容差。`accounting_complete`只表示六份诊断记录完整。全机余额保留正负,可能包含其他guest活动、计数粒度和采样边界;不把它称为某个外部进程的精确CPU,也不能补造旧hosted缺失的全生命周期计数。两台宿主的绝对数值不能直接用于性能比较。

## 证据和公开范围

保留GitHub事件/run/attempt/commit、实际runner/映像、构建来源、host OS/CPU、原始计数及全部观察;分析时从提供者作业元数据独立核对身份。没有修改电源、优先级、亲和性或后台服务的步骤。物理宿主排他性与频率仍未知。

发布器的显式`cpu-boundary`模式只包含本次构建和六位置诊断的固定白名单,与原成本批次分开。它在任何staging之前核对本次raw schema/run/attempt/source,失败和不完整流也保留;测量、sanity及诊断入口identity的stdout/stderr字节不改;bundle内的构建identity日志保留完整内容并执行路径前缀替换。元数据中的实际长/短/转义路径改为别名,子PID及进程清单不公开;清单同时列原始/公开SHA和大小。绝不递归上传checkout、Cargo缓存或任意目录。新分支身份和schema不会被原默认发布模式接受。

只有下载并审核实际产物、源码/构建、全部原字节oracle和CPU等式之后,才将观察写入 #54。若此诊断支持改进后续环境计量,须另行冻结新协议与完整样本计划;不倒改原v1,也不以诊断完成代替受控A/B、allocator/projection及组件归因、预算提案和用户确认。
55 changes: 55 additions & 0 deletions experiments/measure/r54_cpu_hosted.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""One fresh hosted build followed by the separately declared CPU-boundary diagnosis."""
import argparse
import json
from pathlib import Path

from experiments.measure.r54_batch import sha, utc, write_json
from experiments.measure.r54_build import prepare_builds
from experiments.measure.r54_contract import ROOT
from experiments.measure.r54_cpu_diagnostic import execute as diagnose
from experiments.measure.r54_hosted import CPU_BOUNDARY_BRANCHES, hosted_identity
from experiments.measure.r54_inputs import git, require

def execute(output):
output = Path(output).resolve()
require(output.is_relative_to(ROOT / 'local'), 'hosted artifacts must stay under the workspace local directory')
output.mkdir(parents=True, exist_ok=False)
manifest = dict(schema='termforge-r54-cpu-hosted-v1', status='preparing', started_utc=utc(),
source=git(ROOT, 'rev-parse', 'HEAD'), runner_sha256=sha(Path(__file__)),
workflow_sha256=sha(ROOT / '.github/workflows/r54-cpu.yml'),
performance_gate='not_evaluated', diagnostic=None)
try:
manifest['hosted_identity'] = hosted_identity(CPU_BOUNDARY_BRANCHES)
write_json(output / 'hosted.json', manifest)
prepare_builds(output / 'bundle')
declaration = dict(host_kind='hosted_vm', no_known_concurrent_build_or_test=True,
known_concurrent_work=['This job completed setup, tests and both audited builds before diagnosis.',
'GitHub runner service and operating-system background services remain.',
'Physical host scheduling and neighboring virtual machines are unobserved.'],
purpose='One six-observation CPU-boundary diagnostic; no idle qualification or cost acceptance.')
write_json(output / 'environment-declaration.json', declaration)
manifest['status'] = 'diagnosing'
write_json(output / 'hosted.json', manifest)
result = diagnose(output / 'bundle/plan-v1.json', output / 'bundle',
output / 'environment-declaration.json', output / 'diagnostic')
manifest['diagnostic'] = result
require(result['status'] == 'accounting_complete', 'diagnosis stopped; no replacement observations')
manifest['status'] = 'accounting_complete'
except BaseException as error:
manifest.update(status='failed', error=f'{type(error).__name__}: {error}')
raise
finally:
manifest['finished_utc'] = utc()
write_json(output / 'hosted.json', manifest)
return manifest


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--out', type=Path, required=True)
args = parser.parse_args()
print(json.dumps(execute(args.out)))


if __name__ == '__main__':
main()
Loading