[ruthwikdasyam]

Problem

There are no ABC Sim tasks in the evals.

Changes

• ABC Sim scenes as LFS archives (data/.lfs/abc_*.tar.gz).
• openyam-manipulation-sim (one arm) and dual-openyam-abc-sim (both arms) blueprints.
• Suites:
  • abc_sim: all 33 ABC tasks, both OpenYAM arms.
  • abc_sim_single_arm: 3 tasks, one arm.
• Pose graders in dimos/evals/scorers.py.
• Cases find objects with manipulation-localize, and get grasps from the heuristic grasp module.
• Ground truth kept from the agent:
  • peek_stream refuses body_poses.
  • The porcelain cannot call the sim's reset, respawn_at or scene_path (remote_hidden_rpcs).
  • The prompt forbids recordings, data/ and the MuJoCo model.
• Joint feedback a hair past a limit is clamped onto it. Before, IK and RoboPlan rejected the start state.
• Eval image: Node 24 (the dimos agent needs it), libEGL, dimos agent prebuilt.

Testing

• Unit tests for the clamp, hidden RPCs and the env vars.
• Ran 3 cases on EC2 with pi and the dimos agent (Fable 5.1).

[github-actions]

go2 replay realtime (arm64)

Details

Benchmark suite	Current: a782cde	Previous: 5e0ebfd	Ratio
peak memory	1739.906 MB	1715.605 MB	1.01
peak threads	382 threads	379 threads	1.01
network (transport)	2725.832 MB	2727.928 MB	1.00
disk write	1.613 MB	1.93 MB	0.84
instructions	137.299 G	138.562 G	0.99

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
9567	2	9565	68

View the top 2 failed test(s) by shortest run time

experimental.gateway.tests.test_config::test_every_registered_blueprint_is_annotated

Stack Traces | 0.215s run time

def test_every_registered_blueprint_is_annotated() -> None:
        source = annotation_file.load()
        annotated = {name for robot in source["robots"].values() for name in robot["blueprints"]}
        missing = sorted(
            name
            for name in all_blueprints
            if name not in annotated and not annotation_file.HIDDEN_NAME.search(name)
        )
>       assert not missing, f"add these to experimental/gateway/annotations.yaml: {', '.join(missing)}"
E       AssertionError: add these to experimental/gateway/annotations.yaml: dual-openyam-abc-sim, openyam-manipulation-sim
E       assert not ['dual-openyam-abc-sim', 'openyam-manipulation-sim']

annotated  = {'a1z-keyboard-teleop', 'a1z-planner-coordinator', 'a1z-teleop', 'alfred-keyboard-teleop', 'alfred-mls-nav', 'alfred-mls-nav-lidar', ...}
missing    = ['dual-openyam-abc-sim', 'openyam-manipulation-sim']
source     = {'robots': {'a1z': {'blueprints': {'a1z-keyboard-teleop': {'description': "Move a real A1Z's end and its gripper with ...': {'label': 'Arms'}, 'dog': {'label': 'Dogs'}, 'drone': {'label': 'Drones'}, 'humanoid': {'label': 'Humanoids'}, ...}}

.../gateway/tests/test_config.py:325: AssertionError

dimos.codebase_checks.test_no_underscore_assign::test_no_underscore_assignment

Stack Traces | 9.34s run time

def test_no_underscore_assignment():
        """Fail if any file assigns to a bare `_`."""
        dimos_dir = DIMOS_PROJECT_ROOT / "dimos"
        hits = find_underscore_assignments()
        if hits:
            listing = "\n".join(f"  - {p.relative_to(dimos_dir)}:{lineno}" for p, lineno in hits)
>           raise AssertionError(
                f"Found assignment(s) to `_` in dimos/:\n{listing}\n\n"
                "Assigning to `_` is not allowed: it hides an unused variable instead "
                "of removing it. Delete the variable. If you only need the "
                "expression's side effect, evaluate it directly with a call "
                "(`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute "
                "access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine "
                "and not flagged by this rule."
            )
E           AssertionError: Found assignment(s) to `_` in dimos/:
E             - porcelain/test_dimos.py:219
E           
E           Assigning to `_` is not allowed: it hides an unused variable instead of removing it. Delete the variable. If you only need the expression's side effect, evaluate it directly with a call (`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine and not flagged by this rule.

dimos_dir  = PosixPath('.../dimos/dimos/dimos')
hits       = [(PosixPath('.../dimos/dimos/dimos/porcelain/test_dimos.py'), 219)]
listing    = '  - porcelain/test_dimos.py:219'

dimos/codebase_checks/test_no_underscore_assign.py:55: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}