# RECON — Arm A2-id — R1–R5 Date: 2026-07-25. Identified arm; network recon permitted (L8). The workspace arrived pre-staged by the custodian with raw public API captures and corpus bytes (files dated 2026-07-25 18:25–18:28 local, headers dated Sat, 25 Jul 2026 22:25 GMT); every constant below marked *verified-against-bytes* was verified by me from those captures or from my own re-fetches this session (fresh fetches under `work/fetch_fresh/`, `work/pub_fetch/`). No prior-analysis material was encountered in the staging. ## R1 — Public datasets under the `OpenHands` HF org (report, not rank) Source: staged `org_datasets.json` (HF API, HTTP 200). 11 datasets. *verified-against-cited-source* | id | lastModified | license tag | gated | downloads | |---|---|---|---|---| | OpenHands/openhands-feedback | 2024-12-14 | mit | no | 81 | | OpenHands/openhands-evaluation-outputs | 2025-01-18 | mit | no | 114 | | OpenHands/CodeScout_Eval_Rollouts | 2026-03-17 | (none declared) | no | 439 | | OpenHands/CodeScout_Training_Rollouts | 2026-03-17 | (none declared) | no | 90 | | OpenHands/SWE-smith-py-code-search | 2026-03-17 | (none declared) | no | 46 | | OpenHands/SWE-Gym-code-search | 2026-03-17 | (none declared) | no | 38 | | OpenHands/SWE-rebench-code-search | 2026-03-17 | (none declared) | no | 42 | | OpenHands/SWE-bench_Verified-locagent | 2026-03-17 | (none declared) | no | 101 | | OpenHands/SWE-bench_Lite-locagent | 2026-03-17 | (none declared) | no | 60 | | OpenHands/SWE-bench_Pro-locagent | 2026-03-17 | (none declared) | no | 164 | | OpenHands/openhands-index | 2026-06-30 | apache-2.0 | no | 340 | File inventory of the target (HF API `siblings`): `.gitattributes`, `.source_hash`, `README.md`, `instances.parquet`, `test.parquet`. *verified-against-cited-source* ## R2 — Revision history of the target (enumerated; no cross-revision claims) Source: staged `refs.json`, `commits.json` (HTTP 200). *verified-against-cited-source* - 28 commits, 2026-05-29 (`initial commit`) → 2026-06-30 (`Update leaderboard v2026.06.30-3015ac6 (34 models, 40643 instance rows)`, commit `db6504df…` = main). - 27 tags `v-`. Enumeration observations, reported without characterization (L2): (a) tag `v2026.06.30-3015ac6` targets commit `94ac78ad…` while branch `main` targets `db6504df…`, whose commit title carries the same version string; (b) commit-title model counts run …30, 39, 30, 31… across 2026-05-29/30. What a cross-revision comparison would show was not computed and is not asserted. ## R3 — Publisher statements on how the published values are produced Surfaces read (staged + my own pinned fetch): - Dataset README at `db6504df…`: "`average_score` is the mean of the per-benchmark scores actually completed. `categories_completed` tells you how many benchmarks the model has run." "Long-form table of every benchmark instance's outcome … Join to `default` on `id`." "`resolved` … `null` when the archive didn't record an outcome." *verified-against-bytes* (quoted from the shipped card). - `scripts/publish_hf_dataset.py` at source commit `3015ac61…` (sha256 `8f125116bddca13f5d767bbec691811057e2fe9a168de942e9dc64cb41839d39`, HTTP 200, raw.githubusercontent.com): the `default` config is built from `results//scores.json` — per-benchmark `score`, `cost` = `cost_per_instance`, `runtime` = `average_runtime`; `average_* = round(mean over completed, 4)`; `categories_completed = len(completed)`; rows sorted by `average_score` descending. The `instances` config is built separately from `instance_results/.json` sidecars. A `.source_hash` = sha256 over both tables as column-sorted CSV (`float_format="%.10g"`, separator `\n--\n`) is published to detect no-op runs. *verified-against-bytes* (of the script at the pinned commit). - Results-repo README: sidecars "are generated from the `full_archive` URLs in `scores.json`" by `scripts/extract_instance_results.py`; `resolved` is "`null` when the archive does not expose a resolved/unresolved outcome". *verified-against-cited-source* - **The publisher nowhere states that the leaderboard values are computed from the instance table.** Both are described as descending from a common upstream (`scores.json` / archives). *verified-against-cited-source* Looked for and not found on the surfaces examined: (a) any formula tying `_score` to the instance-level `resolved` fraction; (b) the definition of `cost_per_instance` (mean vs total-over-N vs other); (c) any statement of the denominator used for per-benchmark scores; (d) contents of `extract_instance_results.py` (present in repo listing, 23,911 bytes; not fetched — recorded as an unexecuted leg). The GitHub repo `openhands-index-results` declares no license in the API capture (`license: null`) while its README says "MIT License — See repository for details" — reported as-is. *verified-against-cited-source* ## R4 — License of the target HF API `cardData.license: apache-2.0`; tag `license:apache-2.0`; card frontmatter `license: apache-2.0`. Matches Appendix A. Local testing unrestricted regardless (L7). *verified-against-bytes* (card) / *verified-against-cited-source* (API). ## R5 — Appendix A constants vs bytes - `test.parquet`: 44,323 bytes, sha256 `966544ad62ba160fec79c73bdd5295df521743c733f9fbcca1ed0f82833d827a` — **matches**, both in staged copy and in my fresh fetch (HTTP 302→200, `x-repo-commit db6504df…`). *verified-against-bytes* - `instances.parquet`: 435,757 bytes, sha256 `f456e937771bdd45815cacd6458433e0e750be0a2a6bcd5daf91670b151968a5` — **matches**, same two routes. *verified-against-bytes* - A1 `trace-commons/agent-traces`: HF API sha `112ebd4d03ce852b00e935d523107c3d0c9a65bf`, license cc-by-4.0 — **matches** Appendix A. *verified-against-cited-source* - A3 `OpenHands/openhands-feedback`: HF API sha `facf45600d625510b98222ec584a8c9ac59272a2`, license mit — **matches**. *verified-against-cited-source* - Tie censuses "18 of 21" (A1) and "12 of 21" (A3): **not verifiable from any bytes available to this arm**; recorded as unverified constants of prior instruments. *unverified* - Blinding-kit path (`habitat_arms/custodian/…`): **not verified** — reading it would breach this arm's workspace isolation (§4). Declared unexecuted (L9). - Additional shipped constant: `.source_hash` = `886a3ebabd70879b712deb36fcfc190199e9c91a0ea538a89b156a20663fda29` (64-hex, staged fetch); enters the preregistered M-H measurement. *verified-against-bytes* ## Environment for recon Shared MacBook Air (`Mac16,13`, Apple M4, macOS 26.2, arm64), host `MacBook-Air.local`; this arm runs on the shared workstation — **no machine independence claimed** (§3.6). Conda env `habitat` under `/Applications/anaconda3`: Python 3.11.13, numpy 1.26.4, pandas 2.3.2, pyarrow 14.0.2. *verified-against-bytes* (command output this session).