main · last commit
13 days ago ·
7g0stsfu
ah-1cx.1 runner.Status re-parses full events.jsonl every poll, stalling the loop on large runs
Past Stand
bd reopen ah-1cx.1
| Created by | Eugene Blikh |
| Owner | bigbes@gmail.com |
| Created | 2026-07-19T23:37:28Z |
| Started | 2026-08-04T23:32:43Z |
| Updated | 2026-08-04T23:45:37Z |
| Closed | 2026-08-04T23:45:37Z |
Status unconditionally calls ParseEventsFile (reads to EOF, decodes every line) on every 20s poll for every in-progress task (runner.go:209-219), but the parsed pi-session/cost are display-only until finalize, which re-reads via completionFactsFromEvents anyway. On a multi-hundred-MB events stream (ah-07g observed 202MB) this blocks the single-goroutine reconcile loop for seconds per tick, starving supervision of every other task. Fix: skip the full parse for a still-running run — read only the first line for the session id (or defer session/cost to finalize) and keep the cheap os.Stat mtime for the watchdog. Found by core-runtime audit 2026-07-20.
ah-1cx
— Hardening: post-audit bug sweep (2026-07-20)
parent-child
open
Nothing depends on this issue.
| id | ah-1cx.1 |
| content_hash | e94360d1f52dbea42a761d52a15cb74d70d7f0ea533c2469487cf945d307328f |
| title | runner.Status re-parses full events.jsonl every poll, stalling the loop on large runs |
| description | Status unconditionally calls ParseEventsFile (reads to EOF, decodes every line) on every 20s poll for every in-progress task (runner.go:209-219), but the parsed pi-session/cost are display-only until finalize, which re-reads via completionFactsFromEvents anyway. On a multi-hundred-MB events stream (ah-07g observed 202MB) this blocks the single-goroutine reconcile loop for seconds per tick, starving supervision of every other task. Fix: skip the full parse for a still-running run — read only the first line for the session id (or defer session/cost to finalize) and keep the cheap os.Stat mtime for the watchdog. Found by core-runtime audit 2026-07-20. |
| design | |
| acceptance_criteria | |
| notes | |
| status | closed |
| priority | 2 |
| issue_type | bug |
| assignee | NULL |
| estimated_minutes | NULL |
| created_at | 2026-07-19T23:37:28Z |
| created_by | Eugene Blikh |
| owner | bigbes@gmail.com |
| updated_at | 2026-08-04T23:45:37Z |
| closed_at | 2026-08-04T23:45:37Z |
| closed_by_session | |
| external_ref | NULL |
| spec_id | |
| compaction_level | 0 |
| compacted_at | NULL |
| compacted_at_commit | NULL |
| original_size | NULL |
| sender | |
| ephemeral | 0 |
| wisp_type | |
| pinned | 0 |
| is_template | 0 |
| mol_type | |
| work_type | |
| source_system | |
| metadata | �{} |
| source_repo | |
| close_reason | Fixed in f11ef03. Status's per-poll read of events.jsonl is now asymmetric: the still-running path does only the os.Stat it already needed for LastEvent (the stall-watchdog input) plus PeekSessionID, a new bounded head probe that reads at most 256 KiB / 8 leading lines looking for the {"type":"session","id":...} event pi writes as line 1. The full ParseEventsFile no longer sits on the 20s poll path, so a 202 MB stream (ah-07g) costs a running poll the same as an empty one and can no longer freeze the single-goroutine reconcile loop. CostUSD is deliberately left at zero for a running run — cost is a whole-stream sum with no cheap mid-run answer, and the reconciler only persists cost at finalize — and PiSession is explicitly best-effort ("" while the head is unwritten or torn), which is safe because the reconciler only overwrites a stored session id with a non-empty one. Every path that COMPLETES a run still takes the full parse: both exit.json branches via finalizeFromExit (ah-2ef's durable pinning, untouched) and — newly — the crash-confirmed branch, since a confirmed crash writes an immutable finished row and previously relied on the top-of-Status parse for the failure comment's session id and cost. The reader loop was extracted into scanEventLines(r, fn) with an early-stop callback, so the F9a over-long-line rule (skip the line, keep scanning; never bufio.Scanner) is shared by the full parse and the probe and cannot drift; the probe's byte budget is deliberately 4x the read buffer so a leading line wider than the buffer is stepped over rather than ending the probe. Cover: TestStatusRunningDoesNotParseWholeEventStream builds an 8 MB stream with the id on line 1 and a fat cost-bearing turn at the end, asserting Running ⇒ id resolved + CostUSD==0 + LastEvent==mtime, then drops exit.json and asserts the same stream yields the full cost; TestPeekSessionIDIsBounded proves boundedness structurally. Mutation-checked. KNOWN REGRESSION SPLIT OUT: kill-finalized runs now record CostUSD 0 — see the dedicated bead. |
| event_kind | |
| actor | |
| target | |
| payload | |
| await_type | |
| await_id | |
| timeout_ns | 0 |
| waiters | |
| hook_bead | |
| role_bead | |
| agent_state | |
| last_activity | NULL |
| role_type | |
| rig | |
| due_at | NULL |
| defer_until | NULL |
| no_history | 0 |
| started_at | 2026-08-04T23:32:43Z |
| is_blocked | 0 |
| issue_id | ah-1cx.1 |
| label | milestone:v0.1.0 |
| id | 3ee49f68-c026-5f27-b29a-fb985cb9c958 |
| issue_id | ah-1cx.1 |
| type | parent-child |
| created_at | 2026-07-20T02:37:27Z |
| created_by | Eugene Blikh |
| metadata | �{} |
| thread_id | |
| depends_on_issue_id | ah-1cx |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 019f7cbd-e63e-7d32-84c4-b86cf043f6f2 |
| issue_id | ah-1cx.1 |
| event_type | created |
| actor | Eugene Blikh |
| old_value | |
| new_value | |
| comment | NULL |
| created_at | 2026-07-20T02:37:27Z |
| id | 019f7cda-2a1a-7075-b1d9-c6aaeb66af8b |
| issue_id | ah-1cx.1 |
| event_type | label_added |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Added label: milestone:hardening |
| created_at | 2026-07-20T03:08:20Z |
| id | 019f7cdb-eae1-78a6-bedd-9293d27479cd |
| issue_id | ah-1cx.1 |
| event_type | label_removed |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Removed label: milestone:hardening |
| created_at | 2026-07-20T03:10:15Z |
| id | 019f7cdc-fae8-721b-a240-84e7a5944535 |
| issue_id | ah-1cx.1 |
| event_type | label_added |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Added label: milestone:v0.1.0 |
| created_at | 2026-07-20T03:11:24Z |
| id | 019fcf1f-4ea7-7756-bd3e-4a8733778ba2 |
| issue_id | ah-1cx.1 |
| event_type | status_changed |
| actor | Eugene Blikh |
| old_value | {"id":"ah-1cx.1","title":"runner.Status re-parses full events.jsonl every poll, stalling the loop on large runs","description":"Status unconditionally calls ParseEventsFile (reads to EOF, decodes every line) on every 20s poll for every in-progress task (runner.go:209-219), but the parsed pi-session/cost are display-only until finalize, which re-reads via completionFactsFromEvents anyway. On a multi-hundred-MB events stream (ah-07g observed 202MB) this blocks the single-goroutine reconcile loop for seconds per tick, starving supervision of every other task. Fix: skip the full parse for a still-running run — read only the first line for the session id (or defer session/cost to finalize) and keep the cheap os.Stat mtime for the watchdog. Found by core-runtime audit 2026-07-20.","status":"open","priority":2,"issue_type":"bug","owner":"bigbes@gmail.com","created_at":"2026-07-19T23:37:28Z","created_by":"Eugene Blikh","updated_at":"2026-07-19T23:37:28Z","labels":["milestone:v0.1.0"]} |
| new_value | {"status":"in_progress"} |
| comment | NULL |
| created_at | 2026-08-05T02:32:43Z |
| id | 019fcf2b-1d12-73b9-a18f-ba9fd80987cd |
| issue_id | ah-1cx.1 |
| event_type | closed |
| actor | Eugene Blikh |
| old_value | |
| new_value | Fixed in f11ef03. Status's per-poll read of events.jsonl is now asymmetric: the still-running path does only the os.Stat it already needed for LastEvent (the stall-watchdog input) plus PeekSessionID, a new bounded head probe that reads at most 256 KiB / 8 leading lines looking for the {"type":"session","id":...} event pi writes as line 1. The full ParseEventsFile no longer sits on the 20s poll path, so a 202 MB stream (ah-07g) costs a running poll the same as an empty one and can no longer freeze the single-goroutine reconcile loop. CostUSD is deliberately left at zero for a running run — cost is a whole-stream sum with no cheap mid-run answer, and the reconciler only persists cost at finalize — and PiSession is explicitly best-effort ("" while the head is unwritten or torn), which is safe because the reconciler only overwrites a stored session id with a non-empty one. Every path that COMPLETES a run still takes the full parse: both exit.json branches via finalizeFromExit (ah-2ef's durable pinning, untouched) and — newly — the crash-confirmed branch, since a confirmed crash writes an immutable finished row and previously relied on the top-of-Status parse for the failure comment's session id and cost. The reader loop was extracted into scanEventLines(r, fn) with an early-stop callback, so the F9a over-long-line rule (skip the line, keep scanning; never bufio.Scanner) is shared by the full parse and the probe and cannot drift; the probe's byte budget is deliberately 4x the read buffer so a leading line wider than the buffer is stepped over rather than ending the probe. Cover: TestStatusRunningDoesNotParseWholeEventStream builds an 8 MB stream with the id on line 1 and a fat cost-bearing turn at the end, asserting Running ⇒ id resolved + CostUSD==0 + LastEvent==mtime, then drops exit.json and asserts the same stream yields the full cost; TestPeekSessionIDIsBounded proves boundedness structurally. Mutation-checked. KNOWN REGRESSION SPLIT OUT: kill-finalized runs now record CostUSD 0 — see the dedicated bead. |
| comment | NULL |
| created_at | 2026-08-05T02:45:37Z |
No comments.
Close reason
Fixed in f11ef03. Status's per-poll read of events.jsonl is now asymmetric: the still-running path does only the os.Stat it already needed for LastEvent (the stall-watchdog input) plus PeekSessionID, a new bounded head probe that reads at most 256 KiB / 8 leading lines looking for the {"type":"session","id":...} event pi writes as line 1. The full ParseEventsFile no longer sits on the 20s poll path, so a 202 MB stream (ah-07g) costs a running poll the same as an empty one and can no longer freeze the single-goroutine reconcile loop. CostUSD is deliberately left at zero for a running run — cost is a whole-stream sum with no cheap mid-run answer, and the reconciler only persists cost at finalize — and PiSession is explicitly best-effort ("" while the head is unwritten or torn), which is safe because the reconciler only overwrites a stored session id with a non-empty one. Every path that COMPLETES a run still takes the full parse: both exit.json branches via finalizeFromExit (ah-2ef's durable pinning, untouched) and — newly — the crash-confirmed branch, since a confirmed crash writes an immutable finished row and previously relied on the top-of-Status parse for the failure comment's session id and cost. The reader loop was extracted into scanEventLines(r, fn) with an early-stop callback, so the F9a over-long-line rule (skip the line, keep scanning; never bufio.Scanner) is shared by the full parse and the probe and cannot drift; the probe's byte budget is deliberately 4x the read buffer so a leading line wider than the buffer is stepped over rather than ending the probe. Cover: TestStatusRunningDoesNotParseWholeEventStream builds an 8 MB stream with the id on line 1 and a fat cost-bearing turn at the end, asserting Running ⇒ id resolved + CostUSD==0 + LastEvent==mtime, then drops exit.json and asserts the same stream yields the full cost; TestPeekSessionIDIsBounded proves boundedness structurally. Mutation-checked. KNOWN REGRESSION SPLIT OUT: kill-finalized runs now record CostUSD 0 — see the dedicated bead.