~bigbes/agents-dev · parade

main · last commit 13 days ago · 7g0stsfu

← Back to the parade

ah-nyl.12 hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation) Past Stand

status: closed P1 task
bd reopen ah-nyl.12
Created byEugene Blikh
Ownerbigbes@gmail.com
Created2026-07-13T00:41:22Z
Started2026-07-13T01:05:54Z
Updated2026-07-13T01:31:07Z
Closed2026-07-13T01:31:07Z
Description
Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.

H1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -> kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -> return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (<50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.

H2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.

H3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.

H4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like "true"/"123"/"null" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to "true".

H5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.

Constraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.
Notes
H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -> loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation.

Depends on

  • ah-nyl — Stage 1 MVP: poll-driven happy path parent-child closed

Depended on by

Nothing depends on this issue.

No comments.

Close reason

landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck)
  • Eugene Blikh added under epic ah-nyl · 2026-07-13T03:41:22Z
  • Eugene Blikh created the issue · 2026-07-13T03:41:22Z
  • Eugene Blikh updated notes to H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -> loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation. · 2026-07-13T03:42:04Z
  • Eugene Blikh changed status to in_progress · 2026-07-13T04:05:54Z
  • Eugene Blikh closed the issue · 2026-07-13T04:31:06Z
    landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck)
  • Eugene Blikh added label milestone:stage-1 · 2026-07-20T03:08:46Z
  • Eugene Blikh removed label milestone:stage-1 · 2026-07-20T03:10:19Z
Stored rows — what this pane was built from, as read
issues 1 row
id ah-nyl.12
content_hash 7714724648b34e987bf24b58176889b5580a4ff3c05efff661cea5999723a656
title hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)
description Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a. H1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -> kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -> return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (<50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error. H2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both. H3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly. H4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like "true"/"123"/"null" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to "true". H5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change. Constraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.
design
acceptance_criteria
notes H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -> loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation.
status closed
priority 1
issue_type task
assignee NULL
estimated_minutes NULL
created_at 2026-07-13T00:41:22Z
created_by Eugene Blikh
owner bigbes@gmail.com
updated_at 2026-07-13T01:31:07Z
closed_at 2026-07-13T01:31:07Z
closed_by_session
external_ref NULL
spec_id
compaction_level 0
compacted_at NULL
compacted_at_commit NULL
original_size NULL
sender
ephemeral 0
wisp_type
pinned 0
is_template 0
mol_type
work_type
source_system
metadata �{}
source_repo
close_reason landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck)
event_kind
actor
target
payload
await_type
await_id
timeout_ns 0
waiters
hook_bead
role_bead
agent_state
last_activity NULL
role_type
rig
due_at NULL
defer_until NULL
no_history 0
started_at 2026-07-13T01:05:54Z
is_blocked 0
dependencies 1 row
id d179f647-ea51-5b28-9b29-aedbab3e952b
issue_id ah-nyl.12
type parent-child
created_at 2026-07-13T03:41:22Z
created_by Eugene Blikh
metadata �{}
thread_id
depends_on_issue_id ah-nyl
depends_on_wisp_id NULL
depends_on_external NULL
events 6 rows
id 019f58eb-e3fb-7000-bdb5-015c39c069b6
issue_id ah-nyl.12
event_type created
actor Eugene Blikh
old_value
new_value
comment NULL
created_at 2026-07-13T03:41:22Z
id 019f58ec-8784-78f4-b1c8-44728fc87ed9
issue_id ah-nyl.12
event_type updated
actor Eugene Blikh
old_value {"id":"ah-nyl.12","title":"hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)","description":"Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.\n\nH1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -\u003e kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -\u003e return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (\u003c50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.\n\nH2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.\n\nH3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.\n\nH4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like \"true\"/\"123\"/\"null\" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to \"true\".\n\nH5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.\n\nConstraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.\n","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-13T00:41:22Z","created_by":"Eugene Blikh","updated_at":"2026-07-13T00:41:22Z"}
new_value {"notes":"H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -\u003e loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation."}
comment NULL
created_at 2026-07-13T03:42:04Z
id 019f5902-5a44-7bcf-903c-8cbd5f2b30a6
issue_id ah-nyl.12
event_type status_changed
actor Eugene Blikh
old_value {"id":"ah-nyl.12","title":"hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)","description":"Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.\n\nH1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -\u003e kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -\u003e return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (\u003c50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.\n\nH2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.\n\nH3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.\n\nH4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like \"true\"/\"123\"/\"null\" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to \"true\".\n\nH5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.\n\nConstraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.\n","notes":"H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -\u003e loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-13T00:41:22Z","created_by":"Eugene Blikh","updated_at":"2026-07-13T00:42:04Z"}
new_value {"status":"in_progress"}
comment NULL
created_at 2026-07-13T04:05:54Z
id 019f5919-6f31-7dc3-8294-578d88e2546c
issue_id ah-nyl.12
event_type closed
actor Eugene Blikh
old_value
new_value landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck)
comment NULL
created_at 2026-07-13T04:31:06Z
id 019f7cda-9085-795e-84ed-1baf2308465c
issue_id ah-nyl.12
event_type label_added
actor Eugene Blikh
old_value NULL
new_value NULL
comment Added label: milestone:stage-1
created_at 2026-07-20T03:08:46Z
id 019f7cdb-fd6b-71e4-b526-ddcee3931417
issue_id ah-nyl.12
event_type label_removed
actor Eugene Blikh
old_value NULL
new_value NULL
comment Removed label: milestone:stage-1
created_at 2026-07-20T03:10:19Z