main · last commit
13 days ago ·
7g0stsfu
ah-nyl.12 hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)
Past Stand
bd reopen ah-nyl.12
| Created by | Eugene Blikh |
| Owner | bigbes@gmail.com |
| Created | 2026-07-13T00:41:22Z |
| Started | 2026-07-13T01:05:54Z |
| Updated | 2026-07-13T01:31:07Z |
| Closed | 2026-07-13T01:31:07Z |
Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.
H1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -> kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -> return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (<50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.
H2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.
H3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.
H4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like "true"/"123"/"null" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to "true".
H5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.
Constraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.
H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -> loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation.
ah-nyl
— Stage 1 MVP: poll-driven happy path
parent-child
closed
Nothing depends on this issue.
| id | ah-nyl.12 |
| content_hash | 7714724648b34e987bf24b58176889b5580a4ff3c05efff661cea5999723a656 |
| title | hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation) |
| description | Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a. H1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -> kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -> return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (<50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error. H2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both. H3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly. H4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like "true"/"123"/"null" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to "true". H5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change. Constraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/. |
| design | |
| acceptance_criteria | |
| notes | H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -> loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation. |
| status | closed |
| priority | 1 |
| issue_type | task |
| assignee | NULL |
| estimated_minutes | NULL |
| created_at | 2026-07-13T00:41:22Z |
| created_by | Eugene Blikh |
| owner | bigbes@gmail.com |
| updated_at | 2026-07-13T01:31:07Z |
| closed_at | 2026-07-13T01:31:07Z |
| closed_by_session | |
| external_ref | NULL |
| spec_id | |
| compaction_level | 0 |
| compacted_at | NULL |
| compacted_at_commit | NULL |
| original_size | NULL |
| sender | |
| ephemeral | 0 |
| wisp_type | |
| pinned | 0 |
| is_template | 0 |
| mol_type | |
| work_type | |
| source_system | |
| metadata | �{} |
| source_repo | |
| close_reason | landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck) |
| event_kind | |
| actor | |
| target | |
| payload | |
| await_type | |
| await_id | |
| timeout_ns | 0 |
| waiters | |
| hook_bead | |
| role_bead | |
| agent_state | |
| last_activity | NULL |
| role_type | |
| rig | |
| due_at | NULL |
| defer_until | NULL |
| no_history | 0 |
| started_at | 2026-07-13T01:05:54Z |
| is_blocked | 0 |
| id | d179f647-ea51-5b28-9b29-aedbab3e952b |
| issue_id | ah-nyl.12 |
| type | parent-child |
| created_at | 2026-07-13T03:41:22Z |
| created_by | Eugene Blikh |
| metadata | �{} |
| thread_id | |
| depends_on_issue_id | ah-nyl |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 019f58eb-e3fb-7000-bdb5-015c39c069b6 |
| issue_id | ah-nyl.12 |
| event_type | created |
| actor | Eugene Blikh |
| old_value | |
| new_value | |
| comment | NULL |
| created_at | 2026-07-13T03:41:22Z |
| id | 019f58ec-8784-78f4-b1c8-44728fc87ed9 |
| issue_id | ah-nyl.12 |
| event_type | updated |
| actor | Eugene Blikh |
| old_value | {"id":"ah-nyl.12","title":"hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)","description":"Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.\n\nH1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -\u003e kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -\u003e return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (\u003c50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.\n\nH2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.\n\nH3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.\n\nH4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like \"true\"/\"123\"/\"null\" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to \"true\".\n\nH5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.\n\nConstraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.\n","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-13T00:41:22Z","created_by":"Eugene Blikh","updated_at":"2026-07-13T00:41:22Z"} |
| new_value | {"notes":"H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -\u003e loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation."} |
| comment | NULL |
| created_at | 2026-07-13T03:42:04Z |
| id | 019f5902-5a44-7bcf-903c-8cbd5f2b30a6 |
| issue_id | ah-nyl.12 |
| event_type | status_changed |
| actor | Eugene Blikh |
| old_value | {"id":"ah-nyl.12","title":"hardening: data-layer review fast-follows (vikunja pagination, cache refresh, config validation)","description":"Data-layer review @ b86928a returned SHIP with fast-follow hardenings. Implement all. Line refs @ b86928a.\n\nH1 (from MAJOR-latent finding, internal/vikunja/board.go:28-73): Snapshot must NEVER silently return a partial board (reconcile treats missing tasks as vanished -\u003e kills runs, cancels tasks). Fixes: (a) hitting maxSnapshotPages -\u003e return an error, not partial data; (b) when the x-pagination-total-pages header is absent, do NOT treat a short page (\u003c50) alone as end-of-data — continue until an EMPTY page (and stop on empty page in all cases); (c) if x-pagination-result-count (or total-pages) is present, cross-check the accumulated count and error on mismatch. Tests: header-absent short-page continuation, cap-hit error, mismatch error.\n\nH2 (board.go:28-35, 82-88): cache self-refresh on the read path — on a 404 from Snapshot (stale view id) or MoveToBucket's POST (stale bucket id after delete+recreate), refresh views/buckets once and retry once; second failure returns the error. Tests for both.\n\nH3 (internal/config/config.go:179-247): validate db and work_root non-empty (validation-collected errors + tests). NOTE: another fixer may have recently touched internal/config (slug validation, commit message 'config: validate repo slug shape') — pull the latest state of the file and integrate cleanly.\n\nH4 (config.go:114-124, 300-334): env-expanded plain scalars must stay strings — after substituting ${VAR} in a plain (unquoted) scalar, force the node's tag/style to !!str so a value like \"true\"/\"123\"/\"null\" cannot re-type and spuriously fail KnownFields decode. Test with an unquoted ${VAR} expanding to \"true\".\n\nH5 (internal/store/store.go:371-373): add a short code comment on the timestamp columns noting RFC3339Nano TEXT does not sort lexicographically by instant (variable-width fraction) — any future ORDER BY on time columns must ORDER BY id/seq or normalize width. Comment only, no behavior change.\n\nConstraints: scope = internal/vikunja/, internal/config/, internal/store/ (comment only). Conventions: testify + culpa. Build/vet/test -race ONLY those three packages; never ./... . Commits: 'vikunja: harden snapshot pagination and cache refresh' + 'config: require db/work_root, pin env-expanded scalars to !!str'; hooks bypassed; retry on index.lock; no push; no .beads/.\n","notes":"H6 (from store validator, empirically quantified): concurrent store.New on a NONEXISTENT db file -\u003e loser fails with SQLITE_BUSY ~57% (busy_timeout does not cover first-creation/WAL-conversion during connection setup; error surfaces via PingContext, wrapped at store.go:77). Steady-state (existing file) is fully clean (93,867 hammered ops, 0 errors). FIX in internal/store: bounded retry (e.g. up to ~2s with small backoff) around the open/ping/migrate sequence in store.New when the error chain matches SQLITE_BUSY, so daemon + ahub status can race first boot safely. Add a two-process fresh-file race test if cheaply arrangeable in-package (two goroutines with separate Stores on one fresh path is enough to reproduce per the validator). Also note in the package doc that busy_timeout excludes creation.","status":"open","priority":1,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-13T00:41:22Z","created_by":"Eugene Blikh","updated_at":"2026-07-13T00:42:04Z"} |
| new_value | {"status":"in_progress"} |
| comment | NULL |
| created_at | 2026-07-13T04:05:54Z |
| id | 019f5919-6f31-7dc3-8294-578d88e2546c |
| issue_id | ah-nyl.12 |
| event_type | closed |
| actor | Eugene Blikh |
| old_value | |
| new_value | landed 8f1c381/4709405/878857f: pagination never-partial guarantees, cache refresh-on-404, db/work_root validation, !!str pinning, sort-order comment, first-boot BUSY retry + migrate TOCTOU serialization (BEGIN IMMEDIATE + in-tx recheck) |
| comment | NULL |
| created_at | 2026-07-13T04:31:06Z |
| id | 019f7cda-9085-795e-84ed-1baf2308465c |
| issue_id | ah-nyl.12 |
| event_type | label_added |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Added label: milestone:stage-1 |
| created_at | 2026-07-20T03:08:46Z |
| id | 019f7cdb-fd6b-71e4-b526-ddcee3931417 |
| issue_id | ah-nyl.12 |
| event_type | label_removed |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Removed label: milestone:stage-1 |
| created_at | 2026-07-20T03:10:19Z |
No comments.
Close reason