~bigbes/agents-dev · parade

main · last commit 13 days ago · 7g0stsfu

← Back to the parade

ah-wd4.1 SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate? Past Stand

status: closed P2 task
bd reopen ah-wd4.1
Created byEugene Blikh
Ownerbigbes@gmail.com
Created2026-07-20T07:48:24Z
Started2026-08-04T23:36:01Z
Updated2026-08-04T23:50:55Z
Closed2026-08-04T23:50:55Z
Description
GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds.

PROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter.

WHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99).

WHAT TO DETERMINE:
1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)?
2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.)
3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)?
4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent?

HOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with >200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee.

DELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code.
Notes
SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately.

Q1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit > 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit > 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit <= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing.

Q2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included.

Q3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit > 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written.

Q4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically.

VERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, "", IssueFilter{}) is an honest full enumeration. Pending the >600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues.

Depends on

  • ah-wd4 — DECISION: move the task board from Vikunja to beads+Dolt (full swap vs hybrid mirror) parent-child open

Depended on by

Nothing depends on this issue.

No comments.

Close reason

ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues.

Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too.
NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset.

Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips.

Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record.

Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it.

RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count.

Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata.
  • Eugene Blikh added under epic ah-wd4 · 2026-07-20T10:48:23Z
  • Eugene Blikh created the issue · 2026-07-20T10:48:23Z
  • Eugene Blikh changed status to in_progress · 2026-08-05T02:36:00Z
  • Eugene Blikh closed the issue · 2026-08-05T02:50:54Z
    ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata.
Stored rows — what this pane was built from, as read
issues 1 row
id ah-wd4.1
content_hash 3903707241cf6bc246639a717d12ff22a014c91fc14f4d4363549230335222c4
title SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate?
description GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds. PROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter. WHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99). WHAT TO DETERMINE: 1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)? 2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.) 3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)? 4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent? HOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with >200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee. DELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code.
design
acceptance_criteria
notes SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately. Q1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit > 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit > 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit <= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing. Q2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included. Q3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit > 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written. Q4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically. VERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, "", IssueFilter{}) is an honest full enumeration. Pending the >600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues.
status closed
priority 2
issue_type task
assignee NULL
estimated_minutes NULL
created_at 2026-07-20T07:48:24Z
created_by Eugene Blikh
owner bigbes@gmail.com
updated_at 2026-08-04T23:50:55Z
closed_at 2026-08-04T23:50:55Z
closed_by_session
external_ref NULL
spec_id
compaction_level 0
compacted_at NULL
compacted_at_commit NULL
original_size NULL
sender
ephemeral 0
wisp_type
pinned 0
is_template 0
mol_type
work_type
source_system
metadata �{}
source_repo
close_reason ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata.
event_kind
actor
target
payload
await_type
await_id
timeout_ns 0
waiters
hook_bead
role_bead
agent_state
last_activity NULL
role_type
rig
due_at NULL
defer_until NULL
no_history 0
started_at 2026-08-04T23:36:01Z
is_blocked 0
dependencies 1 row
id 28966c8a-3247-59a5-b7ef-11484e120482
issue_id ah-wd4.1
type parent-child
created_at 2026-07-20T10:48:23Z
created_by Eugene Blikh
metadata �{}
thread_id
depends_on_issue_id ah-wd4
depends_on_wisp_id NULL
depends_on_external NULL
events 3 rows
id 019f7e7f-5c6c-7bfe-bdf4-90566080bcc6
issue_id ah-wd4.1
event_type created
actor Eugene Blikh
old_value
new_value
comment NULL
created_at 2026-07-20T10:48:23Z
id 019fcf22-526e-71d4-bd23-1770eaaace27
issue_id ah-wd4.1
event_type status_changed
actor Eugene Blikh
old_value {"id":"ah-wd4.1","title":"SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate?","description":"GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds.\n\nPROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter.\n\nWHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99).\n\nWHAT TO DETERMINE:\n1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)?\n2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.)\n3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)?\n4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent?\n\nHOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with \u003e200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee.\n\nDELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-20T07:48:24Z","created_by":"Eugene Blikh","updated_at":"2026-07-20T07:48:24Z"}
new_value {"notes":"SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately.\n\nQ1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit \u003e 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit \u003e 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit \u003c= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing.\n\nQ2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included.\n\nQ3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit \u003e 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written.\n\nQ4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically.\n\nVERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, \"\", IssueFilter{}) is an honest full enumeration. Pending the \u003e600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues.","status":"in_progress"}
comment NULL
created_at 2026-08-05T02:36:00Z
id 019fcf2f-f560-72f5-956a-81e2dedb09c5
issue_id ah-wd4.1
event_type closed
actor Eugene Blikh
old_value
new_value ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata.
comment NULL
created_at 2026-08-05T02:50:54Z