main · last commit
13 days ago ·
7g0stsfu
ah-wd4.1 SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate?
Past Stand
bd reopen ah-wd4.1
| Created by | Eugene Blikh |
| Owner | bigbes@gmail.com |
| Created | 2026-07-20T07:48:24Z |
| Started | 2026-08-04T23:36:01Z |
| Updated | 2026-08-04T23:50:55Z |
| Closed | 2026-08-04T23:50:55Z |
GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds. PROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter. WHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99). WHAT TO DETERMINE: 1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)? 2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.) 3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)? 4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent? HOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with >200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee. DELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code.
SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately.
Q1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit > 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit > 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit <= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing.
Q2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included.
Q3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit > 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written.
Q4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically.
VERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, "", IssueFilter{}) is an honest full enumeration. Pending the >600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues.
ah-wd4
— DECISION: move the task board from Vikunja to beads+Dolt (full swap vs hybrid mirror)
parent-child
open
Nothing depends on this issue.
| id | ah-wd4.1 |
| content_hash | 3903707241cf6bc246639a717d12ff22a014c91fc14f4d4363549230335222c4 |
| title | SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate? |
| description | GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds. PROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter. WHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99). WHAT TO DETERMINE: 1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)? 2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.) 3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)? 4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent? HOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with >200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee. DELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code. |
| design | |
| acceptance_criteria | |
| notes | SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately. Q1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit > 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit > 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit <= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing. Q2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included. Q3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit > 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written. Q4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically. VERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, "", IssueFilter{}) is an honest full enumeration. Pending the >600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues. |
| status | closed |
| priority | 2 |
| issue_type | task |
| assignee | NULL |
| estimated_minutes | NULL |
| created_at | 2026-07-20T07:48:24Z |
| created_by | Eugene Blikh |
| owner | bigbes@gmail.com |
| updated_at | 2026-08-04T23:50:55Z |
| closed_at | 2026-08-04T23:50:55Z |
| closed_by_session | |
| external_ref | NULL |
| spec_id | |
| compaction_level | 0 |
| compacted_at | NULL |
| compacted_at_commit | NULL |
| original_size | NULL |
| sender | |
| ephemeral | 0 |
| wisp_type | |
| pinned | 0 |
| is_template | 0 |
| mol_type | |
| work_type | |
| source_system | |
| metadata | �{} |
| source_repo | |
| close_reason | ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata. |
| event_kind | |
| actor | |
| target | |
| payload | |
| await_type | |
| await_id | |
| timeout_ns | 0 |
| waiters | |
| hook_bead | |
| role_bead | |
| agent_state | |
| last_activity | NULL |
| role_type | |
| rig | |
| due_at | NULL |
| defer_until | NULL |
| no_history | 0 |
| started_at | 2026-08-04T23:36:01Z |
| is_blocked | 0 |
| id | 28966c8a-3247-59a5-b7ef-11484e120482 |
| issue_id | ah-wd4.1 |
| type | parent-child |
| created_at | 2026-07-20T10:48:23Z |
| created_by | Eugene Blikh |
| metadata | �{} |
| thread_id | |
| depends_on_issue_id | ah-wd4 |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 019f7e7f-5c6c-7bfe-bdf4-90566080bcc6 |
| issue_id | ah-wd4.1 |
| event_type | created |
| actor | Eugene Blikh |
| old_value | |
| new_value | |
| comment | NULL |
| created_at | 2026-07-20T10:48:23Z |
| id | 019fcf22-526e-71d4-bd23-1770eaaace27 |
| issue_id | ah-wd4.1 |
| event_type | status_changed |
| actor | Eugene Blikh |
| old_value | {"id":"ah-wd4.1","title":"SPIKE (gating): does the beads Storage API enumerate ALL issues, or silently paginate?","description":"GATING UNKNOWN for any beads-backed Board adapter — settle this before writing adapter code, and before ah-wd4 can be decided on technical grounds.\n\nPROBLEM: the beads public API (root package github.com/steveyegge/beads, MIT) has NO plain ListIssues. Methods returning issue lists are: SearchIssues(ctx, query, types.IssueFilter), GetReadyWork(ctx, types.WorkFilter), GetIssuesByIDs, GetIssuesByLabel, GetDependencies/GetDependents, ListWisps. A Snapshot implementation would have to use SearchIssues with an empty query plus a filter.\n\nWHY IT IS GATING: agenthubd treats a card missing from a Snapshot as VANISHED — internal/reconcile/reconcile.go:1659 handleVanished KILLS the live run and marks the record cancelled. A silently truncated enumeration therefore destroys running work rather than merely returning less data. The Vikunja adapter defends against exactly this by refusing a truncated bucket instead of dropping tasks (internal/vikunja/board.go:95-99).\n\nWHAT TO DETERMINE:\n1. Does types.IssueFilter (and WorkFilter) have a Limit/Offset field, and what is the ZERO-VALUE behaviour — unlimited, or a silent default (e.g. 50/100)?\n2. Does SearchIssues with an empty query return every issue regardless of status, including closed/deferred/frozen ones? (bd list hides done/frozen categories by default — check whether that filtering lives in the CLI or in the storage layer.)\n3. Is there a transactional way to enumerate, or a count to cross-check against (GetStatistics returns types.Statistics — does it carry a total issue count usable as a completeness assertion)?\n4. If no guarantee exists: can RunInTransaction wrap a count + list so the pair is consistent?\n\nHOW: source is at github.com/steveyegge/beads v1.1.0 (proxy.golang.org zip; module root has beads.go re-exporting internal via type aliases). Read internal/types for IssueFilter/WorkFilter and internal/storage/dolt for the SearchIssues SQL. Then PROVE it empirically with a throwaway Go program against a COPY of the DB (copy .beads/embeddeddolt elsewhere; do not write to the live one) — this repo has 84+ issues, so create a scratch DB with \u003e200 to expose a default limit. Report the exact zero-value semantics and the recommended Snapshot implementation with its completeness guarantee.\n\nDELIVERABLE: a written answer to 1-4 plus the recommended Snapshot approach. Do NOT write adapter code.","status":"open","priority":2,"issue_type":"task","owner":"bigbes@gmail.com","created_at":"2026-07-20T07:48:24Z","created_by":"Eugene Blikh","updated_at":"2026-07-20T07:48:24Z"} |
| new_value | {"notes":"SOURCE ANALYSIS DONE (beads v1.1.2, the version installed locally; clone checked out at tag 20e493e5). Empirical confirmation is running separately.\n\nQ1 — Limit zero-value. types.IssueFilter has a plain 'Limit int' (no Offset; keyset pagination instead, via AfterCreatedAt/AfterID over the (created_at DESC, id ASC) total order). The SQL builder gates the clause on 'filter.Limit \u003e 0' — internal/storage/issueops/search.go:100-110 emits NO LIMIT at all for the zero value and takes the branch its own comment calls 'Pattern A: full 47-column scan (used for unlimited queries)'. Limit \u003e 0 instead takes 'Pattern B', a cheap SELECT id + LIMIT then batch hydration. So the zero value is UNLIMITED, not a silent default page size. GetReadyWork likewise has an explicit unlimited branch for Limit \u003c= 0 (issueops/ready_work.go:165) rather than a loop that would return nothing.\n\nQ2 — status visibility. The closed-hiding is a CLI behaviour, NOT a storage behaviour: cmd/bd/list_filter.go:157, cmd/bd/search.go:114, cmd/bd/query.go:136 and cmd/bd/gate*.go are what set ExcludeStatus. sqlbuild.BuildIssueFilterClauses (internal/storage/sqlbuild/filter.go:53-72) emits a status predicate ONLY when Status, Statuses or ExcludeStatus is explicitly populated. A zero-value IssueFilter therefore returns every status, closed and deferred included.\n\nQ3 — completeness cross-check, with a trap. types.Statistics (types.go:1181) carries TotalIssues, so a count exists. BUT SearchIssuesInTx (issueops/search.go:18-78) queries the 'issues' table AND merges the 'wisps' table unless filter.SkipWisps, preferring the wisp record on an ID collision — so a naive len(result)==TotalIssues assertion is only valid if TotalIssues counts the same union. Being verified empirically. Second trap in the same function: when Limit \u003e 0 the limit is applied to EACH table separately before the merge, so a limited query can return MORE rows than Limit. Neither trap bites a Limit=0 snapshot, but both must be documented in whatever adapter gets written.\n\nQ4 — RunInTransaction exists on the public Storage surface (documented in beads.go:23; the root package re-exports the internal types by alias, e.g. IssueFilter = types.IssueFilter), so a count+list pair can be wrapped in one transaction. Being verified empirically.\n\nVERDICT SO FAR: the gating fear (silent pagination destroying live runs via handleVanished) does NOT appear to be real at the source level — SearchIssues(ctx, \"\", IssueFilter{}) is an honest full enumeration. Pending the \u003e600-issue scale test that proves no default page size hides at a larger corpus than this repo's 86 issues.","status":"in_progress"} |
| comment | NULL |
| created_at | 2026-08-05T02:36:00Z |
| id | 019fcf2f-f560-72f5-956a-81e2dedb09c5 |
| issue_id | ah-wd4.1 |
| event_type | closed |
| actor | Eugene Blikh |
| old_value | |
| new_value | ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata. |
| comment | NULL |
| created_at | 2026-08-05T02:50:54Z |
No comments.
Close reason
ANSWERED — the gating fear is NOT real. beads v1.1.2, verified by source reading AND by measurement against copies of the live DB plus scratch DBs of 700 and 2500 issues. Q1 Limit zero-value: UNLIMITED, no silent default. IssueFilter.Limit is a plain int and the SQL builder gates the clause on Limit > 0 (issueops/search.go:100-110, 'Pattern A: full scan (used for unlimited queries)'). Measured whole: 87/87 on the live copy, 700/700, 2500/2500. Explicit limits track exactly and saturate only at the true row count — 1,5,50,100,250,500,1000 all exact on the 2500-row DB, so there is no round-number cap hiding anywhere. IterIssues drains identically. WorkFilter{Limit:0} and {Limit:-1} both return the full 240 ready set, so GetReadyWork has a real unlimited branch too. NEW FINDING NOT IN THE BRIEF: IssueFilter.Offset is DEAD on this read path — grep finds zero uses in issueops/ and sqlbuild/, and empirically Offset:0/5/10 with Limit:5 all return page 0. A hand-rolled Limit+Offset pager against beads would loop on page 0 forever. Only the internal/storage/domain/db builder honours Offset. Q2 status visibility: an empty query returns EVERY status, unconditionally. Live copy: closed=65, in_progress=9, open=13. Scale DB: 140 each of blocked/closed/deferred/in_progress/open. The hiding is entirely CLI-side (cmd/bd/list_filter.go:150-159, which also excludes pinned and every custom status categorised done or frozen, and only when --status/--all/--ready/--pinned are all absent). Same DB: 'bd list --json' 560, 'bd list --all -n 0' 700, API zero-filter 740 — the last gap being wisps, which the CLI list path skips. Q3 completeness cross-check — the suspicion was RIGHT and it is not theoretical. Statistics.TotalIssues is 'SELECT COUNT(*) FROM issues' and nothing else (issueops/statistics.go:16-32); the wisps table is never touched. On a 700-issue + 40-wisp DB: SearchIssues=740, CountIssues=740, TotalIssues=700. So len(result)==stats.TotalIssues is WRONG and passes on agent-hub today only because our wisps table is empty — a latent bug the current data hides. The correct cross-check is Storage.CountIssues, explicitly built to mirror SearchIssues's wisps-merge semantics ('GH#4387 count/list parity contract'); parity verified across 15 filter shapes including label-driven DISTINCT paths. Two hazards measured: (i) with Limit > 0 the limit is applied PER TABLE then merged, so Limit:100 returns 140; (ii) a cross-table duplicate ID makes CountIssues exceed SearchIssues by exactly the number of dups (741 vs 740) — and the comment at count.go:52-58 claiming SearchIssuesInTx 'errors loudly' on that is STALE for v1.1.2: search.go:60-72 silently prefers the wisp record. Q4 RunInTransaction: NO, not through the public API. Two independent blockers, both tested. storage.Transaction exposes SearchIssues but no CountIssues, no GetStatistics, no Iter* — one enumeration primitive and nothing to check it against. And embedded Dolt holds a process-exclusive lock for the transaction's duration, so store-level calls from inside the callback fail with 'the database is locked by another dolt process'. A transaction does give an atomic snapshot (read-your-writes confirmed, clean rollback, zero extra Dolt commits), but the snapshot cannot be independently verified from within it. RECOMMENDED SNAPSHOT (see the bead's design field for the full form): build the filter in ONE place with Limit and Offset pinned to 0 and never caller-supplied; do count → list → count; assert (n==len(list) || n2==len(list)) and that IDs are unique; on failure REFUSE with an error rather than returning a short list, exactly as internal/vikunja/board.go:95-99 does — reconcile.go handleVanished kills the live run for any record missing from a snapshot, and ports.go already specifies that a Snapshot error makes the reconciler skip the whole iteration, so a refusal costs one poll interval while a short list destroys running work. Do NOT set SkipWisps by default (NoHistory beads live in wisps with ephemeral=0 and would become invisible ⇒ killed runs) and do NOT wrap the snapshot in RunInTransaction (no verifiability gained, and a process-exclusive lock on a poll loop starves every other store call). Perf is a non-issue at board scale: 126 ms to list 2500 rows, 59 ms for 740, 34 ms to count. Build note for whoever writes the adapter: the beads module needs CGO_ENABLED=1 AND ICU headers (go-icu-regex fails with 'unicode/regex.h' not found otherwise) — on this mac, CGO_CFLAGS/CXXFLAGS=-I/opt/homebrew/opt/icu4c@77/include and CGO_LDFLAGS=-L.../lib -licuuc -licui18n -licudata.