~bigbes/tarantool-etcd · parade

main · last commit 1 month ago · 2npkec0r

← Back to the parade

tarantool-etcd-16d Implement Maintenance.Snapshot RPC (streaming DB snapshot) Lined Up

status: open P2 feature milestone:m2-production
bd update tarantool-etcd-16d --claim bd close tarantool-etcd-16d
Created byEugene Blikh
Ownerbigbes@gmail.com
Created2026-05-20T05:47:59Z
Updated2026-07-17T14:27:36Z
Description
The etcd v3 Maintenance.Snapshot RPC streams a full backend snapshot to the client (server-streaming: SnapshotResponse chunks carry remaining_bytes, blob, and a trailing SHA256 hash; etcdctl/clientv3 reassemble into a bolt .db file). Today it is deliberately stubbed: json.lua:13 documents /v3/maintenance/snapshot as 501, and there is no handler in maintenance.lua nor a routes entry in defs.lua (Maintenance methods top out at MoveLeader [5]).

What's needed:
- Decide what a 'snapshot' means for a Tarantool-backed store. Real etcd ships a bolt DB; we'd need to either (a) produce an etcd-compatible bolt snapshot the upstream tooling can restore, or (b) define our own snapshot format (likely a box snapshot / dump of the etcd_* spaces + revision sequence) and document the divergence. (a) is a large undertaking; (b) is the pragmatic path but breaks 'etcdctl snapshot restore' interop.
- Wire the server-streaming method: add SnapshotRequest/SnapshotResponse schemas to defs.lua, a routes entry, a streaming handler in maintenance.lua, and stream support on both wires (gRPC HTTP/2 + JSON gateway chunked).
- Emit remaining_bytes + blob chunks + final hash per the etcd contract.

Document the chosen approach + any divergence in docs/TODO.md and (if it can't fully pass conformance) add to harness/gaps.go.
Notes
Cross-wire streaming: gRPC server-streaming + JSON gateway chunked POST. Mirror existing streaming patterns (Watch). Acceptance: etcdctl snapshot save succeeds against our server, OR documented divergence if we ship a non-bolt format.

Depends on

No outgoing dependencies.

Depended on by

Unblocks — everything waiting on this, transitively

  • tarantool-etcd-7ey — Replica route: connect as read-only replica to an existing etcd cluster related open
  • tarantool-etcd-zws — Replica route: revision alignment with upstream parent-child open
  • tarantool-etcd-hlx — Replica route: watch-tail live sync blocks open
  • tarantool-etcd-75u — Replica route: write handling (reject vs proxy-to-leader) blocks open
  • tarantool-etcd-uee — Replica route: bootstrap snapshot import parent-child open
  • tarantool-etcd-tt6 — Replica route: lease / auth state mirroring (scope v1 vs later) parent-child open
  • tarantool-etcd-jg3 — Replica route: config block + role plumbing parent-child open
  • tarantool-etcd-cnj — [epic] Raft-learner replica — join an etcd cluster as a non-voting member related open
  • tarantool-etcd-umc — Learner: config block + role plumbing parent-child open
  • tarantool-etcd-axs — Learner: rafthttp peer transport over nghttp2 blocks open
  • tarantool-etcd-boq — Learner: raft state machine (follower/learner subset) blocks open
  • tarantool-etcd-6js — Learner: apply loop — InternalRaftRequest -> existing handlers blocks open
  • tarantool-etcd-q5f — Learner: snapshot receive (bbolt parse + bucket translation) blocks open
  • tarantool-etcd-ovo — Learner: durable applied_index (WAL-equivalent for restart) blocks open
  • tarantool-etcd-zkg — Learner: conformance — join real etcd cluster, assert log replication blocks open
  • tarantool-etcd-4yb — Learner: raftpb + InternalRaftRequest proto schemas parent-child open
  • tarantool-etcd-6zn — Learner: MemberAddAsLearner handshake + cluster version negotiation parent-child open

No comments.

  • Eugene Blikh created the issue · 2026-05-20T08:47:58Z
  • Eugene Blikh added label milestone:m2-production · 2026-05-21T14:37:06Z
Stored rows — what this pane was built from, as read
issues 1 row
id tarantool-etcd-16d
content_hash c3ed04edeb2dc3f191d8cd287f502242a4fe98be2442a5343e1a416695f6572a
title Implement Maintenance.Snapshot RPC (streaming DB snapshot)
description The etcd v3 Maintenance.Snapshot RPC streams a full backend snapshot to the client (server-streaming: SnapshotResponse chunks carry remaining_bytes, blob, and a trailing SHA256 hash; etcdctl/clientv3 reassemble into a bolt .db file). Today it is deliberately stubbed: json.lua:13 documents /v3/maintenance/snapshot as 501, and there is no handler in maintenance.lua nor a routes entry in defs.lua (Maintenance methods top out at MoveLeader [5]). What's needed: - Decide what a 'snapshot' means for a Tarantool-backed store. Real etcd ships a bolt DB; we'd need to either (a) produce an etcd-compatible bolt snapshot the upstream tooling can restore, or (b) define our own snapshot format (likely a box snapshot / dump of the etcd_* spaces + revision sequence) and document the divergence. (a) is a large undertaking; (b) is the pragmatic path but breaks 'etcdctl snapshot restore' interop. - Wire the server-streaming method: add SnapshotRequest/SnapshotResponse schemas to defs.lua, a routes entry, a streaming handler in maintenance.lua, and stream support on both wires (gRPC HTTP/2 + JSON gateway chunked). - Emit remaining_bytes + blob chunks + final hash per the etcd contract. Document the chosen approach + any divergence in docs/TODO.md and (if it can't fully pass conformance) add to harness/gaps.go.
design
acceptance_criteria
notes Cross-wire streaming: gRPC server-streaming + JSON gateway chunked POST. Mirror existing streaming patterns (Watch). Acceptance: etcdctl snapshot save succeeds against our server, OR documented divergence if we ship a non-bolt format.
status open
priority 2
issue_type feature
assignee NULL
estimated_minutes NULL
created_at 2026-05-20T05:47:59Z
created_by Eugene Blikh
owner bigbes@gmail.com
updated_at 2026-07-17T14:27:36Z
closed_at NULL
closed_by_session
external_ref NULL
spec_id
compaction_level 0
compacted_at NULL
compacted_at_commit NULL
original_size NULL
sender
ephemeral 0
wisp_type
pinned 0
is_template 0
mol_type
work_type
source_system
metadata <binary>
source_repo
close_reason
event_kind
actor
target
payload
await_type
await_id
timeout_ns 0
waiters
hook_bead
role_bead
agent_state
last_activity NULL
role_type
rig
due_at NULL
defer_until NULL
no_history 0
started_at NULL
is_blocked 0
labels 1 row
issue_id tarantool-etcd-16d
label milestone:m2-production
dependencies 2 rows
id 8d82773f-f842-5a59-b8cd-0485e5c180f0
issue_id tarantool-etcd-7ey
type related
created_at 2026-05-20T09:18:20Z
created_by Eugene Blikh
metadata <binary>
thread_id
depends_on_issue_id tarantool-etcd-16d
depends_on_wisp_id NULL
depends_on_external NULL
id d4f76ef9-87c1-5190-a6ed-3fd8e4154650
issue_id tarantool-etcd-uee
type related
created_at 2026-05-20T09:18:20Z
created_by Eugene Blikh
metadata <binary>
thread_id
depends_on_issue_id tarantool-etcd-16d
depends_on_wisp_id NULL
depends_on_external NULL
events 2 rows
id 8371d325-d005-5ace-a739-97167a9019ab
issue_id tarantool-etcd-16d
event_type created
actor Eugene Blikh
old_value
new_value
comment NULL
created_at 2026-05-20T08:47:58Z
id 8adefbb6-838b-5fb5-9704-dbe7ad4fa000
issue_id tarantool-etcd-16d
event_type label_added
actor Eugene Blikh
old_value NULL
new_value NULL
comment Added label: milestone:m2-production
created_at 2026-05-21T14:37:06Z