main · last commit
1 month ago ·
2npkec0r
tarantool-etcd-7ey Replica route: connect as read-only replica to an existing etcd cluster
Lined Up
bd update tarantool-etcd-7ey --claim
bd close tarantool-etcd-7ey
| Created by | Eugene Blikh |
| Owner | bigbes@gmail.com |
| Created | 2026-05-20T05:48:15Z |
| Updated | 2026-07-17T14:27:36Z |
Add a mode where this server attaches to an existing/real etcd cluster as a read-only replica (mirror), serving reads locally while transparently keeping in sync with the upstream etcd. Motivation: lets the Tarantool store act as a read scale-out / local cache in front of a real etcd deployment, and is a stepping stone toward migration (shadow the production cluster, compare, then cut over). Design questions to resolve: - Sync mechanism: use etcd's Watch (clientv3) from revision 0 to stream the full keyspace + tail live, persisting events into our etcd_* spaces and advancing our revision counter to match upstream. Need an initial Range snapshot (or Snapshot RPC) for bootstrap, then Watch with progress notifications + compaction handling (ErrCompacted -> re-snapshot). - Revision alignment: our global revision sequence must track upstream's mod_revision exactly so reads return etcd-consistent revisions. This constrains the local revision allocator while in replica mode. - Write handling: reject writes locally with a redirect/error, OR proxy them to the upstream leader and wait for the change to come back over the watch (read-your-writes). Pick one; proxying is closer to etcd's learner/proxy semantics. - Lease/Watch/Auth: leases and auth state also need mirroring or pass-through. Scope what's in v1 (KV-only read replica) vs later. - Config: new role config block (e.g. replica.upstream_endpoints, replica.mode = mirror|proxy, credentials). Likely a multi-phase epic; this ticket captures the umbrella. Split bootstrap-snapshot, watch-tail-sync, revision-alignment, and write-proxying into child issues once the design is settled.
Reuses clientv3 Watch semantics we already understand from conformance work (ErrCompacted=11, progress notify). Consider relation to the Tarantool-native replicaset path (tarantool_rs3) — this is the opposite direction (follow a foreign etcd, not our own Raft).
tarantool-etcd-16d
— Implement Maintenance.Snapshot RPC (streaming DB snapshot)
related
open
tarantool-etcd-zws
— Replica route: revision alignment with upstream
parent-child
tarantool-etcd-75u
— Replica route: write handling (reject vs proxy-to-leader)
parent-child
tarantool-etcd-uee
— Replica route: bootstrap snapshot import
parent-child
tarantool-etcd-tt6
— Replica route: lease / auth state mirroring (scope v1 vs later)
parent-child
tarantool-etcd-jg3
— Replica route: config block + role plumbing
parent-child
tarantool-etcd-cnj
— [epic] Raft-learner replica — join an etcd cluster as a non-voting member
related
tarantool-etcd-hlx
— Replica route: watch-tail live sync
parent-child
tarantool-etcd-zws
— Replica route: revision alignment with upstream
parent-child
open
tarantool-etcd-hlx
— Replica route: watch-tail live sync
blocks
open
tarantool-etcd-75u
— Replica route: write handling (reject vs proxy-to-leader)
blocks
open
tarantool-etcd-uee
— Replica route: bootstrap snapshot import
parent-child
open
tarantool-etcd-tt6
— Replica route: lease / auth state mirroring (scope v1 vs later)
parent-child
open
tarantool-etcd-jg3
— Replica route: config block + role plumbing
parent-child
open
tarantool-etcd-cnj
— [epic] Raft-learner replica — join an etcd cluster as a non-voting member
related
open
tarantool-etcd-umc
— Learner: config block + role plumbing
parent-child
open
tarantool-etcd-axs
— Learner: rafthttp peer transport over nghttp2
blocks
open
tarantool-etcd-boq
— Learner: raft state machine (follower/learner subset)
blocks
open
tarantool-etcd-6js
— Learner: apply loop — InternalRaftRequest -> existing handlers
blocks
open
tarantool-etcd-q5f
— Learner: snapshot receive (bbolt parse + bucket translation)
blocks
open
tarantool-etcd-zkg
— Learner: conformance — join real etcd cluster, assert log replication
blocks
open
tarantool-etcd-ovo
— Learner: durable applied_index (WAL-equivalent for restart)
blocks
open
tarantool-etcd-4yb
— Learner: raftpb + InternalRaftRequest proto schemas
parent-child
open
tarantool-etcd-6zn
— Learner: MemberAddAsLearner handshake + cluster version negotiation
parent-child
open
| id | tarantool-etcd-7ey |
| content_hash | be7b04f2cbf862eb22cfe43df8812d0bcb20c7b247f8edc485850c3fb44cbe34 |
| title | Replica route: connect as read-only replica to an existing etcd cluster |
| description | Add a mode where this server attaches to an existing/real etcd cluster as a read-only replica (mirror), serving reads locally while transparently keeping in sync with the upstream etcd. Motivation: lets the Tarantool store act as a read scale-out / local cache in front of a real etcd deployment, and is a stepping stone toward migration (shadow the production cluster, compare, then cut over). Design questions to resolve: - Sync mechanism: use etcd's Watch (clientv3) from revision 0 to stream the full keyspace + tail live, persisting events into our etcd_* spaces and advancing our revision counter to match upstream. Need an initial Range snapshot (or Snapshot RPC) for bootstrap, then Watch with progress notifications + compaction handling (ErrCompacted -> re-snapshot). - Revision alignment: our global revision sequence must track upstream's mod_revision exactly so reads return etcd-consistent revisions. This constrains the local revision allocator while in replica mode. - Write handling: reject writes locally with a redirect/error, OR proxy them to the upstream leader and wait for the change to come back over the watch (read-your-writes). Pick one; proxying is closer to etcd's learner/proxy semantics. - Lease/Watch/Auth: leases and auth state also need mirroring or pass-through. Scope what's in v1 (KV-only read replica) vs later. - Config: new role config block (e.g. replica.upstream_endpoints, replica.mode = mirror|proxy, credentials). Likely a multi-phase epic; this ticket captures the umbrella. Split bootstrap-snapshot, watch-tail-sync, revision-alignment, and write-proxying into child issues once the design is settled. |
| design | |
| acceptance_criteria | |
| notes | Reuses clientv3 Watch semantics we already understand from conformance work (ErrCompacted=11, progress notify). Consider relation to the Tarantool-native replicaset path (tarantool_rs3) — this is the opposite direction (follow a foreign etcd, not our own Raft). |
| status | open |
| priority | 2 |
| issue_type | feature |
| assignee | NULL |
| estimated_minutes | NULL |
| created_at | 2026-05-20T05:48:15Z |
| created_by | Eugene Blikh |
| owner | bigbes@gmail.com |
| updated_at | 2026-07-17T14:27:36Z |
| closed_at | NULL |
| closed_by_session | |
| external_ref | NULL |
| spec_id | |
| compaction_level | 0 |
| compacted_at | NULL |
| compacted_at_commit | NULL |
| original_size | NULL |
| sender | |
| ephemeral | 0 |
| wisp_type | |
| pinned | 0 |
| is_template | 0 |
| mol_type | |
| work_type | |
| source_system | |
| metadata | <binary> |
| source_repo | |
| close_reason | |
| event_kind | |
| actor | |
| target | |
| payload | |
| await_type | |
| await_id | |
| timeout_ns | 0 |
| waiters | |
| hook_bead | |
| role_bead | |
| agent_state | |
| last_activity | NULL |
| role_type | |
| rig | |
| due_at | NULL |
| defer_until | NULL |
| no_history | 0 |
| started_at | NULL |
| is_blocked | 0 |
| issue_id | tarantool-etcd-7ey |
| label | milestone:m5-clustering |
| id | 0ec497b5-b954-52f2-853f-28088b75001e |
| issue_id | tarantool-etcd-zws |
| type | parent-child |
| created_at | 2026-05-20T09:18:15Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 48652ac8-6e0e-5561-a798-7f3235f74ba3 |
| issue_id | tarantool-etcd-75u |
| type | parent-child |
| created_at | 2026-05-20T09:18:16Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 5a814ebe-5453-5f01-9812-89df538a0702 |
| issue_id | tarantool-etcd-uee |
| type | parent-child |
| created_at | 2026-05-20T09:18:14Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 5f327e65-dfdc-54aa-9495-f350a54e3af9 |
| issue_id | tarantool-etcd-tt6 |
| type | parent-child |
| created_at | 2026-05-20T09:18:16Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 80a097c8-8dcf-53ed-9a90-909694bcb010 |
| issue_id | tarantool-etcd-jg3 |
| type | parent-child |
| created_at | 2026-05-20T09:18:13Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 8d82773f-f842-5a59-b8cd-0485e5c180f0 |
| issue_id | tarantool-etcd-7ey |
| type | related |
| created_at | 2026-05-20T09:18:20Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-16d |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 9192386c-c056-5244-abe5-cbbb0bede32b |
| issue_id | tarantool-etcd-cnj |
| type | related |
| created_at | 2026-05-20T09:47:29Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | e59ef424-b68e-5f66-a10f-627c82f52942 |
| issue_id | tarantool-etcd-hlx |
| type | parent-child |
| created_at | 2026-05-20T09:18:15Z |
| created_by | Eugene Blikh |
| metadata | <binary> |
| thread_id | |
| depends_on_issue_id | tarantool-etcd-7ey |
| depends_on_wisp_id | NULL |
| depends_on_external | NULL |
| id | 704155e7-0669-5bfa-8777-08c8a01e0a06 |
| issue_id | tarantool-etcd-7ey |
| event_type | created |
| actor | Eugene Blikh |
| old_value | |
| new_value | |
| comment | NULL |
| created_at | 2026-05-20T08:48:14Z |
| id | d0d26b03-32e3-55e5-88d4-eff502e16069 |
| issue_id | tarantool-etcd-7ey |
| event_type | label_added |
| actor | Eugene Blikh |
| old_value | NULL |
| new_value | NULL |
| comment | Added label: milestone:m5-clustering |
| created_at | 2026-05-21T14:37:43Z |
No comments.