~bigbes/tarantool-etcd · parade

main · last commit 1 month ago · 2npkec0r

← Back to the parade

tarantool-etcd-8ss Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration) Past Stand

status: closed P2 feature @Eugene Blikh
bd reopen tarantool-etcd-8ss
Created byEugene Blikh
Ownerbigbes@gmail.com
Created2026-05-20T11:16:17Z
Started2026-05-20T11:16:28Z
Updated2026-05-20T11:38:01Z
Closed2026-05-20T11:38:01Z
Description
Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.
Notes
VERIFIED end-to-end on EE 3.7.0 (up-file/seed/migrate/probe/reload all pass; migrate idempotent). Root cause of the user's failure: EE config.etcd source pins to Endpoints[0] without failover, so endpoints must be LEADER-FIRST (not peers-first). Also fixed: leader-migrated-last via per-step recompute, _wait-healthy gate, and a pipefail bug (grep -vx on last node exited 1 after work was done). Example complete.

Depends on

No outgoing dependencies.

Depended on by

Nothing depends on this issue.

No comments.

Close reason

Example written and verified end-to-end on Tarantool EE 3.7.0.
  • Eugene Blikh created the issue · 2026-05-20T14:16:16Z
  • Eugene Blikh claimed · 2026-05-20T14:16:28Z
  • Eugene Blikh updated notes to Files written + phases 1-2 (up-file/seed/probe/config-show) verified on Community Tarantool 3.8. Ports 3501-3503/2382-2482-2582 (distinct from example-1's 3401/2379). migrate+reload need an EE binary (config.etcd source) so were not run here; mechanics mirror the working tarantool-ee-config example. EE guard on migrate verified to fire. · 2026-05-20T14:22:44Z
  • Eugene Blikh updated notes to v2: hardened migrate after EE run hit 'etcdserver: no leader' on the 2nd node (election churn from restarting a node that became leader). Fixes: (1) recompute leader each step, always migrate a follower, leader last; (2) _wait-healthy gate (all live nodes agree on one leader) between steps; (3) endpoints peers-first/self-last; (4) append logs not truncate. Rolling-restart pacing + leader-pick logic verified on Community; EE etcd-source fetch still needs an EE re-run to confirm. · 2026-05-20T14:30:21Z
  • Eugene Blikh updated notes to VERIFIED end-to-end on EE 3.7.0 (up-file/seed/migrate/probe/reload all pass; migrate idempotent). Root cause of the user's failure: EE config.etcd source pins to Endpoints[0] without failover, so endpoints must be LEADER-FIRST (not peers-first). Also fixed: leader-migrated-last via per-step recompute, _wait-healthy gate, and a pipefail bug (grep -vx on last node exited 1 after work was done). Example complete. · 2026-05-20T14:38:01Z
  • Eugene Blikh closed the issue · 2026-05-20T14:38:01Z
    Example written and verified end-to-end on Tarantool EE 3.7.0.
Stored rows — what this pane was built from, as read
issues 1 row
id tarantool-etcd-8ss
content_hash 848bb73c6cac418f63bae8d196061ccb5bca2ddd670d7fc794f0dde560c28cab
title Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration)
description Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.
design
acceptance_criteria
notes VERIFIED end-to-end on EE 3.7.0 (up-file/seed/migrate/probe/reload all pass; migrate idempotent). Root cause of the user's failure: EE config.etcd source pins to Endpoints[0] without failover, so endpoints must be LEADER-FIRST (not peers-first). Also fixed: leader-migrated-last via per-step recompute, _wait-healthy gate, and a pipefail bug (grep -vx on last node exited 1 after work was done). Example complete.
status closed
priority 2
issue_type feature
assignee Eugene Blikh
estimated_minutes NULL
created_at 2026-05-20T11:16:17Z
created_by Eugene Blikh
owner bigbes@gmail.com
updated_at 2026-05-20T11:38:01Z
closed_at 2026-05-20T11:38:01Z
closed_by_session
external_ref NULL
spec_id
compaction_level 0
compacted_at NULL
compacted_at_commit NULL
original_size NULL
sender
ephemeral 0
wisp_type
pinned 0
is_template 0
mol_type
work_type
source_system
metadata <binary>
source_repo
close_reason Example written and verified end-to-end on Tarantool EE 3.7.0.
event_kind
actor
target
payload
await_type
await_id
timeout_ns 0
waiters
hook_bead
role_bead
agent_state
last_activity NULL
role_type
rig
due_at NULL
defer_until NULL
no_history 0
started_at 2026-05-20T11:16:28Z
is_blocked 0
events 6 rows
id 7a595ffb-8f3d-5b9e-a441-7a24d36f7931
issue_id tarantool-etcd-8ss
event_type updated
actor Eugene Blikh
old_value {"id":"tarantool-etcd-8ss","title":"Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration)","description":"Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.","status":"in_progress","priority":2,"issue_type":"feature","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-20T11:16:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-20T11:16:28Z","started_at":"2026-05-20T11:16:28Z"}
new_value {"notes":"Files written + phases 1-2 (up-file/seed/probe/config-show) verified on Community Tarantool 3.8. Ports 3501-3503/2382-2482-2582 (distinct from example-1's 3401/2379). migrate+reload need an EE binary (config.etcd source) so were not run here; mechanics mirror the working tarantool-ee-config example. EE guard on migrate verified to fire."}
comment NULL
created_at 2026-05-20T14:22:44Z
id 83edfeeb-7a05-588a-a6c0-a8ef5541250f
issue_id tarantool-etcd-8ss
event_type updated
actor Eugene Blikh
old_value {"id":"tarantool-etcd-8ss","title":"Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration)","description":"Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.","notes":"v2: hardened migrate after EE run hit 'etcdserver: no leader' on the 2nd node (election churn from restarting a node that became leader). Fixes: (1) recompute leader each step, always migrate a follower, leader last; (2) _wait-healthy gate (all live nodes agree on one leader) between steps; (3) endpoints peers-first/self-last; (4) append logs not truncate. Rolling-restart pacing + leader-pick logic verified on Community; EE etcd-source fetch still needs an EE re-run to confirm.","status":"in_progress","priority":2,"issue_type":"feature","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-20T11:16:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-20T11:30:21Z","started_at":"2026-05-20T11:16:28Z"}
new_value {"notes":"VERIFIED end-to-end on EE 3.7.0 (up-file/seed/migrate/probe/reload all pass; migrate idempotent). Root cause of the user's failure: EE config.etcd source pins to Endpoints[0] without failover, so endpoints must be LEADER-FIRST (not peers-first). Also fixed: leader-migrated-last via per-step recompute, _wait-healthy gate, and a pipefail bug (grep -vx on last node exited 1 after work was done). Example complete."}
comment NULL
created_at 2026-05-20T14:38:01Z
id 8e61e59e-dfbe-5aad-a6cc-52102b681d64
issue_id tarantool-etcd-8ss
event_type updated
actor Eugene Blikh
old_value {"id":"tarantool-etcd-8ss","title":"Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration)","description":"Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.","notes":"Files written + phases 1-2 (up-file/seed/probe/config-show) verified on Community Tarantool 3.8. Ports 3501-3503/2382-2482-2582 (distinct from example-1's 3401/2379). migrate+reload need an EE binary (config.etcd source) so were not run here; mechanics mirror the working tarantool-ee-config example. EE guard on migrate verified to fire.","status":"in_progress","priority":2,"issue_type":"feature","assignee":"Eugene Blikh","owner":"bigbes@gmail.com","created_at":"2026-05-20T11:16:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-20T11:22:45Z","started_at":"2026-05-20T11:16:28Z"}
new_value {"notes":"v2: hardened migrate after EE run hit 'etcdserver: no leader' on the 2nd node (election churn from restarting a node that became leader). Fixes: (1) recompute leader each step, always migrate a follower, leader last; (2) _wait-healthy gate (all live nodes agree on one leader) between steps; (3) endpoints peers-first/self-last; (4) append logs not truncate. Rolling-restart pacing + leader-pick logic verified on Community; EE etcd-source fetch still needs an EE re-run to confirm."}
comment NULL
created_at 2026-05-20T14:30:21Z
id c59c84bd-94d1-5ad4-9166-ee94910aadb8
issue_id tarantool-etcd-8ss
event_type claimed
actor Eugene Blikh
old_value {"id":"tarantool-etcd-8ss","title":"Example 2: 3-node EE replicaset self-hosting its own etcd config (file → etcd migration)","description":"Add examples/tarantool-ee-self-hosted-etcd: boot a 3-instance EE replicaset (election failover) from a local cluster-config.yaml where every node runs app.roles.etcd (replicated KV store, read_pref=any). Seed the cluster's OWN etcd store with that config, then rolling-restart each instance to bootstrap from TT_CONFIG_ETCD_* (no --config). Document the cold-boot circularity: self-hosted etcd-config is HA under rolling restart but needs the local file retained as a cold-boot seed. Justfile + README + cluster-config.yaml.","status":"open","priority":2,"issue_type":"feature","owner":"bigbes@gmail.com","created_at":"2026-05-20T11:16:17Z","created_by":"Eugene Blikh","updated_at":"2026-05-20T11:16:17Z"}
new_value {"assignee":"Eugene Blikh","status":"in_progress"}
comment NULL
created_at 2026-05-20T14:16:28Z
id cf901dca-197b-5b51-9755-0f738ae2a0a0
issue_id tarantool-etcd-8ss
event_type created
actor Eugene Blikh
old_value
new_value
comment NULL
created_at 2026-05-20T14:16:16Z
id d26de571-bf81-5a27-b0c2-e5968bca9515
issue_id tarantool-etcd-8ss
event_type closed
actor Eugene Blikh
old_value
new_value Example written and verified end-to-end on Tarantool EE 3.7.0.
comment NULL
created_at 2026-05-20T14:38:01Z