~bigbes/tarantool-etcd · parade

main · last commit 1 month ago · 2npkec0r

← Back to the parade

tarantool-etcd-kbc Conformance test: gRPC Watch on a follower replica Lined Up

status: open P2 task milestone:m1-conformance
bd update tarantool-etcd-kbc --claim bd close tarantool-etcd-kbc
Created byEugene Blikh
Ownerbigbes@gmail.com
Created2026-05-19T14:11:02Z
Updated2026-05-19T14:11:02Z
Description
tests/conformance/replicaset_smoke_test.go covers ReadReplicatesToFollowers (Range on each follower after a leader Put) but nothing exercises a Watch stream *pinned to a follower endpoint* while mutations happen on the leader. The Lua integration test (test/grpc_integration_test.lua:547) explicitly punts: "True replicaset behaviors (watch deliveries across nodes ...) need a real Raft cluster and live in the Go harness."

Watch on a follower has its own moving parts that Range does not:
- the watcher fiber lives on the follower
- kv_history rows arrive via replication (relay → applier), not via the local mutation
- the box.on_commit broadcast must fire on the applier path (see memory feedback_tarantool_trigger_broadcast — broadcast from on_commit, not on_replace, so the woken reader sees committed state on the follower side too)

Without this test, a regression in the follower-side watch wake-up would slip through the rs3 suite.
Design
- Reuse mustDial(t, []string{followerEndpoint}) from existing rs3 tests — same dial pattern as ReadReplicatesToFollowers.
- Use a short context.WithTimeout (5 s wall, 2 s wait-for-event) — replication lag on the local rs3 harness is sub-ms.
- Use harness.UniqueKey(t) for key isolation.
- Skip on non-rs3 backend (the single-node tarantool backend has no follower).
- Mirror the synchronous "wait for created chunk" pattern jsonclient/watch already uses, even on the gRPC client — avoids the race where the Put commits before the Watch is registered.
Acceptance criteria
1. New test in tests/conformance/replicaset_smoke_test.go that:
   a. Spawns the rs3 backend, identifies leader + followers via rs.LeaderEndpoint / rs.Endpoints.
   b. Opens a Watch stream pinned to each follower endpoint (one subtest per follower).
   c. Issues a Put on the leader.
   d. Asserts the follower-side watcher receives the PUT event within a short deadline (≤2 s on a quiet box).
   e. Asserts the event header.revision matches the leader-side Put response revision.
2. Additional subtest: range watch ("/rs_smoke/watch/", "\0") on a follower receives multiple events in revision order when the leader issues several Puts.
3. Optional subtest: prev_kv populated correctly on the follower-side event when WithPrevKV is requested.
4. Suite stays green with CONFORMANCE_BACKEND=tarantool_rs3 go test -count=1 -run TestReplicasetSmoke -v.
Notes
Source: gap surfaced while porting docs/TODO.md to bd on 2026-05-19. Cross-refs: tests/conformance/replicaset_smoke_test.go ReadReplicatesToFollowers (line ~160), test/grpc_integration_test.lua:547 comment about which behaviors live where, memory feedback_tarantool_trigger_broadcast.

Depends on

No outgoing dependencies.

Depended on by

Nothing depends on this issue.

No comments.

  • Eugene Blikh created the issue · 2026-05-19T17:11:01Z
  • Eugene Blikh added label milestone:m1-conformance · 2026-05-21T14:37:00Z
Stored rows — what this pane was built from, as read
issues 1 row
id tarantool-etcd-kbc
content_hash 3b334f8edc7b7d1335eb359e0d4f364dd8acf1cbb760567eb695a1b7e940b54f
title Conformance test: gRPC Watch on a follower replica
description tests/conformance/replicaset_smoke_test.go covers ReadReplicatesToFollowers (Range on each follower after a leader Put) but nothing exercises a Watch stream *pinned to a follower endpoint* while mutations happen on the leader. The Lua integration test (test/grpc_integration_test.lua:547) explicitly punts: "True replicaset behaviors (watch deliveries across nodes ...) need a real Raft cluster and live in the Go harness." Watch on a follower has its own moving parts that Range does not: - the watcher fiber lives on the follower - kv_history rows arrive via replication (relay → applier), not via the local mutation - the box.on_commit broadcast must fire on the applier path (see memory feedback_tarantool_trigger_broadcast — broadcast from on_commit, not on_replace, so the woken reader sees committed state on the follower side too) Without this test, a regression in the follower-side watch wake-up would slip through the rs3 suite.
design - Reuse mustDial(t, []string{followerEndpoint}) from existing rs3 tests — same dial pattern as ReadReplicatesToFollowers. - Use a short context.WithTimeout (5 s wall, 2 s wait-for-event) — replication lag on the local rs3 harness is sub-ms. - Use harness.UniqueKey(t) for key isolation. - Skip on non-rs3 backend (the single-node tarantool backend has no follower). - Mirror the synchronous "wait for created chunk" pattern jsonclient/watch already uses, even on the gRPC client — avoids the race where the Put commits before the Watch is registered.
acceptance_criteria 1. New test in tests/conformance/replicaset_smoke_test.go that: a. Spawns the rs3 backend, identifies leader + followers via rs.LeaderEndpoint / rs.Endpoints. b. Opens a Watch stream pinned to each follower endpoint (one subtest per follower). c. Issues a Put on the leader. d. Asserts the follower-side watcher receives the PUT event within a short deadline (≤2 s on a quiet box). e. Asserts the event header.revision matches the leader-side Put response revision. 2. Additional subtest: range watch ("/rs_smoke/watch/", "\0") on a follower receives multiple events in revision order when the leader issues several Puts. 3. Optional subtest: prev_kv populated correctly on the follower-side event when WithPrevKV is requested. 4. Suite stays green with CONFORMANCE_BACKEND=tarantool_rs3 go test -count=1 -run TestReplicasetSmoke -v.
notes Source: gap surfaced while porting docs/TODO.md to bd on 2026-05-19. Cross-refs: tests/conformance/replicaset_smoke_test.go ReadReplicatesToFollowers (line ~160), test/grpc_integration_test.lua:547 comment about which behaviors live where, memory feedback_tarantool_trigger_broadcast.
status open
priority 2
issue_type task
assignee NULL
estimated_minutes NULL
created_at 2026-05-19T14:11:02Z
created_by Eugene Blikh
owner bigbes@gmail.com
updated_at 2026-05-19T14:11:02Z
closed_at NULL
closed_by_session
external_ref NULL
spec_id
compaction_level 0
compacted_at NULL
compacted_at_commit NULL
original_size NULL
sender
ephemeral 0
wisp_type
pinned 0
is_template 0
mol_type
work_type
source_system
metadata <binary>
source_repo
close_reason
event_kind
actor
target
payload
await_type
await_id
timeout_ns 0
waiters
hook_bead
role_bead
agent_state
last_activity NULL
role_type
rig
due_at NULL
defer_until NULL
no_history 0
started_at NULL
is_blocked 0
labels 1 row
issue_id tarantool-etcd-kbc
label milestone:m1-conformance
events 2 rows
id 1bd893f0-4205-55f7-8f79-5f804792e5f7
issue_id tarantool-etcd-kbc
event_type created
actor Eugene Blikh
old_value
new_value
comment NULL
created_at 2026-05-19T17:11:01Z
id c69b7e72-02c3-59d6-b136-100c1d1da170
issue_id tarantool-etcd-kbc
event_type label_added
actor Eugene Blikh
old_value NULL
new_value NULL
comment Added label: milestone:m1-conformance
created_at 2026-05-21T14:37:00Z