Overview
WhatsApp-Rust is a high-performance, async Rust library for the WhatsApp Web API. The project follows a modular, layered architecture that separates protocol concerns from runtime concerns, enabling platform-agnostic core logic with pluggable backends.Workspace Structure
The project is organized as a Cargo workspace with multiple crates:Three main crates
wacore - platform-agnostic core
Location:wacore/
Purpose: Contains core logic for the WhatsApp binary protocol, cryptography primitives, IQ protocol types, runtime abstraction, and state management traits.
Key Features:
- Zero runtime dependencies — no Tokio, no async-std, only
futures,async-trait,async-lock, andasync-channel - 32-bit target support — uses
portable-atomicfor 64-bit atomics with a software fallback on platforms without nativeAtomicU64(ARM32, MIPS, etc.) Runtimetrait for pluggable async executors (Tokio, async-std, WASM, etc.)Transport,TransportFactory, andHttpClienttraits for pluggable networkingBackendtrait for pluggable storage- Cryptographic operations (Signal Protocol, Noise Protocol)
- Type-safe protocol node builders
waproto - protocol buffers
Location:waproto/
Purpose: Houses WhatsApp’s Protocol Buffers definitions compiled to Rust structs.
Build Process:
build.rs always runs, reads the committed binary descriptor (src/whatsapp.desc), verifies its SHA-256, and writes whatsapp.rs and tags.rs into OUT_DIR. Neither generated file is committed. To regenerate after modifying whatsapp.proto:
Message- All message typesWebMessageInfo- Message metadataHistorySync- Chat historySyncActionValue- App state mutations
Whatsapp-rust - main client
Location:src/
Purpose: Integrates wacore with concrete implementations (Tokio runtime, SQLite storage, ureq HTTP, Tokio WebSocket), provides the high-level Bot builder and Client API.
Key Features:
TokioRuntime— defaultRuntimeimplementation (gated ontokio-runtimefeature)- Typestate
BotBuilder— compile-time enforcement that all 4 required components are provided - SQLite persistence (pluggable via
Backendtrait) - Event bus system
- Feature modules (groups, media, newsletters, communities, etc.)
Runtime abstraction
The library is fully runtime-agnostic. All async operations go through four pluggable trait abstractions defined inwacore:
The
Runtime trait requires four methods plus two optional methods with defaults:
AbortHandle is #[must_use] — dropping the handle aborts the spawned task. Call .detach() on the handle for fire-and-forget tasks that should run to completion independently. See custom backends — AbortHandle for implementation details.
As of #1124, spawn_detached exists so fire-and-forget callers no longer have to write spawn(future).detach() themselves. The trait-level default above is exactly that call: self.spawn(future).detach(). It still builds the AbortHandle and immediately drops it, so the default alone saves nothing. That AbortHandle boxes a dyn FnOnce capturing the executor’s own handle — the cost this method exists to avoid. The saving comes from TokioRuntime’s own override, which calls tokio::spawn(future) directly and never constructs an AbortHandle at all. #1124 migrated two of the library’s own fire-and-forget spawns to spawn_detached: the event-callback dispatcher in bot.rs, and the read loop’s spawned-processing branch (see WebSocket handling). Those two pick up the saving automatically under the bundled TokioRuntime. Override spawn_detached yourself only if your executor can spawn a task with no cancellation bookkeeping at all.
The yield_frequency() method controls how often the client cooperatively yields during tight async loops (such as processing incoming frames). It returns the number of items to process before yielding. The default value is 10. Single-threaded runtimes should return 1 to avoid starving the event loop, while multi-threaded runtimes can use higher values or rely on yield_now() returning None.
On WASM targets, Send bounds are automatically removed via #[cfg(target_arch = "wasm32")].
The BotBuilder uses a typestate pattern with four type parameters <B, T, H, R> (Backend, Transport, HttpClient, Runtime). The build() method is only callable when all four are Provided, making missing-component errors compile-time instead of runtime.
See custom backends for implementing your own runtime, transport, HTTP client, or storage backend.
Key Components
Client
Location:src/client.rs
Purpose: Orchestrates connection lifecycle, event bus, and high-level operations. The synchronization primitive follows the shape of the state, not a single default. State whose critical section awaits uses async-lock (runtime-agnostic, not Tokio-specific). State whose critical section never awaits — a clone, a store, a set op — uses a std::sync lock instead. State that is built once and never replaced uses std::sync::OnceLock. On a path that must produce a Send future (e.g. a spawned task), a std::sync::MutexGuard isn’t Send, so holding one across an .await there is a compile error rather than something a reviewer has to catch by hand (#1227).
- Connection management
- Request/response routing
- Event dispatching
- Session management
PersistenceManager
Location:src/store/persistence_manager.rs
Purpose: Manages all state changes and persistence.
- Never modify
Devicestate directly - Use
DeviceCommand+process_command() - For read-only:
get_device_snapshot()
Signal Protocol
Location:wacore/libsignal/ & src/store/signal*.rs
Purpose: End-to-end encryption via Signal Protocol implementation.
Features:
- Double Ratchet algorithm
- Pre-key bundles
- Session management
- Sender keys for groups
Socket & Handshake
Location:src/socket/, src/handshake.rs
Purpose: WebSocket connection and Noise Protocol handshake.
Flow:
- WebSocket connection
- Noise handshake (XX pattern)
- Encrypted frame exchange
Module Interactions
Layer Responsibilities
wacore layer (platform-agnostic)
- Protocol logic
- State traits
- Cryptographic helpers
- Data models
Whatsapp-rust layer (runtime)
- Runtime orchestration
- Storage integration
- User-facing API
Protocol entry points
Incoming Messages
Flow:src/message.rs → Signal decryption → Event dispatch
Incoming stanzas are decoded as Arc<OwnedNodeRef> (zero-copy from the network buffer) and routed through per-chat message queues:
Outgoing Messages
Flow:src/send.rs → Signal encryption → Socket send
Outgoing stanzas are built as owned Node values via NodeBuilder:
Socket Communication
Flow:src/socket/ → Noise framing → Transport
Connection Lifecycle
Auto-Reconnection
The client implements robust reconnection handling with stream error awareness:is_connected field uses an AtomicBool to track whether the noise socket is established. This avoids a TOCTOU race that previously occurred when try_lock() on the noise socket mutex failed under contention, causing false-negative connection checks and silent ack drops.
Connection timeout: Both the transport connection and version fetch are wrapped in a 20-second timeout (TRANSPORT_CONNECT_TIMEOUT), matching WhatsApp Web’s MQTT and DGW defaults. This prevents dead networks from blocking on the OS TCP SYN timeout (~60-75s). Both operations run in parallel via tokio::join!.
Reconnection flow:
- Connection lost →
cleanup_connection_state()(see disconnect cleanup) - Check
enable_auto_reconnect→ exit if disabled (401, 409, 516 disable this) - Check
expected_disconnect→ immediate reconnect if expected (e.g., 515) - Stability-gated backoff reset: the counter resets to base only if the connection was authenticated (
<success>) for at least 30s (STABLE_CONNECTION_RESET, WA Web’sresetDelay) and no explicit penalty (429, manualreconnect()) is pending for this cycle — a penalty survives even a stable connection (WA WebcancelReset). The stability window is measured against a monotonic clock (connected_at, awacore::time::Instant), so a system clock jump can no longer make a young connection look stable or a genuinely stable one look young (whatsapp-rust#1379). - Calculate Fibonacci backoff delay (1s, 1s, 2s, 3s, 5s, 8s… max 900s with +/-10% jitter)
- When you call
disconnect(),logout(), orsignal_shutdown_sync(), the client interrupts the backoff immediately and returns fromrun(). If the delay completes first, it attempts reconnection with a 20s connect timeout.
Pause / resume
Added in PR #1265.Client::pause()/resume() fill the gap between disconnect() (terminal — no way back for that Client) and reconnect() (comes back on the library’s own schedule). pause() drops the connection and parks the run() supervision loop until resume() releases it, on the caller’s own timeline. The client is not terminal while paused; it is between connections, on purpose — see point 6 below for the one case where a pause still ends the loop.
run()’s loop checkspausedat the top of every iteration, before each connect attempt, not only after one ends. Apause()can land as easily during a reconnect backoff as during a live connection. Findingpausedset, the loop logs and parks onwait_while_paused(). That helper re-checkspaused/is_runningagainstsession_state_notifier, so adisconnect()mid-pause still ends the loop instead of waiting for aresume()that is never coming.connect()itself refuses withConnectError::Pausedup front. The connect graph also re-checks aconnect_refusal()(shutdown-or-paused) at every checkpoint — before the transport opens, after the handshake, and at the final publish — comparing the connection attempt’s capturedpause_generationagainst the current one. That generation compare, not just thepausedflag, is what catches an attempt that spanned apause()and aresume()while it was mid-handshake: a level-triggered read ofpausedalone would seefalseand let it through.- The final refusal-check-and-publish in the connect graph and
pause()’s own capture of what it is tearing down share theconnection_publishmutex. Apause()can therefore never read “no connection” from an attempt one statement away from publishing one. pause_teardown_pendingis a one-shot fact, not a live re-read ofpaused. It’s set by whichever path ends the connection — the run loop’sConnectError::Pausedbranch, orpause()itself — so aresume()landing in the gap between the teardown starting and the run loop noticing does not leave the loop believing the ending connection deserves the ordinary Fibonacci penalty.resume()is synchronous and deliberately does not wait on apause()still in flight. That teardown ends in an untimed socket close, and queuing behind it would make “come back on my word” hostage to an unresponsive transport.resume()just clearspaused, firespause_state_notifier, and lets the run loop reconnect immediately.- The
enable_auto_reconnectcheck inrun()’s post-connection logic runs before thepause_teardown_pending/pausedcheck that would otherwise park the loop. So ifenable_auto_reconnectwas alreadyfalsewhenpause()ended a connection (or interrupted an in-flight attempt), the loop takes the auto-reconnect-disabled branch instead: it callsstop_supervision_loop()and exits for good, the same as an ordinary auto-reconnect-disabled disconnect.is_terminal()reads that combination (!enable_auto_reconnect && !is_running && !is_connected) as terminal, so the “not terminal” guarantee in the overview above holds only whileenable_auto_reconnectstays set.
pause() publishes the paused flag before it flushes and closes the socket. can_reach_server(), wait_for_socket(), and wait_for_connected()/is_fully_ready() all treat a paused client as unreachable for that whole teardown window, not just once the socket is actually down — otherwise a caller could be handed the very connection the application just asked to close. await_connection is the one place that deliberately keeps waiting through a pause rather than giving up. It already waits out a 900s backoff the same way. Nothing on the next connection re-issues a consumer’s task, so giving up early would drop the work instead of merely deferring it.
pause() does not dispatch Event::Disconnected (same reasoning as reconnect() — the application asked for this, so it isn’t news), and it is not a protocol-level presence change. Its interaction with enable_auto_reconnect (mechanics point 6 above) is the same ordering that lets handle_stream_error make a client terminal (409, 516) through that flag without a pause racing ahead of it.
See pause() / resume() in the Client API reference for the caller-facing contract.
Disconnect cleanup
When a connection is lost ordisconnect() is called, cleanup_connection_state() resets all connection-scoped state to prevent stale data from leaking into the next connection. It runs from run() after the message loop exits (and disconnect() also invokes it directly); the function is idempotent and race-tolerant, so it is not duplicated inside the message loop on transport disconnect events and resetting twice is harmless:
Bounded teardown (whatsapp-rust#1410). Every teardown path’s transport close —
disconnect(), reconnect(), reconnect_immediately(), pause(), and this cleanup routine — is now bounded by TRANSPORT_CLOSE_TIMEOUT. abort_sender() runs first, closing the sender task’s job channel and aborting it outright; any send it cuts off (queued or already in flight) fails with EncryptSendError::channel_closed() rather than hanging — a send racing a teardown was going to fail once the socket closed anyway. See WebSocket & Noise Protocol — Disconnect for the full mechanics.Chat lane invalidation is critical for correctness. Without it, stale message processing workers from the previous connection survive reconnects, holding outdated Signal session state that causes decryption failures on the new connection.
Flush-before-clear (v0.6). The signal cache is now flushed to the backend before being cleared on disconnect. Previously the cache was dropped immediately, so a just-advanced sender-key chain that hadn’t been persisted yet was lost — on the next send the client would treat the chain as fresh and re-distribute the SKDM to every group device unnecessarily. Disconnect is therefore no longer a “forget all Signal state” operation; it’s a “persist, then forget” one. This is a behavior change for anyone relying on the old drop-everything semantics.Settled-only eviction. Teardown now calls
SignalStoreCache::clear_after_flush() instead of an unconditional clear. A write that lands between the flush releasing its lock and teardown running — e.g. a concurrently raised sender-key reservation — used to be silently discarded, which could let ciphertext reach the wire before its new durability ceiling was ever persisted. Teardown now only evicts a store (sessions, identities, sender keys) once it has no dirty, deleted, checked-out, or pending-wire-gate entries; anything installed after the flush stays resident for the next successful flush to settle. See Signal Protocol — clean reload vs. crash recovery for how this interacts with the counter/iteration lease.- 401 (unauthorized): Disables auto-reconnect, emits
LoggedOut - 409 (conflict): Disables auto-reconnect, emits
StreamReplaced - 429 (rate limited): Adds 5 extra Fibonacci steps to backoff, then reconnects; also suppresses the next stability-gated backoff reset (see above)
- 515 (expected): Immediate reconnect without backoff
- 516 (device removed): Disables auto-reconnect, emits
LoggedOut <xml-not-well-formed/>(no numeric code): Force-closes the socket and reconnects with standard backoff — WA Web treats a malformed frame as unrecoverable for the current stream
Message loop (read loop)
Theread_messages_loop runs on the run() caller’s task and uses select_biased! to multiplex shutdown signals with transport events. Frame decryption is sequential (noise counter ordering), but node processing uses a hybrid inline/concurrent strategy:
- Inline:
success,failure,stream:error(connection state),messageand status-broadcaststatus(arrival order for per-chat queues),ib(offline sync tracking) - Spawned concurrently: all other stanzas (receipts, notifications, presence, etc.)
<status from="status@broadcast"> stanza carries the same E2EE payload as <message from="status@broadcast"> — the server can deliver either shape — so it’s retagged and routed through the identical inline enqueue path, and acked with class="status" plus the local device’s own JID. A <status> from a @newsletter sender is a different shape and keeps the router path instead.
Any top-level stanza the router doesn’t recognize is nacked (NackReason::UnrecognizedStanza) when it carries both id and from, rather than left unanswered — see the Nack on unrecognized stanzas note in Offline sync below.
After processing a batch of multiple frames, the loop refreshes last_data_received so the keepalive loop sees the batch completion time rather than the arrival time — preventing false-positive dead-socket triggers during large offline sync batches. The loop also cooperatively yields every yield_frequency() frames to avoid starving other tasks.
See WebSocket & Noise Protocol - Message loop for implementation details.
Keepalive loop
The keepalive loop runs as a separate spawned task, fully decoupled from the read loop. This ensures keepalive pings are never blocked by frame processing — even during large offline sync batches that take seconds to drain. The two loops communicate solely through atomic timestamps (last_data_received and first_send_since_recv — the dead-socket watchdog anchor). Both are monotonic wacore::time::Instants, not wall-clock milliseconds, so a system clock adjustment (NTP resync, waking from suspend) can’t be misread as elapsed time and trip the watchdog on a live socket (whatsapp-rust#1379). There is no “last send” timestamp: nothing reads one, and stamping every frame written would cost a clock read on the client’s hottest path.
- Sends ping every 15-30 seconds (randomized, matching WA Web’s
15 * (1 + random())) - Skips ping if data was received within the minimum interval (connection proven alive)
- Sends ping before dead-socket check to prevent false-positive reconnects on idle-but-healthy connections
- Waits up to 20s for response
- Checks dead socket on every tick (not just after failures) — catches scenarios where pending IQs caused the ping to be skipped, or where the ping succeeded but the connection died immediately after
- Detects dead socket, triggering immediate reconnection, if no data has been received for 20s since the first send after the last receive (
first_send_since_recv) — matching WA Web’sdeadSocketTimer.onOrBefore; subsequent sends do not push this deadline back out - Fatal errors (
Socket,Disconnected,NotConnected,InternalChannelClosed) cause the keepalive loop to exit immediately - Three consecutive transient (
Timeout/ServerError/ParseError) failures forcereconnect_immediately()and exit the loop, rather than pinging forever against a socket that still receives but never answers (whatsapp-rust#1410) - The loop exits as soon as
connection_generationno longer matches the value it was spawned with — the shutdown signal and generation are captured once, at spawn, rather than read from inside the task, since the task’s first poll can land after a reconnect has already completed (whatsapp-rust#1410) - Error classification is exhaustive and compile-time enforced — adding a new error variant without handling it causes a build failure
Offline sync
When reconnecting, the client tracks offline message sync progress:- Receive
<ib><offline_preview count="N"/>→ start tracking, reset counters - Process messages with
offlineattribute → increment counter - Receive
<ib><offline/>→ sync complete - Emit
OfflineSyncCompleted— or, if the connection ends before step 3,OfflineSyncInterrupted(see below)
offline_preview but stops sending stanzas before the end marker (<ib><offline/>) arrives, an inactivity watchdog forces completion — mirroring WhatsApp Web’s ShiftTimer / OFFLINE_STANZA_TIMEOUT_MS. The watchdog arms with the first batch request and re-arms on every offline stanza; because it compares a counter across a sleep rather than reading a clock per stanza, it fires somewhere in [60s, 120s) after the last one, not at a fixed 60 seconds. On expiry the client logs a warning, completes the drain with the count actually processed, and OfflineSyncCompleted fires — but only while the connection that owns the drain is still up.
Interrupted resume (PR #1380): A resume that ends because its connection drops — rather than because the end marker arrived or the stall timer fired — never claims completion. Every path that tears down connection-scoped state (connect()’s reset, cleanup_connection_state(), and the internal waiters’ own generation checks) reports the drain’s end by emitting Event::OfflineSyncInterrupted { total, delivered } exactly once, so a consumer watching only for OfflineSyncCompleted no longer mistakes silence for either success or failure. The event itself says nothing about what gets redelivered — that still follows the pre-existing commit-batch ack contract, unchanged by this PR: see Inbound Durability → Batching for exactly which batch a mid-drain disconnect does and doesn’t redeliver on the next connection’s offline_preview. See OfflineSyncInterrupted.
Concurrency gating: During offline sync, the client restricts message processing to a single concurrent task (1 semaphore permit) to preserve ordering. Once sync completes — either by the server end marker, all expected items arriving, or timeout — the semaphore is expanded to 64 permits, switching to parallel message processing.
The drain→live transition also flushes the tail of the inbound commit batch: the last accumulated batch of decrypted messages commits (buffer → Signal flush → durability hook → acks →
Event::Messages) before the semaphore widens, so no live-mode message is processed ahead of it. If that tail commit fails, the transition is deferred and retried every 3 seconds while the client stays in single-permit drain mode; OfflineSyncCompleted still fires immediately so startup waiters are not blocked on the retry.- Each semaphore swap increments an atomic
message_semaphore_generationcounter - When a task acquires a permit, it checks whether the generation has changed since it started waiting
- If the generation changed (meaning the semaphore was swapped while the task was blocked), the task drops the stale permit and re-acquires from the new semaphore
- This loop continues until the task holds a permit from the current-generation semaphore
pkmsg messages (which carry Sender Key Distribution Messages for group chats) could be silently dropped during the offline-to-online transition. Without this safety mechanism, a dropped pkmsg would cause all subsequent skmsg messages from that sender to fail with NoSenderKeyState, since the SKDM they depended on was never processed.
State reset: On reconnect or cleanup, all offline sync state is reset (counters, timing, and the semaphore is replaced with a fresh single-permit instance) so stale state does not leak into the next connection attempt — reported first as an interrupted resume, per above, if a drain was still active.
Pull-batch backlog drain (v0.6): Offline resume now drives the same pull-batch loop WA Web uses to drain the backlog: stanzas that the client can’t process (unrecognized <enc type>, known-but-empty <enc> content, duplicates, ciphertexts that decrypt-fail terminally) are transport-acked alongside their retry receipt so the server stops re-delivering them. Before this, such a stanza fell through classify_incoming_message silently, so the server kept replaying it from the offline queue every reconnect until <stream:error> closed the stream. The drain logic also acks duplicate-message PDOs that previously hit the silent-drop branch in handle_decrypted_plaintext, and preserves the original recipient attribute (via Client::spawn_node_transport_ack, which echoes the raw NodeRef instead of rebuilding from MessageInfo) so LID-routed offline stanzas don’t trigger <stream:error> on the ack.
The Meta AI bot’s
msmsg (<enc type="msmsg">) encryption type was the original motivating case for this drain — it could not be decrypted, only acked. Since the bot-secret decryption landed (see Bot message decryption), msmsg stanzas are decrypted and dispatched as normal Event::Messages; the drain still covers genuinely unrecognized or undecryptable enc types.max-retry reached in the PDO recovery state machine), the client now emits a <nack reason="…"> carrying a structured NackReason code instead of silently dropping the message. The full set of 21 codes (ParsingError, InvalidProtobuf, MissingMessageSecret, etc.) mirrors WA Web’s reason set so the server stops retransmitting once it sees a terminal nack — see Protocol → Nack reasons.
Ack SKDM-only session decrypts: A pkmsg/msg that decrypts successfully but carries only a Sender Key Distribution Message (no user-facing content to dispatch) is now explicitly acked. Previously it could decrypt, skip event dispatch, and leave no ack — so the server kept replaying it from the offline queue. The fix closes that gap so SKDM-only stanzas drain like any other processed message.
Nack unparseable message stanzas: A <message> stanza whose required id/from attrs (and participant, for group/status messages) are missing or carry an invalid JID now gets an immediate <ack error="487"> (NackReason::ParsingError) instead of a bare warning log and silent drop. parse_message_info itself became fail-fast for the same fields — it no longer falls back to a lenient default JID. Consumers driving their own ack/nack flow for intercepted stanzas can reach the same typed responses via Client::acknowledge_stanza/reject_stanza — see Manual stanza acknowledgement.
Top-level <status> stanzas are handled and acked. The server can deliver E2EE status updates as a top-level <status from="status@broadcast"> stanza instead of wrapping them in <message from="status@broadcast">. This shape differs from <message> only in its tag, so the client retags it and runs it through the same pipeline, and acks it with class="status" (the class the server names in the corresponding <stream:error><ack class="status"/></stream:error>) plus the local device’s own JID, matching WA Web’s sendAck. Before this, the tag went unhandled — the stanza was neither processed nor acked, so the server kept redelivering it and periodically recycled the stream to demand the ack it was owed.
Nack on unrecognized stanzas: Answering nothing at all when the stanza router doesn’t recognize a top-level tag is what let one unhandled <status> stanza (above) stall a stream for days — the server queued it forever and cycled the connection roughly every 50 minutes asking for its ack. Any stanza the router declines now gets <nack reason="…"> with NackReason::UnrecognizedStanza when it’s addressable (id and from present, the same guard WA Web’s createNackFromStanza uses); the nack replaces the deferred ack, so the server never gets both for one stanza. This only fires for a tag with no handler at all — see Protocol → Nack reasons.
Critical app-state sync (pairing bootstrap)
Right after a fresh pairing (and on any reconnect before the account’s critical app-state collections have synced), the client fetches theCriticalBlock and CriticalUnblockLow collections — blocked contacts and push name — via a batched IQ. Decoding those snapshots requires the app-state sync-key-share, an E2E message the primary phone sends automatically, which can arrive late if a heavy history sync is saturating the stream at the same time.
A single 180-second deadline (CRITICAL_SYNC_TIMEOUT_SECS, matching WhatsApp Web’s WAWebSyncBootstrap) bounds the whole critical-sync path:
- A watchdog task is armed first against this deadline, before anything else runs. As of PR #1291, the watchdog no longer owns retrying a bad answer (see below) — its only job is to force a reconnect if the batched sync produces no answer at all within the window, for example a request that never gets an IQ response. Whichever of the sync and the watchdog settles first claims the outcome; the other stands aside rather than fight over the same reconnect.
- The client waits up to 10 seconds (
KEY_SHARE_GRACE_SECS) for the auto-shared key before running the batched critical-collections IQ. This grace period is purely an optimization to skip a redundant explicit key request in the common fast case — it does not gate correctness. - If a collection still can’t be decoded because its key hasn’t landed, the client sends an explicit
AppStateSyncKeyRequest— fanned out to every discovered companion device the same way as the non-critical path below — and waits for the re-share. For this initial critical bootstrap, the wait is bounded by whatever time remains on the shared 180s deadline (rather than a short fixed wait), so a key that arrives late — or is never auto-shared at all — still has a chance to recover on the same connection instead of failing the sync outright.
400/404 — terminal for this connection, not permanently), retryable, or skipped (another in-flight sync or patch send already held it) — rather than a bare success/failure. As of PR #1291, every shape the batch can come back in now reaches the same conclusion: the connection gets announced. By the time this sync runs, set_passive(false) has already gone out and the socket is delivering offline stanzas, so withholding Connected would leave a consumer with no signal that anything had connected while messages kept arriving. What didn’t sync is reported instead, and retried in the background rather than by holding the announcement hostage to it:
- All synced: the client dispatches
Connectedand the watchdog is cancelled. - Any collection fatal: repeating the request on this connection would get the same answer, so the client stops waiting on it, dispatches
Connectedanyway, and follows it withEvent::AppStateSyncFailed(connected: true).- A fatal outcome does not retry on its own connection. Recovering that collection needs a fresh connection — WhatsApp Web’s
COMPANION_SYNCD_SNAPSHOT_FATAL_RECOVERYpath is explicitly gated off forcritical_block. This client now implements that escalation (see Peer snapshot recovery below) forcritical_unblock_lowand the non-critical collections, but not here: it answers a snapshot MAC that fails local verification, a different failure from the IQ-level400/404refusal this bullet covers, so a batch that comes backFatalstill has no automatic recovery on this connection regardless of which collection it names. - The account is usable in the meantime, but missing whatever that collection carries.
critical_blockincludes thesetting_pushNamemutation, so presence stays unavailable until the next connection syncs it — unless the push name was already known some other way (e.g.Bot::with_push_name, or learned earlier via history sync or a prior session). - WhatsApp Web instead notifies the primary device and logs out on a fatal
critical_block; this client does not end a session on its own.
- A fatal outcome does not retry on its own connection. Recovering that collection needs a fresh connection — WhatsApp Web’s
- Retryable and/or skipped collections outstanding (a transient error, or another writer holding the collection — including a batch that failed transport-side before producing any per-collection buckets, which is reported as every requested collection being retryable): the client dispatches
ConnectedandEvent::AppStateSyncFailed(connected: true) the same way, then hands the leftovers to the background sync that follows the bootstrap. Both buckets get one more attempt as part of that sync’s own batched request; whatever is stillretryableafter that enters the same bounded backoff every background app-state retry uses — up toAPP_STATE_RETRY_MAX_ROUNDS(8) rounds doubling from 1 second (a theoretical one-hour cap the round limit never lets it reach, so in practice up to ~128 seconds) — roughly a four-minute window — before giving up and emitting a finalAppStateSyncFailedfor whatever is still unsynced. A round spent waiting for a socket (no connection to retry over) isn’t charged as an attempt. Askippedcollection isn’t itself re-queued into that scheduler, matching the non-critical case described below — the equivalent work is understood to be happening via whichever operation already holds it. - Nothing answers at all within the 180s deadline (the batched sync itself never returns): the watchdog fires and forces a reconnect. This is the one shape left that can’t announce a connection, because there is no answer to announce; the next connection runs the critical bootstrap again from the top.
Connected nor the failure report fires for this one; the replacement reports for itself once its own sync finishes. If instead the client has been asked to pause or disconnect, or the server has since rejected the session (429/503, which clear login state inline without tearing down the socket), Connected is withheld but Event::AppStateSyncFailed still fires with connected: false, and the leftovers still go to the background sync. WhatsApp Web logs out (socketLogout) when the critical sync doesn’t close cleanly; this client keeps the session and reports the gap instead.
Non-critical app-state sync (background regular collections, group server_sync, and the ib dirty-resync path) is unaffected by this deadline — those callers keep waiting up to 10 seconds (APP_STATE_KEY_REQUEST_TIMEOUT) for a missing key before giving up and re-syncing on a later cycle. That request fans out concurrently to every other device on the account (discovered via the same usync device-list lookup used for message routing, current device excluded), falling back to the known primary alone if discovery fails or returns nothing. This dedup-override behavior is shared with the critical bootstrap path (step 3 above): any active wait — critical or non-critical — overrides an existing 24-hour passive dedup stamp for that key, shortening it to a bounded retry window so it isn’t starved by an earlier passive request’s cooldown. These background syncs report incomplete outcomes through the same Event::AppStateSyncFailed event (always with connected: true, since the client was already connected). Only their retryable collections back off and retry automatically, on the same schedule as the critical path; a fatal collection reported this way gets no automatic retry either, and needs the account’s next fresh connection.
The net effect: a key-share that’s delayed by a saturated stream during pairing no longer strands the critical sync until the 180s watchdog forces a reconnect — it recovers via the explicit request within the same window, so contacts and push name sync reliably on the first connection. And a collection that comes back retryable, refused, or held by another writer no longer strands the connection either — the client connects immediately and reports exactly what didn’t sync via Event::AppStateSyncFailed, instead of looping a silent reconnect against an answer that, for a refusal, was never going to change. Only the retryable/skipped case actually gets retried in the background; a refusal is reported once and left for the account’s next fresh connection to pick up.
Behavior change (PR #1291): before this, only the “all synced” and “fatal” outcomes above announced the connection. A retryable or skipped outcome — including a batch that failed outright before producing any buckets — used to return in silence: an authenticated, no-longer-passive connection with no
Connected and no AppStateSyncFailed, recoverable only by the 180s watchdog forcing a reconnect. If the server kept refusing or failing the same collection, that reconnect looped forever without ever announcing. A consumer that relied on the eventual reconnect producing a working session now instead gets Connected immediately alongside an actionable AppStateSyncFailed, and must treat that report as something to act on rather than something to wait out.Peer snapshot recovery
A snapshot MAC mismatch is the one app-state failure the server can’t resolve by retrying: it serves the same bytes every time, they fail local verification the same way, and the collection stays at version 0 forever — so every mutation written to it (a chat marked read, a mute, an archive) keeps refusing with a conflict that can never clear. WhatsApp Web’s answer is to ask the primary device to resend the collection outright (COMPANION_SYNCD_SNAPSHOT_FATAL_RECOVERY); this client implements the same escalation.
Trigger. Applying a synced collection — via the batched sync path or the single-collection patch-send path — raises AppStateError::SnapshotMACMismatch or SnapshotMACMissing. Both escalate; no other apply failure does (a missing app-state key is answered by a key share instead, and a bad decode by nothing).
Two exclusions. critical_block is never asked for — rebuilding the block list from a primary that may itself be behind risks talking to someone who was blocked, matching WA Web. And the whole escalation is off when the enable_peer_snapshot_recovery ab-prop reads explicit 0/false; the prop simply being absent (this client isn’t in WhatsApp’s staged rollout) still proceeds, since silence isn’t a refusal.
The ask. Client::request_syncd_snapshot_recovery sends the collection name to the primary as a fire-and-forget peer (PDO) message — the sync paths call it automatically on a matching failure, and it’s public so a caller can invoke it directly. AppStateProcessor dedupes per collection: an unanswered request suppresses a repeat for 120 seconds, and once a reply has been claimed and is being applied, that window extends to 900 seconds so the apply itself (an app-state key repair, then up to 450 seconds waiting on the collection’s own sync reservation) isn’t preempted by its own retry.
The reply. The request and the reply both travel as ordinary peer (PDO) messages, over the same Signal-encrypted session as any other device-to-device message — nothing here is sent outside that channel. What’s skipped is the app-state encryption layer on top of it: SyncdSnapshotRecovery carries the primary’s version, ltHash, and every mutation’s SyncActionData unencrypted at that layer, so only each record’s index MAC needs re-deriving here, from the app-state key. That’s what makes a collection recoverable even when its value encryption is precisely what couldn’t be followed before. The version and ltHash are written as the primary sent them rather than recomputed, since recomputing them would be this side re-deriving the very value it just failed to agree on. A reply over the snapshot_recovery_max_mutations_count_allowed ab-prop (default 2000 records) is refused rather than applied. Correlation is by request id first (the id this client minted for the ask) and by collection name second — nothing else ties an unprompted peer message back to a specific request — so a reply for the wrong collection, or one answering a request that already expired or was superseded, is dropped rather than applied.
Applied mutations dispatch through the same path an ordinary sync uses — the existing Event::ContactUpdate, mute, archive, etc. events — so there is no separate event for a recovered collection. What changed is observable via memory_report’s app_state_recovery_requests field: collections currently awaiting or applying a reply.
Deferred device sync
During offline sync, the client may receive group messages from devices not yet present in the local device registry (for example, a companion device that was paired while the client was offline). Rather than firing a network request for each unknown device individually, the client batches these into aPendingDeviceSync set.
Flow:
- During offline message processing,
is_from_known_device()detects an unrecognized sender device - The sender’s user JID is added to
PendingDeviceSync(deduplicated — each user is queued at most once) - A retry receipt is sent so the sender will redeliver the message after the device list is updated
- When
<ib><offline/>arrives (offline sync complete), the client waits 2 seconds (OFFLINE_DEVICE_SYNC_DELAY, matching WhatsApp Web) - All batched user JIDs are flushed in a single bulk usync request via
flush_pending_device_sync() - If the flush fails, the JIDs are re-enqueued for the next attempt
PendingDeviceSync state is cleared on reconnect to prevent stale entries from leaking across connections.
The <devices><update hash="…"/></notification> hash-only path described in Storage → Granular cache patching resolves its target contact and joins this same schedule_unknown_device_sync() batching — offline resumes queue it here instead of firing an immediate usync mid-drain.
Location: src/pending_device_sync.rs, src/handlers/ib.rs, src/usync.rs
See also: Unknown device detection for the detection mechanism during group message decryption.
History sync pipeline
History sync transfers chat history from the phone to the linked device. The pipeline is designed for minimal RAM usage through a multi-layered zero-copy strategy.Processing flow
RAM optimization layers
- Heuristic pre-allocation with
compressed_size_hint— the decompression buffer is pre-allocated using a 4x multiplier on the compressed blob’sfile_length(clamped to 256 bytes – 8 MiB). When the notification providesfile_length, this avoids repeatedVecreallocation during decompression. The hint comes from the decrypted (but still compressed) blob size, which is a better estimate than the encrypted size that includes MAC/padding overhead - Compressed bytes retained — after streaming extraction, the original compressed input is handed back as the event payload. Queued events cost O(compressed) rather than O(decompressed). Peak extraction memory is approximately the largest single conversation
- Hand-rolled protobuf parser — instead of decoding the entire
HistorySyncmessage tree (which allocates every nested message), the core walks varint tags manually and only extracts field 2 (conversations) and field 7 (pushnames) Byteszero-copy slicing — decompressed data is wrapped in a reference-countedBytesbuffer; each conversation is extracted asbuf.slice(pos..end), which is an Arc refcount increment with no per-conversation heap allocation- Bounded channel streaming — an
async_channel::bounded::<Bytes>(4)streams conversation bytes from the blocking parser thread to the async event dispatcher, providing backpressure with only ~4 conversations in-flight LazyHistorySyncwrapper — the compressed payload is wrapped in aLazyHistorySyncwith cheap metadata (sync type, chunk order, progress) available without decoding. Full protobuf decoding only happens if the event handler calls.get(). Clone is a refcount bump (no cache carried over). Consumers use.stream()for memory-bounded incremental decoding,.decompress()for one-shot inflation, or.compressed_bytes()for zero-copy access to the stored payload- Compile-time callback elimination — when no event handlers are registered, the callback is
None, causing the parser to skip conversation extraction entirely at the protobuf level - Secret-presence pre-scan — before running buffa decode on each
HistorySyncMsg, a shallow varint walk checks whether the message carriesmessage_secretat any level (WebMessageInfo.message_secretorMessage.message_context_info.message_secret). Messages without a secret (the majority in production blobs) are discarded immediately — no struct decode, no allocation. The scan mirrors protobuf merge semantics so repeated field occurrences and malformed bytes are handled identically to a full decode - Shared conversation id —
HistoryMsgSecretRecord.chat_idisArc<str>, allocated once per conversation and reference-counted into every record within that conversation (10k clones → 500 on the bench fixture).msg_idusesCompactString(inline for typical 20–22 char WA IDs) andsecretusesSecretBytes(inline for secrets ≤32 bytes) - Filter-before-materialize retention hook —
process_history_sync_bytes_filteredruns a caller-supplied retention predicate against a borrowedHistoryMsgSecretRecordRef(a zero-copy view into the not-yet-built record) before allocating the owned, heap-backedHistoryMsgSecretRecord. A record the predicate rejects is discarded without ever being materialized.process_history_sync_bytes— andprocess_history_syncitself — keep their existing accept-all behavior by wrapping the filtered entry point with an always-truepredicate, so callers that don’t own a retention policy are unaffected. Bench (synthetic 500-conversation blob, upstream PR’s rejection-heavy fixture): allocation churn 21.61 MiB → 14.00 MiB, allocation count ~84k → ~26k. The reduction scales with how much of the record set the predicate rejects — the default accept-all behavior sees none of it. Seewacore— history_sync types for the exported signatures - Streaming record-visitor path —
process_history_sync_bytes_with_record_visitor(closure-based) andprocess_history_sync_bytes_with_record_sink(trait-based, viaHistoryMsgSecretRecordVisitor) go further than the filter hook above. You can build your own storage row directly from the borrowedHistoryMsgSecretRecordRef. This ensures the ownedHistoryMsgSecretRecordis never allocated, even for accepted records. The visitor trait’s optionalreserveandretained_item_sizehooks let you size your own collection (e.g., a batched SQL insert buffer) up front. Bench (allocator-instrumented synthetic history extraction): 20.20 MiB → 14.43 MiB allocated (-28.6%); CodSpeed history stream-drain memory: 243.8 KB → 115.2 KB (2.1× less). Seewacore— history_sync types - Inline PN↔LID mappings and tctoken candidates, cached group senders —
HistoryLidMapping.phone_number/.lidandTcTokenCandidate.idareCompactString, andTcTokenCandidate.tc_tokenisSmallVec<[u8; 32]>: typical phone/LID user parts (11–16 digits) and typical tctoken payloads (16–24 bytes) fit inline on 64-bit targets (CompactString’s inline capacity is smaller on 32-bit/wasm32, and a longer value in either type spills to the heap), so the PN↔LID harvest — including the two indexesdedupe_lid_mappingsbuilds — and the tctoken extraction allocate nothing for typical inputs. The tctoken extraction’s chat-kind check now reuses the same borrowedparse_jid_refscan used for the PN/LID guess, instead of building a second ownedJidjust to readserver. On thewhatsapp-rustside,HistorySecretSeedCollectorcaches the previous group message’s(raw participant, Arc<str> sender)pair — group history arrives in bursts from the same sender, so a repeat of the raw field reuses the cachedArcinstead of re-parsing the JID and re-rendering it; the cache is keyed on the raw field and cleared on every chat change, so it can’t disagree with a fresh resolve. Bench (bench_process_history_sync, rebuilt mixed DM/group fixture): 7507 → 3507 allocations (-53%). Deliberate breaking change:HistoryLidMapping.phone_number/.lidandTcTokenCandidate.idmove fromString, andTcTokenCandidate.tc_tokenfromVec<u8>. Both new typesDereftostr/[u8], so a call site that consumes them as&str/&[u8](method calls,.parse(), comparisons against&str) still compiles via deref coercion; a call site typed explicitly as&String/&Vec<u8>does not coerce — seewacore— Allocation optimizations for the full migration note (#1349)
Skip mode
For bots that don’t need chat history,skip_history_sync() sends a receipt so the phone stops retrying uploads but downloads nothing. See Bot - History Sync.
Concurrency Patterns
Per-Chat Lanes
Prevents race conditions where a later message is processed before the PreKey message. Each chat gets a lane combining an enqueue lock, a worker-liveness lock, and an unbounded channel into a single cached entry. Backpressure comes from capping the number of cached lanes rather than messages within a lane: thechat_lanes cache itself has a capacity (chat_lanes_capacity, default 5,000) and evicts idle lanes once full. This is a soft cap — a lane with an in-flight message is protected from eviction (see the note below), so if every cached lane happens to be active at once, the map can briefly exceed capacity rather than evicting a live lane and letting a second worker start on the same chat:
Active lanes survive capacity eviction (v0.6). Every queued item is a
QueuedChatMessage { node, lane_liveness }, where lane_liveness is a clone of the lane’s enqueue_lock. The cache’s evict_guard refuses to evict a lane while any in-flight message still holds that clone (Arc::strong_count(&lane.enqueue_lock) > 1); the worker drops its copy only after it finishes processing that message. Previously, a lane could be capacity-evicted right after its worker dequeued a message, and a later stanza for the same chat would then miss the cache and spawn a second worker — letting two workers process the same chat concurrently and out of order. Idle lanes (no in-flight message) remain evictable exactly as before.Idle lane workers self-exit after 60 seconds (v0.7). A lane’s worker awaits the inbound-message handler inline instead of boxing it per message, so the spawned task holds that future’s whole state machine (~9 KiB) for as long as the worker runs — message or no message. Left running for the connection’s lifetime, that is one such future per chat that ever spoke, bounded only by
chat_lanes_capacity; an account active in a few thousand groups could park tens of MiB in idle workers, and the capacity-eviction guard above only ever protected active lanes from that cost, not idle ones. A worker now closes its queue and exits after LANE_IDLE_TIMEOUT (60s) of silence — the idle timer is armed only once the queue is empty, so a busy lane never pays for it. This is independent of the capacity eviction above: an idle worker exits on its own schedule regardless of how much headroom chat_lanes_capacity has left.A message that arrives for a closed lane replaces it. enqueue_lock and worker_running carry over from the predecessor rather than being re-minted — the chat’s enqueue order stays a single total order across the swap (whichever lane generation a handler happened to fetch), and the successor worker takes worker_running before processing its first message, so it can never run concurrently with the predecessor still draining whatever raced its idle close.Per-device session locks
Prevents concurrent Signal protocol operations on the same session. Each device JID gets its own lock, keyed by protocol address strings generated byto_protocol_address_string() (format: user[:device]@server.0):
WAWebSendUserMsgJob and WAWebDBDeviceListFanout behavior. The local registry is checked first; a network fetch is only triggered on a cache miss to avoid unnecessary LID-migration side effects. Session locks are acquired for all involved devices in sorted order to prevent deadlocks. The build_session_lock_keys() helper resolves encryption JIDs (normalizing the recipient to bare form via to_non_ad()), sorts by (server, user, device) using cmp_for_lock_order(), and deduplicates. The session_guards_for() helper then takes each device’s lock as its mutex is resolved from the sorted JIDs, rather than resolving the whole set before locking any of them (PR #1131); acquisition order is still jids order, which is what keeps two overlapping sends from deadlocking.
The peer message path (single-device) acquires a single lock for the resolved encryption JID.
Group SKDM fan-out now shares the DM per-device session locks (v0.6).
prepare_group_stanza’s pairwise SKDM fan-out (encrypt_for_devices_with_sessions) mutates each target device’s pairwise Signal session the same way the DM path does, but it was previously only covered by the per-(group, sender) sender-key chain lock — a disjoint key from the DM path’s per-device session locks. A concurrent DM (or another group send) sharing a device could therefore race that device’s pairwise ratchet: both sides load chain index N and both store N+1, silently dropping one advance. If the lost advance carried the SKDM, that member never received the sender key and every subsequent skmsg was undecryptable for it until a retry re-distributed.prepare_group_stanza now acquires the SKDM targets’ per-device session locks — via the new SendContextResolver::lock_device_sessions() hook, whose Client implementation reuses build_session_lock_keys() + session_guards_for() so both paths serialize on the identical mutexes — before taking the sender-key chain lock, and releases them right after the SKDM fan-out (the skmsg chain encrypt that follows only touches the sender-key chain, not any pairwise session). Lock order is always session locks → chain lock on every path, so this cannot introduce a deadlock. The hook defaults to a no-op, so custom SendContextResolver implementations (tests, benches) are unaffected unless they opt in.sort_dedup_by_user() to deduplicate participants before device resolution, and sort_dedup_by_device() to deduplicate resolved device JIDs after LID conversion — both operate in-place on sorted Vec<Jid> without HashSet allocations.
Sender-key chain lock (group receive)
The group receive path (Pass 2 of the two-pass decryption model) acquires a per-(group, sender) lock around each skmsg’s group_decrypt call, keyed by the same sender_key_name the sender-key store uses:
(group, sender) — reachable when a chat lane is capacity-evicted while its worker is still draining, and a later stanza for that chat misses the cache and spawns a second worker at the same connection generation — could both load the sender-key chain, advance it, and store their result, with the last store silently winning and dropping a chain step. This mirrors the per-device session lock the 1:1 send/receive paths already hold around their Signal ratchet mutations.
Chat-lane eviction trigger closed (v0.6). The specific double-worker path described above — a chat lane evicted while its worker was still draining — is now closed by the active-lane eviction guard: a lane with an in-flight message can no longer be capacity-evicted, so a later stanza for that chat can no longer spawn a second worker. The idle-exit lane replacement added since (v0.7) is synchronized the same way, via
worker_running, so it likewise cannot start a second worker for a chat whose predecessor is still draining. This chain lock remains in place as defense-in-depth against any other path that could produce two concurrent decrypt workers for the same (group, sender).Background Saver
Periodic persistence with dirty flag optimization:Feature Organization
Location:src/features/
src/download.rs and src/upload.rs as separate top-level modules.
Pattern: Features are accessed through accessor methods on Client:
State management flow
Best Practices
State Management
Async Operations
Error Handling
#[from] or #[source]) instead of stringifying inner errors. Callers can walk std::error::Error::source() to downcast to the original cause.
Related Sections
Authentication
Learn about QR code and pair code flows
Events
Understand the event system and handlers
Storage
Explore storage backends and state management
Getting Started
Build your first WhatsApp bot