Architecture

​

Overview

​

Workspace Structure

whatsapp-rust/
├── src/                          # Main client library
├── wacore/                       # Platform-agnostic core
│   ├── binary/                   # WhatsApp binary protocol
│   ├── libsignal/                # Signal Protocol implementation
│   ├── appstate/                 # App state management
│   ├── noise/                    # Noise Protocol handshake
│   └── derive/                   # Derive macros
├── waproto/                      # Protocol Buffers definitions
├── storages/sqlite-storage/      # SQLite backend
├── transports/tokio-transport/   # Tokio WebSocket transport
├── http_clients/ureq-client/     # HTTP client for media
├── tests/e2e/                    # End-to-end test suite
└── examples/                     # Example applications (benchmarks)

​

Three main crates

​

wacore - platform-agnostic core

• Zero runtime dependencies — no Tokio, no async-std, only futures, async-trait, async-lock, and async-channel
• 32-bit target support — uses portable-atomic for 64-bit atomics with a software fallback on platforms without native AtomicU64 (ARM32, MIPS, etc.)
• Runtime trait for pluggable async executors (Tokio, async-std, WASM, etc.)
• Transport, TransportFactory, and HttpClient traits for pluggable networking
• Backend trait for pluggable storage
• Cryptographic operations (Signal Protocol, Noise Protocol)
• Type-safe protocol node builders

wacore/
├── binary/           // Binary protocol encoding/decoding
├── libsignal/        // E2E encryption
├── noise/            // Noise Protocol handshake
├── appstate/         // App state sync protocol
├── derive/           // Derive macros (EmptyNode, ProtocolNode, StringEnum)
├── iq/               // Type-safe IQ protocol types
├── net.rs            // Transport, HttpClient trait definitions
├── runtime.rs        // Runtime trait + AbortHandle
├── protocol/         // ProtocolNode trait, keepalive, retry
├── time.rs           // Pluggable time provider + portable Instant
├── types/
│   ├── events.rs     // Event definitions
│   └── message.rs    // Message types
└── store/
    ├── traits.rs     // Storage trait definitions (Backend, SignalStore, etc.)
    └── device.rs     // Device state model

​

waproto - protocol buffers

scripts/regenerate-proto-desc.sh   # updates whatsapp.desc + .sha256
cargo build -p waproto

• Message - All message types
• WebMessageInfo - Message metadata
• HistorySync - Chat history
• SyncActionValue - App state mutations

​

Whatsapp-rust - main client

• TokioRuntime — default Runtime implementation (gated on tokio-runtime feature)
• Typestate BotBuilder — compile-time enforcement that all 4 required components are provided
• SQLite persistence (pluggable via Backend trait)
• Event bus system
• Feature modules (groups, media, newsletters, communities, etc.)

​

Runtime abstraction

pub trait Runtime: Send + Sync + 'static {
    fn spawn(&self, future: Pin<Box<dyn Future<Output = ()> + Send + 'static>>) -> AbortHandle;
    fn sleep(&self, duration: Duration) -> Pin<Box<dyn Future<Output = ()> + Send>>;
    fn spawn_blocking(&self, f: Box<dyn FnOnce() + Send + 'static>) -> Pin<Box<dyn Future<Output = ()> + Send>>;
    fn yield_now(&self) -> Option<Pin<Box<dyn Future<Output = ()> + Send>>>;

    /// How often to yield in tight loops (every N items). Defaults to 10.
    fn yield_frequency(&self) -> u32 { 10 }
}

​

Key Components

​

Client

pub struct Client {
    pub(crate) core: wacore::client::CoreClient,
    pub(crate) persistence_manager: Arc<PersistenceManager>,
    pub(crate) media_conn: Arc<RwLock<Option<MediaConn>>>,
    pub(crate) noise_socket: Arc<Mutex<Option<Arc<NoiseSocket>>>>,
    // ... connection state, caches, locks
}

• Connection management
• Request/response routing
• Event dispatching
• Session management

​

PersistenceManager

pub struct PersistenceManager {
    device: Arc<RwLock<Device>>,
    backend: Arc<dyn Backend>,
    dirty: Arc<AtomicBool>,
    save_notify: Arc<Notify>,
}

• Never modify Device state directly
• Use DeviceCommand + process_command()
• For read-only: get_device_snapshot()

​

Signal Protocol

• Double Ratchet algorithm
• Pre-key bundles
• Session management
• Sender keys for groups

​

Socket & Handshake

1. WebSocket connection
2. Noise handshake (XX pattern)
3. Encrypted frame exchange

​

Module Interactions

​

Layer Responsibilities

​

wacore layer (platform-agnostic)

• Protocol logic
• State traits
• Cryptographic helpers
• Data models

// wacore/src/iq/groups.rs
pub struct GroupQueryIq {
    group_jid: Jid,
}

impl IqSpec for GroupQueryIq {
    type Response = GroupInfoResponse;
    fn build_iq(&self) -> InfoQuery<'static> { /* ... */ }
    fn parse_response(&self, response: &NodeRef<'_>) -> Result<Self::Response> { /* ... */ }
}

​

Whatsapp-rust layer (runtime)

• Runtime orchestration
• Storage integration
• User-facing API

// src/features/groups.rs
impl<'a> Groups<'a> {
    pub async fn get_metadata(&self, jid: &Jid) -> Result<GroupMetadata> {
        // Use wacore IqSpec for protocol
        self.client.execute(GroupQueryIq::new(jid)?).await
    }
}

​

Protocol entry points

​

Incoming Messages

// src/message.rs
pub async fn handle_message(client: &Arc<Client>, node: &Arc<OwnedNodeRef>) {
    // 1. Extract encrypted message from NodeRef
    // 2. Decrypt via Signal Protocol
    // 3. Commit (or accumulate into a batch during the offline drain)
    // 4. Dispatch Event::Messages
}

​

Outgoing Messages

// src/send.rs
pub async fn send_message(client: &Arc<Client>, msg: &Message) {
    // 1. Encrypt via Signal Protocol
    // 2. Build protocol node (owned Node)
    // 3. Send via NoiseSocket
}

​

Socket Communication

// src/socket/mod.rs
impl NoiseSocket {
    pub async fn send_node(&self, node: Node) -> Result<()> {
        // 1. Marshal to binary
        // 2. Encrypt with Noise
        // 3. Frame and send
    }
}

​

Connection Lifecycle

​

Auto-Reconnection

// Client fields for connection and reconnection management
is_connected: Arc<AtomicBool>,                    // Lock-free connection state (Acquire/Release)
pub enable_auto_reconnect: Arc<AtomicBool>,       // Toggle auto-reconnect
pub auto_reconnect_errors: Arc<AtomicU32>,        // Error count for backoff
pub(crate) expected_disconnect: Arc<AtomicBool>,  // Expected vs unexpected
pub(crate) connection_generation: Arc<AtomicU64>, // Detect stale tasks

1. Connection lost → cleanup_connection_state() (see disconnect cleanup)
2. Check enable_auto_reconnect → exit if disabled (401, 409, 516 disable this)
3. Check expected_disconnect → immediate reconnect if expected (e.g., 515)
4. Stability-gated backoff reset: the counter resets to base only if the connection was authenticated (<success>) for at least 30s (STABLE_CONNECTION_RESET_MS, WA Web’s resetDelay) and no explicit penalty (429, manual reconnect()) is pending for this cycle — a penalty survives even a stable connection (WA Web cancelReset)
5. Calculate Fibonacci backoff delay (1s, 1s, 2s, 3s, 5s, 8s… max 900s with +/-10% jitter)
6. Wait → attempt reconnection (with 20s connect timeout)

​

Disconnect cleanup

• 401 (unauthorized): Disables auto-reconnect, emits LoggedOut
• 409 (conflict): Disables auto-reconnect, emits StreamReplaced
• 429 (rate limited): Adds 5 extra Fibonacci steps to backoff, then reconnects; also suppresses the next stability-gated backoff reset (see above)
• 515 (expected): Immediate reconnect without backoff
• 516 (device removed): Disables auto-reconnect, emits LoggedOut
• <xml-not-well-formed/> (no numeric code): Force-closes the socket and reconnects with standard backoff — WA Web treats a malformed frame as unrecoverable for the current stream

​

Message loop (read loop)

• Inline: success, failure, stream:error (connection state), message and status-broadcast status (arrival order for per-chat queues), ib (offline sync tracking)
• Spawned concurrently: all other stanzas (receipts, notifications, presence, etc.)

​

Keepalive loop

const KEEP_ALIVE_INTERVAL_MIN: Duration = Duration::from_secs(15);
const KEEP_ALIVE_INTERVAL_MAX: Duration = Duration::from_secs(30);
const KEEP_ALIVE_RESPONSE_DEADLINE: Duration = Duration::from_secs(20);
const DEAD_SOCKET_TIME: Duration = Duration::from_secs(20);

• Sends ping every 15-30 seconds (randomized, matching WA Web’s 15 * (1 + random()))
• Skips ping if data was received within the minimum interval (connection proven alive)
• Sends ping before dead-socket check to prevent false-positive reconnects on idle-but-healthy connections
• Waits up to 20s for response
• Checks dead socket on every tick (not just after failures) — catches scenarios where pending IQs caused the ping to be skipped, or where the ping succeeded but the connection died immediately after
• Detects dead socket, triggering immediate reconnection, if no data has been received for 20s since the first send after the last receive (first_send_since_recv_ms) — matching WA Web’s deadSocketTimer.onOrBefore; subsequent sends do not push this deadline back out
• Fatal errors (Socket, Disconnected, NotConnected, InternalChannelClosed) cause the keepalive loop to exit immediately
• Error classification is exhaustive and compile-time enforced — adding a new error variant without handling it causes a build failure

​

Offline sync

pub(crate) struct OfflineSyncMetrics {
    pub active: AtomicBool,
    pub total_messages: AtomicUsize,
    pub processed_messages: AtomicUsize,
    pub start_time: Mutex<Option<wacore::time::Instant>>,
}

1. Receive <ib><offline_preview count="N"/> → start tracking, reset counters
2. Process messages with offline attribute → increment counter
3. Receive <ib><offline/> → sync complete
4. Emit OfflineSyncCompleted event

1. Each semaphore swap increments an atomic message_semaphore_generation counter
2. When a task acquires a permit, it checks whether the generation has changed since it started waiting
3. If the generation changed (meaning the semaphore was swapped while the task was blocked), the task drops the stale permit and re-acquires from the new semaphore
4. This loop continues until the task holds a permit from the current-generation semaphore

​

Critical app-state sync (pairing bootstrap)

1. A watchdog task is armed first against this deadline, before anything else runs. If critical sync hasn’t completed by then, the client reconnects to retry — WhatsApp Web instead logs out (socketLogout) at this point, but reconnecting preserves credentials and lets auto-reconnect retry the sync.
2. The client waits up to 10 seconds (KEY_SHARE_GRACE_SECS) for the auto-shared key before running the batched critical-collections IQ. This grace period is purely an optimization to skip a redundant explicit key request in the common fast case — it does not gate correctness.
3. If a collection still can’t be decoded because its key hasn’t landed, the client sends an explicit AppStateSyncKeyRequest — fanned out to every discovered companion device the same way as the non-critical path below — and waits for the re-share. For this initial critical bootstrap, the wait is bounded by whatever time remains on the shared 180s deadline (rather than a short fixed wait), so a key that arrives late — or is never auto-shared at all — still has a chance to recover on the same connection instead of failing the sync outright.

​

Deferred device sync

1. During offline message processing, is_from_known_device() detects an unrecognized sender device
2. The sender’s user JID is added to PendingDeviceSync (deduplicated — each user is queued at most once)
3. A retry receipt is sent so the sender will redeliver the message after the device list is updated
4. When <ib><offline/> arrives (offline sync complete), the client waits 2 seconds (OFFLINE_DEVICE_SYNC_DELAY, matching WhatsApp Web)
5. All batched user JIDs are flushed in a single bulk usync request via flush_pending_device_sync()
6. If the flush fails, the JIDs are re-enqueued for the next attempt

// src/pending_device_sync.rs
pub(crate) struct PendingDeviceSync {
    pending: async_lock::Mutex<HashSet<Jid>>,
}

​

History sync pipeline

​

Processing flow

​

RAM optimization layers

1. Heuristic pre-allocation with compressed_size_hint — the decompression buffer is pre-allocated using a 4x multiplier on the compressed blob’s file_length (clamped to 256 bytes – 8 MiB). When the notification provides file_length, this avoids repeated Vec reallocation during decompression. The hint comes from the decrypted (but still compressed) blob size, which is a better estimate than the encrypted size that includes MAC/padding overhead
2. Compressed bytes retained — after streaming extraction, the original compressed input is handed back as the event payload. Queued events cost O(compressed) rather than O(decompressed). Peak extraction memory is approximately the largest single conversation
3. Hand-rolled protobuf parser — instead of decoding the entire HistorySync message tree (which allocates every nested message), the core walks varint tags manually and only extracts field 2 (conversations) and field 7 (pushnames)
4. Bytes zero-copy slicing — decompressed data is wrapped in a reference-counted Bytes buffer; each conversation is extracted as buf.slice(pos..end), which is an Arc refcount increment with no per-conversation heap allocation
5. Bounded channel streaming — an async_channel::bounded::<Bytes>(4) streams conversation bytes from the blocking parser thread to the async event dispatcher, providing backpressure with only ~4 conversations in-flight
6. LazyHistorySync wrapper — the compressed payload is wrapped in a LazyHistorySync with cheap metadata (sync type, chunk order, progress) available without decoding. Full protobuf decoding only happens if the event handler calls .get(). Clone is a refcount bump (no cache carried over). Consumers use .stream() for memory-bounded incremental decoding, .decompress() for one-shot inflation, or .compressed_bytes() for zero-copy access to the stored payload
7. Compile-time callback elimination — when no event handlers are registered, the callback is None, causing the parser to skip conversation extraction entirely at the protobuf level
8. Secret-presence pre-scan — before running buffa decode on each HistorySyncMsg, a shallow varint walk checks whether the message carries message_secret at any level (WebMessageInfo.message_secret or Message.message_context_info.message_secret). Messages without a secret (the majority in production blobs) are discarded immediately — no struct decode, no allocation. The scan mirrors protobuf merge semantics so repeated field occurrences and malformed bytes are handled identically to a full decode
9. Shared conversation id — HistoryMsgSecretRecord.chat_id is Arc<str>, allocated once per conversation and reference-counted into every record within that conversation (10k clones → 500 on the bench fixture). msg_id uses CompactString (inline for typical 20–22 char WA IDs) and secret uses SecretBytes (inline for secrets ≤32 bytes)
10. Filter-before-materialize retention hook — process_history_sync_bytes_filtered runs a caller-supplied retention predicate against a borrowed HistoryMsgSecretRecordRef (a zero-copy view into the not-yet-built record) before allocating the owned, heap-backed HistoryMsgSecretRecord. A record the predicate rejects is discarded without ever being materialized. process_history_sync_bytes — and process_history_sync itself — keep their existing accept-all behavior by wrapping the filtered entry point with an always-true predicate, so callers that don’t own a retention policy are unaffected. Bench (synthetic 500-conversation blob, upstream PR’s rejection-heavy fixture): allocation churn 21.61 MiB → 14.00 MiB, allocation count ~84k → ~26k. The reduction scales with how much of the record set the predicate rejects — the default accept-all behavior sees none of it. See wacore — history_sync types for the exported signatures
11. Streaming record-visitor path — process_history_sync_bytes_with_record_visitor (closure-based) and process_history_sync_bytes_with_record_sink (trait-based, via HistoryMsgSecretRecordVisitor) go further than the filter hook above. You can build your own storage row directly from the borrowed HistoryMsgSecretRecordRef. This ensures the owned HistoryMsgSecretRecord is never allocated, even for accepted records. The visitor trait’s optional reserve and retained_item_size hooks let you size your own collection (e.g., a batched SQL insert buffer) up front. Bench (allocator-instrumented synthetic history extraction): 20.20 MiB → 14.43 MiB allocated (-28.6%); CodSpeed history stream-drain memory: 243.8 KB → 115.2 KB (2.1× less). See wacore — history_sync types

​

Skip mode

​

Concurrency Patterns

​

Per-Chat Lanes

pub(crate) chat_lanes: Cache<Jid, ChatLane>,
// ChatLane { enqueue_lock, queue_tx }
// Each queue: async_channel::unbounded::<QueuedChatMessage>()

​

Per-device session locks

pub(crate) session_locks: Cache<String, Arc<async_lock::Mutex<()>>>,
// Key examples: "5511999887766@c.us.0", "123456789:33@lid.0"

​

Sender-key chain lock (group receive)

let chain_lock = adapter.sender_key_store.sender_key_lock(&sender_key_name).await;
// ...
let _chain_guard = chain_lock.lock().await;
let decrypt_result = group_decrypt(ciphertext, &mut adapter.sender_key_store, &sender_key_name).await;

​

Background Saver

impl PersistenceManager {
    pub fn run_background_saver(self: Arc<Self>, runtime: Arc<dyn Runtime>, interval: Duration) {
        runtime.spawn(Box::pin(async move {
            loop {
                // Wait for notification or interval
                self.save_to_disk().await;
            }
        }));
    }
}

​

Feature Organization

features/
├── mod.rs              // Feature exports
├── blocking.rs         // Block/unblock contacts
├── chat_actions.rs     // Archive, pin, mute, star
├── chatstate.rs        // Typing indicators
├── community.rs        // Community management
├── contacts.rs         // Contact operations
├── groups.rs           // Group management
├── media_reupload.rs   // Media re-upload for retry
├── mex.rs              // GraphQL MEX queries
├── newsletter.rs       // Newsletter/channel operations
├── polls.rs            // Poll creation and voting
├── presence.rs         // Presence updates
├── profile.rs          // Profile management
├── signal.rs           // Signal protocol feature operations
├── status.rs           // Status updates
└── tctoken.rs          // Trusted contact tokens

// Access features through the client
let metadata = client.groups().get_metadata(&group_jid).await?;
let result = client.groups().create_group(options).await?;
client.presence().set_available().await?;

​

State management flow

​

Best Practices

​

State Management

// Use DeviceCommand for state changes
client.persistence_manager
    .process_command(DeviceCommand::SetPushName(name))
    .await;

// Never modify Device directly
let mut device = client.device.write().await;
device.push_name = name; // DON'T DO THIS

​

Async Operations

// Wrap blocking I/O in spawn_blocking
let result = tokio::task::spawn_blocking(move || {
    // Heavy crypto or blocking HTTP
    expensive_operation()
}).await?;

// Never block the async runtime
let result = expensive_operation(); // Stalls all tasks

​

Error Handling

use thiserror::Error;
use anyhow::Result;

#[derive(Debug, Error)]
pub enum SocketError {
    #[error("socket is closed")]
    SocketClosed,
    #[error("noise cipher operation failed")]
    Cipher(#[from] NoiseError),
}

// Use anyhow::Result for functions with multiple error types
pub async fn complex_operation() -> Result<()> {
    // Automatically converts errors with ?
    socket_operation()?;
    storage_operation()?;
    Ok(())
}

​

Related Sections