> ## Documentation Index > Fetch the complete documentation index at: https://whatsapp-rust.jlucaso.com/llms.txt > Use this file to discover all available pages before exploring further. # wacore > Platform-agnostic WhatsApp protocol implementation ## Overview `wacore` is the core WhatsApp protocol implementation for whatsapp-rust. It's designed to be **platform-agnostic** with no runtime dependencies on Tokio or specific databases, making it portable across different async runtimes and storage backends. ```toml theme={null} [dependencies] wacore = "0.6" ``` ## Philosophy wacore contains all the core logic for: * Binary protocol encoding/decoding * Cryptographic primitives (AES-GCM, Signal Protocol) * IQ protocol types and specifications * Runtime abstraction (`Runtime` trait for pluggable async executors) * Network abstractions (`Transport`, `TransportFactory`, `HttpClient` traits) * State management traits (`Backend`, `SignalStore`, `AppSyncStore`, etc.) * Message builders and parsers It has **zero runtime dependencies** — no Tokio, no async-std, only `futures`, `async-trait`, `async-lock`, and `async-channel` for async primitives. This makes `wacore` portable to any async runtime, including WASM targets. The main `whatsapp-rust` crate provides concrete implementations (Tokio runtime, SQLite storage, ureq HTTP client, Tokio WebSocket transport). ## Key Exports ### Re-exported Crates ```rust theme={null} pub use aes_gcm; // AES-GCM encryption pub use wacore_appstate as appstate; // App state sync pub use wacore_noise as noise; // Noise Protocol pub use wacore_libsignal as libsignal; // Signal Protocol ``` ### Derive Macros ```rust theme={null} pub use wacore_derive::{EmptyNode, ProtocolNode, StringEnum}; ``` * **`EmptyNode`** - For protocol nodes with only a tag (no attributes) * **`ProtocolNode`** - For protocol nodes with string attributes * **`StringEnum`** - For enums with string representations See [Binary Protocol](/advanced/binary-protocol) for usage examples. ### Framing ```rust theme={null} pub use wacore_noise::framing; // WebSocket frame encoding/decoding ``` ## Core Modules ### Protocol & Binary Type-safe protocol node builders and parsers XML utilities for protocol nodes ### Cryptography Signal Protocol implementation for E2E encryption Noise Protocol XX for handshake encryption ### IQ Protocol ```rust theme={null} pub mod iq; ``` Type-safe IQ request/response specifications: * **`blocklist`** - Block/unblock contacts * **`chatstate`** - Typing indicators, presence * **`contacts`** - Contact synchronization * **`dirty`** - Dirty bit checking * **`groups`** - Group management operations * **`keepalive`** - Connection keepalive * **`mediaconn`** - Media server connections * **`mex`** - Message Extension queries * **`passive`** - Passive IQ handling * **`prekeys`** - Prekey distribution * **`privacy`** - Privacy settings * **`props`** - Server properties and A/B experiment configs. The companion [`abprops`](#a-b-props-registry) module ships the typed flag registry, and `props::WATCHED` lists the flags the library itself reads * **`spam_report`** - Spam reporting * **`tctoken`** - Temporary client tokens * **`usync`** - User synchronization. Ships a typed query/response model (`UsyncQuery`, `UsyncProtocol`, `UsyncResponse`, ...) covering every USync subprotocol observed in WhatsApp Web (device lists, contact/LID/username lookup, status, bot profiles, features). See [USync](/api/usync) See [Architecture](/concepts/architecture) for the IQ protocol pattern. #### A/B props registry `wacore::iq::abprops` is an auto-generated, vendored snapshot of WhatsApp Web's A/B-props registry. Each WA Web registry becomes a `pub mod` (currently `web`), and each flag becomes a typed `pub const AbProp` named after its key in screaming snake case (for example `web::PRIVACY_TOKEN_SENDING_ON_GROUP_CREATE`). Every entry carries: * `name` — the wire name the server uses in `` * `code` — the numeric `config_code` * `value_type` — `AbPropType::Bool`, `Int`, `Float`, or `Str` * `default` — the registry default applied when the server omits the flag Pass these constants to [`AbPropsCache`](/api/client#ab-props-cache) (`is_enabled`, `get`, `get_int`, `watch`, `watch_many`) instead of raw `u32` codes. The library only materializes the consts you reference, so the rest of the \~2,000-flag registry adds no binary weight. ```rust theme={null} use wacore::iq::abprops::web; let flag = web::ADMIN_REVOKE_RECEIVER; println!("{} (code {}) defaults to {:?}", flag.name, flag.code, flag.default); ``` `wacore::iq::props::WATCHED` is the slice of flags the library itself reads — useful as a starting point if you want to extend the cache's interest set. A small `props::stale` module preserves flags the client still references but that the current WA Web bundle no longer ships. ### Message Handling Message encryption and decryption Message sending logic Media download and decryption Media encryption and upload preparation ### State Management ```rust theme={null} pub mod store; ``` * **`Device`** - Core device state structure * **`DeviceCommand`** - State mutation commands * **`traits`** - Backend trait definitions (`Backend`, `SessionStore`, etc.) * **`ab_props`** - In-memory A/B experiment property cache (`AbPropsCache`), populated from `fetch_props()` on each connection. Features query this cache using typed flag constants from the [`abprops` registry](#a-b-props-registry) (e.g., privacy token attachment on group operations). See [State Management](/advanced/state-management) for the command pattern. ### Runtime & Networking ```rust theme={null} pub mod runtime; // Runtime trait, AbortHandle, timeout(), blocking() pub mod net; // Transport, TransportFactory, HttpClient, TransportEvent ``` The `runtime` module defines the `Runtime` trait that all async operations go through: ```rust theme={null} pub trait Runtime: Send + Sync + 'static { fn spawn(&self, future: Pin + Send + 'static>>) -> AbortHandle; fn sleep(&self, duration: Duration) -> Pin + Send>>; fn spawn_blocking(&self, f: Box) -> Pin + Send>>; fn yield_now(&self) -> Option + Send>>>; /// How often to yield in tight loops (every N items). Defaults to 10. fn yield_frequency(&self) -> u32 { 10 } } ``` | Method | Purpose | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `spawn` | Spawn a background task, returning an `AbortHandle` for cancellation. The handle is `#[must_use]` — dropping it aborts the task. Call `.detach()` for fire-and-forget tasks | | `sleep` | Return a future that completes after a duration | | `spawn_blocking` | Offload a blocking closure to a thread pool | | `yield_now` | Cooperatively yield; return `None` if unnecessary (e.g., multi-threaded runtimes) | | `yield_frequency` | How many items to process before yielding in tight loops (default: `10`) | #### Helper functions The `runtime` module also provides two runtime-agnostic helper functions that work with any `Runtime` implementation: **`timeout`** — Race a future against a deadline using the runtime's `sleep` implementation: ```rust theme={null} pub async fn timeout( rt: &dyn Runtime, duration: Duration, future: F, ) -> Result where F: Future, ``` Returns `Ok(value)` if the future completes before the duration, or `Err(Elapsed)` if it times out. This is the runtime-agnostic replacement for `tokio::time::timeout` — the library uses it internally for phash validation, media retry timeouts, session establishment, and IQ response waiting. **`blocking`** — Offload a blocking closure and return its result: ```rust theme={null} pub async fn blocking( rt: &dyn Runtime, f: impl FnOnce() -> T + Send + 'static, ) -> T ``` Wraps `Runtime::spawn_blocking` with a oneshot channel to ferry the closure's return value back to the caller. On WASM, the closure runs inline since there is only one thread. The `net` module defines the networking abstractions: * **`Transport`** — active connection for sending/receiving raw bytes * **`TransportFactory`** — creates new transport instances and event streams * **`HttpClient`** — HTTP request execution (buffered and streaming) * **`TransportEvent`** — `Connected`, `DataReceived(Bytes)`, `Disconnected` On WASM targets (`target_arch = "wasm32"`), all `Send` bounds are automatically removed. ### Connection & Pairing Noise Protocol handshake QR code pairing Phone number pairing SHORTCAKE\_PASSKEY companion-linking crypto (ephemeral identity commit/reveal, verification code, HKDF/AES-GCM pairing envelope, handoff proof) Transport and HTTP client traits `shortcake` is pure and platform-agnostic (no Tokio, wasm-buildable) — it only builds/parses the deterministic protocol payloads. The one non-reproducible step, obtaining a WebAuthn assertion, lives in `whatsapp_rust::passkey` (the `PasskeyAuthenticator` seam). See [Authentication — Passkey linking](/concepts/authentication#passkey-linking-shortcake_passkey). ### Specialized Features App state synchronization (contacts, settings) Message history synchronization User device list synchronization Signal Protocol prekey generation #### history\_sync types `HistoryMsgSecretRecord` carries the per-message secret extracted by the history-sync pipeline for E2E key expansion. ```rust theme={null} pub struct HistoryMsgSecretRecord { pub chat_id: Arc, // shared per conversation pub from_me: bool, pub key_participant: Option, pub web_msg_participant: Option, pub msg_id: CompactString, // inline for typical 20–22 char WA IDs pub secret: SecretBytes, // inline ≤32 bytes, heap for larger pub timestamp: Option, pub is_poll_or_event: bool, pub is_bot_invocation: bool, } ``` `SecretBytes` avoids heap allocation for secrets ≤32 bytes (covering the typical 32-byte Signal message secret) and falls back to `Vec` for larger values. ```rust theme={null} pub enum SecretBytes { Inline { len: u8, buf: [u8; 32] }, Heap(Vec), } impl SecretBytes { pub const INLINE_CAP: usize = 32; pub fn as_slice(&self) -> &[u8]; pub fn into_vec(self) -> Vec; } ``` Implements `Deref`, `From<&[u8]>`, `From>`, `PartialEq`, `Eq`, and `Debug`. Use `as_slice()` or deref for reads; `into_vec()` to convert to `Vec`. For `Bytes`-owned input, `wacore::history_sync` also exports: ```rust theme={null} pub fn process_history_sync_bytes( compressed_data: Bytes, own_user: Option<&str>, retain_blob: bool, ) -> Result pub fn process_history_sync_bytes_filtered( compressed_data: Bytes, own_user: Option<&str>, retain_blob: bool, record_filter: F, ) -> Result where F: for<'a> FnMut(HistoryMsgSecretRecordRef<'a>) -> bool ``` `own_user` scopes `from_me` classification to the local account's JID (pass `None` if that check should be skipped). `retain_blob` controls whether the returned `HistorySyncResult` keeps a `Bytes` handle on the original compressed input (`compressed_bytes`, consumed by `LazyHistorySync`) — pass `false` to drop it immediately after parsing. `process_history_sync_bytes` is a `Bytes`-based sibling of `process_history_sync` (unchanged) — both accept every record. `process_history_sync_bytes_filtered` additionally takes a `record_filter` predicate that runs against a borrowed `HistoryMsgSecretRecordRef<'a>` before its owned `HistoryMsgSecretRecord` counterpart is allocated, so a caller that owns a retention policy (for example, dropping records outside a retention window) can reject them without paying for materialization: ```rust theme={null} pub struct HistoryMsgSecretRecordRef<'a> { pub conversation_index: usize, pub chat_id: &'a str, pub from_me: bool, pub key_participant: Option<&'a str>, pub web_msg_participant: Option<&'a str>, pub msg_id: &'a str, pub secret: &'a [u8], pub timestamp: Option, pub is_poll_or_event: bool, pub is_bot_invocation: bool, } ``` `conversation_index` is a zero-based counter shared across records from the same conversation, letting a filter cache per-conversation classification instead of recomputing it per record. See [Architecture — RAM optimization layers](/concepts/architecture#ram-optimization-layers) for the measured allocation win. If you want to skip the intermediate owned `HistoryMsgSecretRecord` entirely, `wacore::history_sync` also exports a streaming visitor path: ```rust theme={null} pub trait HistoryMsgSecretRecordVisitor { fn visit(&mut self, record: HistoryMsgSecretRecordRef<'_>) -> usize; fn reserve(&mut self, _additional: usize) {} fn retained_item_size(&self) -> Option { None } } pub fn process_history_sync_bytes_with_record_visitor( compressed_data: Bytes, own_user: Option<&str>, retain_blob: bool, visitor: F, ) -> Result where F: for<'a> FnMut(HistoryMsgSecretRecordRef<'a>) pub fn process_history_sync_bytes_with_record_sink( compressed_data: Bytes, own_user: Option<&str>, retain_blob: bool, visitor: V, ) -> Result where V: HistoryMsgSecretRecordVisitor ``` `process_history_sync_bytes_with_record_visitor` takes a plain `FnMut` closure for the common case. The closure builds your own row directly from the borrowed `HistoryMsgSecretRecordRef`. This ensures the owned `HistoryMsgSecretRecord` is never allocated. `process_history_sync_bytes_with_record_sink` takes a full `HistoryMsgSecretRecordVisitor` implementation instead of a closure. The `visit` return value reports the byte size you retained for that record for accounting. The optional `reserve` and `retained_item_size` hooks let you size your own collection (e.g., a batched SQL insert buffer) up front rather than growing it one record at a time. Reach for `process_history_sync_bytes_filtered` (above) when a simple accept/reject predicate is enough. Reach for the visitor/sink pair when you also want to avoid materializing the owned record. ### Time ```rust theme={null} pub mod time; ``` The `time` module centralizes all timestamp handling. It exposes two independent clocks that should not be confused: * **Wall clock** (`TimeProvider`, `now_millis`, `now_utc`) — answers "what time is it?". May jump backwards across NTP syncs, manual adjustments, or leap-second smearing. Backed by `chrono::Utc::now()` on native targets. * **Monotonic clock** (`MonotonicProvider`, `Instant`) — answers "how much time passed?". Never moves backwards and is immune to NTP adjustments. Backed by `std::time::Instant` on native targets. On `wasm32-unknown-unknown` there is no built-in wall-clock source. If you do not register a provider before the first timestamp, `now_millis()` returns `0` (Unix epoch) and logs a single warning. Always call `set_time_provider` with a `Date.now()`-backed (or equivalent) provider during WASM startup — see [Custom wall-clock provider](#custom-wall-clock-provider) below. Use the wall clock for stanza timestamps, app-state mutations, and log lines. Use the monotonic clock for timeouts, retry backoff, and latency measurements. Conflating the two silently corrupts elapsed-time logic whenever the system clock is adjusted mid-measurement. Each clock can be overridden globally — useful in environments where the standard implementations are unavailable (e.g., WASM) or for deterministic testing. #### Wall-clock functions | Function | Return type | Description | | ------------------------ | ----------------------- | ----------------------------------------------------------------------- | | `now_millis()` | `i64` | Current time in milliseconds since Unix epoch | | `now_secs()` | `i64` | Current time in seconds since Unix epoch | | `now_utc()` | `DateTime` | Current time as a `chrono::DateTime` | | `from_secs(ts)` | `Option>` | Convert Unix timestamp (seconds) to `DateTime` | | `from_secs_or_now(ts)` | `DateTime` | Like `from_secs`, falling back to `now_utc()` for out-of-range values | | `from_millis(ts)` | `Option>` | Convert Unix timestamp (milliseconds) to `DateTime` | | `from_millis_or_now(ts)` | `DateTime` | Like `from_millis`, falling back to `now_utc()` for out-of-range values | #### Custom wall-clock provider Implement the `TimeProvider` trait and call `set_time_provider` before any time functions are used: ```rust theme={null} use wacore::time::{TimeProvider, set_time_provider}; struct FixedTime; impl TimeProvider for FixedTime { fn now_millis(&self) -> i64 { 1700000000000 // Fixed timestamp for testing } } // Must be called before any time functions are used set_time_provider(FixedTime).expect("provider already set"); ``` `set_time_provider` returns `Err` if a provider has already been set. The provider uses `OnceLock` internally, so it can only be configured once per process. On `wasm32` targets the epoch fallback is not cached, so a later `set_time_provider` call still takes effect even if a timestamp was read during startup. A typical browser embedder wires `Date.now()` through `wasm-bindgen`: ```rust theme={null} use wacore::time::{TimeProvider, set_time_provider}; struct JsDateProvider; impl TimeProvider for JsDateProvider { fn now_millis(&self) -> i64 { js_sys::Date::now() as i64 } } // Register before constructing the client or sending stanzas. set_time_provider(JsDateProvider).expect("provider already set"); ``` #### Instant `Instant` is a portable monotonic instant that replaces `std::time::Instant`, which is unavailable on `wasm32-unknown-unknown`. On native targets it wraps `std::time::Instant` via the default `MonotonicProvider` and exposes nanosecond resolution. The `Instant` type is `Copy` and supports `Add` and `Sub` (returning `Duration`), with saturating arithmetic to prevent overflow. ```rust theme={null} use wacore::time::Instant; let start = Instant::now(); // ... do work ... let elapsed = start.elapsed(); // std::time::Duration ``` #### Custom monotonic provider Implement the `MonotonicProvider` trait and call `set_monotonic_provider` before any `Instant` is captured. The provider must return nanoseconds since an arbitrary fixed reference and never return a smaller value than a previous call. ```rust theme={null} use wacore::time::{MonotonicProvider, set_monotonic_provider}; struct PerfNowProvider; impl MonotonicProvider for PerfNowProvider { fn now_nanos(&self) -> u64 { // e.g., wrap performance.now() in browsers, or hrtime in Node (js_sys::performance().now() * 1_000_000.0) as u64 } } set_monotonic_provider(PerfNowProvider).expect("provider already set"); ``` On `wasm32` targets without a registered provider, the fallback derives nanos from the wall clock (clamped to non-decreasing) and quantizes to milliseconds. Embedders targeting browsers, Node, or WASI should register a sub-millisecond provider for accurate latency measurements. `set_monotonic_provider` returns `Err` if a provider has already been set. Like the wall-clock provider, it uses `OnceLock` internally and can only be configured once per process. ### Utilities * **`client`** - Client context traits * **`ib`** - Identity byte utilities * **`proto_helpers`** - Protobuf conversion helpers * **`reporting_token`** - Reporting token generation * **`request`** - Request building utilities * **`stanza`** - Common stanza builders * **`sticker_pack`** - Sticker pack creation helpers (see [sticker packs](/guides/sending-messages#sticker-packs)) * **`time`** - Pluggable wall-clock and monotonic-clock providers, plus portable `Instant` (see [time](#time) above) * **`types`** - Common type definitions (JID, events, messages). Includes `types::jid` utilities for zero-allocation JID comparison (`cmp_for_lock_order`), buffer-reusing address formatting (`write_protocol_address_to`), and in-place sorted deduplication (`sort_dedup_by_user`, `sort_dedup_by_device`) * **`version`** - WhatsApp version constants * **`webp`** - WebP format utilities (animated sticker detection) ### webp module The `webp` module provides utilities for working with WebP image files. It is re-exported as `whatsapp_rust::webp`. #### is\_animated Detects whether a WebP file contains animation frames by parsing RIFF/VP8X headers and scanning for ANIM/ANMF chunks. ```rust theme={null} pub fn is_animated(data: &[u8]) -> bool ``` Raw WebP file bytes. Returns `true` if the WebP file is animated, `false` otherwise (including for invalid or too-short input). **Example:** ```rust theme={null} use whatsapp_rust::webp; let webp_bytes = std::fs::read("sticker.webp")?; if webp::is_animated(&webp_bytes) { println!("Animated sticker"); } else { println!("Static sticker"); } ``` This function is used internally by `create_sticker_pack_zip` to set the `is_animated` field on each sticker proto entry. You can also use it directly when you need to classify WebP files before processing. ## Submodule Packages wacore is split into several workspace crates: ### wacore-binary **Location:** `wacore/binary` Binary protocol encoding/decoding using WhatsApp's custom format. ```rust theme={null} use wacore_binary::{ CompactString, jid::Jid, node::Node, builder::NodeBuilder, marshal::{marshal, unmarshal_ref}, }; let node = NodeBuilder::new("message") .attr("type", "text") .attr("to", "15551234567@s.whatsapp.net") .build(); let bytes = marshal(&node)?; let decoded = unmarshal_ref(&bytes)?; ``` **Key exports:** * `CompactString` - Re-export of `compact_str::CompactString`, used by `Jid.user`, `NodeValue::String`, and `NodeContent::String` * `jid::{Jid, JidRef, Server, JidExt}` - WhatsApp JID types. `Server` is an enum (`#[repr(u8)]`) with variants for all known WhatsApp server domains (`Pn`, `Lid`, `Group`, `Broadcast`, `Newsletter`, `Hosted`, `HostedLid`, `Messenger`, `Interop`, `Bot`, `Legacy`). `JidExt` provides helper methods (`is_group()`, `is_newsletter()`, etc.) for both owned and borrowed JID types. `jid::parse_jid_ref(s: &str) -> Option>` parses the common user/group/LID/bot shapes directly into a borrowed `JidRef` with no allocation; it returns `None` for edge cases the compatibility fallback handles, so callers that need those cases parse via `s.parse::()` instead. `Jid`'s own `FromStr` impl is built on top of `parse_jid_ref` * `node::{Node, NodeRef, NodeStr, NodeValue, ValueRef, OwnedNodeRef, AttrsVec}` - Protocol node types. `Node` (owned) for building outgoing stanzas, `NodeRef` (borrowed) for reading received stanzas, `NodeStr` for borrowed-or-inline decoded strings, `OwnedNodeRef` for yoke-based zero-copy self-referential nodes shared as `Arc`. `AttrsVec` is `SmallVec<[(Cow<'static, str>, NodeValue); 2]>`, the inline-capable backing store for `Attrs` (≤2 attrs stay on the stack; see [Binary Protocol](/advanced/binary-protocol#inline-attribute-storage)). The entire `NodeRef` type family (`NodeRef`, `NodeStr`, `ValueRef`, `JidRef`, `NodeContentRef`, `OwnedNodeRef`) implements `serde::Serialize`, producing output identical to their owned counterparts — enabling zero-copy serialization without converting to `Node` first * `builder::NodeBuilder` - Fluent node builder with `new(&'static str)` / `new_dynamic(String)`, `attr()`, `jid_attr()`, `children()`, `bytes()`, `string_content()`, and `apply_content()` chaining methods * `marshal::*` - Binary marshaling functions * `attrs::{AttrParser, AttrParserRef}` - Attribute parsing utilities for owned `Node` and borrowed `NodeRef` respectively * `token` - Token dictionary ### wacore-libsignal **Location:** `wacore/libsignal` Signal Protocol implementation for end-to-end encryption. ```rust theme={null} use wacore::libsignal::{ core::SessionCipher, protocol::{PreKeyBundle, PublicKey}, store::SessionStore, }; ``` **Key modules:** * `core` - Core session cipher logic * `crypto` - Cryptographic primitives (HKDF, HMAC, AES) * `protocol` - Protocol message types * `store` - Store trait definitions See [Signal Protocol](/advanced/signal-protocol) for encryption details. ### wacore-noise **Location:** `wacore/noise` Noise Protocol implementation for handshake encryption. Supports the XX, IK, and XXfallback patterns to match WhatsApp Web's behavior on cold start, resumed reconnects, and server-driven recovery. ```rust theme={null} use wacore::noise::{ NoiseHandshake, HandshakeUtils, XxHandshakeState, IkHandshakeState, XxFallbackHandshakeState, IkServerHelloOutcome, VerifiedServerCertChain, build_handshake_header, }; ``` **Key exports:** * `NoiseState` - Generic Noise state machine * `NoiseHandshake` - WhatsApp-specific handshake wrapper * `XxHandshakeState` - Three-message XX handshake (cold start / fallback) * `IkHandshakeState` - Resumed IK handshake using a cached server static * `XxFallbackHandshakeState` - Server-driven recovery from a stale IK static, continuing the existing transcript * `IkServerHelloOutcome` - Either `Continue` (IK succeeds) or `Fallback` (pivot into XXfallback) * `VerifiedServerCertChain` - Output of XX/XXfallback; persisted by the client to enable IK on the next connect * `HandshakeUtils` - Protocol message building/parsing * `framing` - WebSocket frame encoding * `build_edge_routing_preintro` - Edge routing helper See [WebSocket & Noise Protocol — Noise Protocol Handshake](/advanced/websocket-handling#noise-protocol-handshake) for the full pattern selection and failure-handling logic. ### wacore-appstate **Location:** `wacore/appstate` App state synchronization for contacts, settings, and metadata. ```rust theme={null} use wacore::appstate::{ process_snapshot, process_patch, expand_app_state_keys, Mutation, }; ``` **Key exports:** * `process_snapshot` - Process full state snapshots * `process_patch` - Apply incremental patches * `Mutation` - State mutation records * `LTHash` - LTHash implementation for integrity * `expand_app_state_keys` - Key derivation ### wacore-derive **Location:** `wacore/derive` Procedural macros for protocol node generation. ```rust theme={null} use wacore::{EmptyNode, ProtocolNode, StringEnum}; #[derive(EmptyNode)] #[protocol(tag = "participants")] pub struct ParticipantsRequest; #[derive(ProtocolNode)] #[protocol(tag = "query")] pub struct QueryRequest { #[attr(name = "request", default = "interactive")] pub request_type: String, } #[derive(StringEnum)] pub enum Action { #[str = "block"] Block, #[str = "unblock"] Unblock, } ``` ## Usage in main library The main `whatsapp-rust` crate uses wacore modules throughout: ```rust theme={null} // Binary protocol use wacore_binary::jid::Jid; use wacore_binary::builder::NodeBuilder; // Signal Protocol use wacore::libsignal::store::SessionStore; use wacore::libsignal::protocol::PreKeyBundle; // App state use wacore::appstate::{ process_snapshot, Mutation, }; // IQ specs use wacore::iq::privacy as privacy_settings; // Store traits use wacore::store::traits::Backend; // Protobuf helpers use wacore::proto_helpers; ``` ## Design Principles ### Platform-Agnostic No dependencies on: * Tokio or any async runtime — uses only `futures`, `async-trait`, `async-lock`, `async-channel` * Specific database implementations * File system operations This allows users to provide their own: * **Runtime:** Tokio (default), async-std, smol, WASM, etc. — implement `Runtime` (4 methods) * **Storage:** SQLite (default), PostgreSQL, in-memory, etc. — implement `Backend` (4 sub-traits) * **Transport:** Tokio WebSocket (default), custom protocols — implement `TransportFactory` + `Transport` * **HTTP client:** ureq (default), reqwest, surf, etc. — implement `HttpClient` ### Type Safety Strong typing throughout: * `Jid` with `Server` enum for WhatsApp identifiers — server type is an enum variant, not a string * `Node` / `NodeRef` for protocol messages (owned / borrowed) * Validated newtypes (e.g., `GroupSubject` with length limits) * Enum variants with `StringEnum` for protocol values ### Zero-copy where possible * `Cow<'static, str>` for owned `Node.tag` and `Attrs` keys — known protocol strings (from the token dictionary) are borrowed as static references with zero heap allocation, while unknown strings fall back to owned `String` * `NodeStr<'a>` for borrowed `NodeRef.tag`, `AttrsRef` keys, `ValueRef::String`, and `JidRef.user` — a borrowed-or-inline string type where the `Owned` variant uses `CompactString` (inline up to 24 bytes) instead of heap-allocated `String`, reducing allocation pressure during decoding * `Server` enum (`#[repr(u8)]`) for `Jid.server` — a `Copy` type that requires zero allocation, replacing the previous `Cow<'static, str>` string-based server field * `OwnedNodeRef` — yoke-based self-referential node that owns the decompressed network buffer while `NodeRef` borrows string/byte payloads directly from it. Received stanzas flow through the system as `Arc` for cheap shared zero-copy access * **Zero-copy `Serialize`** — the entire `NodeRef` type family (`NodeRef`, `NodeStr`, `ValueRef`, `JidRef`, `NodeContentRef`, `OwnedNodeRef`) implements `serde::Serialize`, producing output identical to their owned counterparts. This allows serializing received stanzas directly from the network buffer without converting to owned `Node` types first. See [Binary Protocol — Zero-copy serialization](/advanced/binary-protocol#zero-copy-serialization) for details * `NodeRef` for borrowed node parsing * `AttrParserRef` for attribute iteration * `marshal_ref` for encoding without cloning ## Benchmarks The project includes two categories of benchmarks: **protocol-level** (iai-callgrind, deterministic instruction counts) and **integration-level** (real client operations with allocation tracking). ### Protocol benchmarks (iai-callgrind) wacore includes a suite of [iai-callgrind](https://github.com/iai-callgrind/iai-callgrind) benchmarks that measure instruction counts for core protocol operations. These benchmarks run under Valgrind's Callgrind tool, producing deterministic, low-noise measurements that are suitable for CI regression tracking. #### Prerequisites Running the protocol benchmarks requires: * **Nightly Rust** (the project pins `nightly-2026-04-05`) * **Valgrind** installed on your system * **iai-callgrind-runner** (`cargo install iai-callgrind-runner --version 0.16.1`) #### Available suites | Suite | Location | What it measures | | --------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `send_receive_benchmark` | `wacore/benches/send_receive_benchmark.rs` | Full send/receive pipeline — DM send, DM receive, group send (steady-state skmsg), group send with SKDM distribution (10/50/256 participants), and group receive. Uses real `prepare_peer_stanza` and `prepare_group_stanza` functions with in-memory Signal stores | | `reporting_token_benchmark` | `wacore/benches/reporting_token_benchmark.rs` | Reporting token generation — key derivation, token calculation, and full generation pipeline for simple and extended messages | | `binary_benchmark` | `wacore/binary/benches/binary_benchmark.rs` | Binary protocol encoding/decoding — marshal and unmarshal operations for various node sizes and structures | | `libsignal_benchmark` | `wacore/libsignal/benches/libsignal_benchmark.rs` | Signal Protocol operations — session establishment, message encrypt/decrypt, sender key distribution, and group cipher operations | #### Running protocol benchmarks ```bash theme={null} # Run all benchmarks across the workspace cargo bench --workspace # Run a specific benchmark suite cargo bench -p wacore --bench send_receive_benchmark cargo bench -p wacore --bench reporting_token_benchmark cargo bench -p wacore-binary --bench binary_benchmark cargo bench -p wacore-libsignal --bench libsignal_benchmark ``` Each benchmark produces instruction counts and optionally generates flamegraph SVGs for profiling hotspots. ### Integration benchmarks The `bench-integration` test suite (`tests/bench-integration/`) measures real-world client operations end-to-end, including wall-clock time and heap allocation counts. It uses a custom `CountingAlloc` global allocator that tracks every allocation and byte count, then produces `customSmallerIsBetter` JSON output for CI trend tracking. #### Scenarios | Scenario | What it measures | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `connect_to_ready` | Client creation through `Connected` event — includes transport connect, Noise handshake, and initial sync | | `send_message` | Single DM send (sender side) — protobuf encoding, Signal encrypt, node marshal, and WebSocket write. Also measures amortized cost over 20 consecutive sends | | `send_and_receive_message` | Full round-trip — send a DM and wait for delivery on the receiver. Also measures amortized cost over 20 round-trips | | `reconnect` | Disconnect and reconnect cycle — measures the time and allocations to re-establish a session | Each scenario reports three metrics: `alloc_count` (number of heap allocations), `alloc_bytes` (total bytes allocated), and `wall_ms` (elapsed wall-clock time). Amortized scenarios divide totals by the number of iterations for per-operation costs. #### Running integration benchmarks Integration benchmarks require a mock WhatsApp server (e.g., [Bartender](https://github.com/whiskeysockets-devtools/bartender)): ```bash theme={null} # Start the mock server (Docker) docker run -d -p 8080:8080 ghcr.io/whiskeysockets-devtools/bartender:latest # Run the benchmarks MOCK_SERVER_URL="wss://127.0.0.1:8080/ws/chat" \ cargo run -p bench-integration --release ``` The output is a JSON array of `customSmallerIsBetter` entries printed to stdout, suitable for consumption by [github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark). Integration benchmarks require the `danger-skip-tls-verify` feature, which is enabled automatically via the `bench-integration` crate's `Cargo.toml`. You can also enable DHAT heap profiling with `--features dhat-heap` for detailed allocation flamegraphs. ### Allocation optimizations The library includes several allocation-reduction strategies that the integration benchmarks track: * **Thread-local zlib pool** — The binary protocol decompressor (`decompress_zlib_pooled`) reuses a thread-local `flate2::Decompress` instance (\~48 KB internal state) and output buffer across calls, avoiding per-frame heap allocations * **`CompactString` for JIDs** — JID user fields use `compact_str::CompactString` which stores strings up to 24 bytes inline (no heap allocation), covering the vast majority of phone numbers and LID identifiers * **`Server` enum** — JID server fields use a `#[repr(u8)]` enum instead of heap-allocated strings, making JID construction and comparison zero-allocation * **Zero-copy node decoding** — Received stanzas are decoded as `NodeRef` borrowing directly from the network buffer via `OwnedNodeRef` (yoke-based self-referential type), avoiding cloning string/byte payloads during decode * **`Arc` sharing in `LidPnEntry`** — `LidPnEntry.lid` and `.phone_number` are `Arc` instead of `String`. `LidPnCache` reuses the entry's own `Arc`s as the map keys for both lookup directions, so each identifier is allocated once per mapping rather than once as key and again inside the entry. Because this cache is unbounded by design, the saving compounds over the contact base (\~40–80 B per mapping, \~0.5–1 MB for 10k contacts). Constructors accept `impl Into>`, so `String` and `&str` call sites are unaffected; direct field reads return `Arc` — use `&*entry.lid` for `&str` comparisons. Wire/persistence format is unchanged. * **Pre-allocated buffers** — History sync decompression uses a `compressed_size_hint` with a 4x multiplier for buffer pre-allocation, reducing `Vec` reallocation during decompression * **Inline attribute storage** — `Attrs` uses `AttrsVec` (`SmallVec<[...; 2]>`) instead of `Vec`. Nodes with ≤2 attributes (the common per-recipient fanout shapes `to` and `enc`) keep their attributes on the stack alongside the node, eliminating the per-node allocation for attribute storage. Allocation count: −27% per DM stanza (15→11 allocs); −40% for group fanout with 800 participants (4012→2412 allocs) * **Secret-presence pre-scan in history sync** — before buffa decodes a `HistorySyncMsg` (30+ fields, String allocations), a shallow varint walk checks whether the message carries `message_secret` at any level. Messages without a secret are skipped entirely. Bench (20k messages, secret-dense fixture): −29.5% allocations (56,020 → 39,520), −4.3% allocated bytes. Production blobs are secret-sparse so the saving is larger in practice * **Inline storage in `HistoryMsgSecretRecord`** — `chat_id` is `Arc` (allocated once per conversation, shared across all records in that conversation), `msg_id` is `CompactString` (inline for typical 20–22 char WA IDs on 64-bit targets; smaller inline limit on 32-bit/wasm32), and `secret` is `SecretBytes` (inline for secrets ≤32 bytes, heap for larger) * **Filter-before-materialize in history sync** — `process_history_sync_bytes_filtered` runs a caller-supplied retention predicate against a borrowed [`HistoryMsgSecretRecordRef`](#history_sync-types) before the owned `HistoryMsgSecretRecord` is built; a rejected record is never materialized. Bench (500-conversation blob, upstream PR's rejection-heavy fixture): allocation churn 21.61 MiB → 14.00 MiB, allocation count \~84k → \~26k. The reduction scales with how much of the record set the predicate rejects — the default `process_history_sync_bytes` (accept-all) sees none of it * **Streaming visitor/sink for history-sync records** — `process_history_sync_bytes_with_record_visitor` and `process_history_sync_bytes_with_record_sink` (via the [`HistoryMsgSecretRecordVisitor`](#history_sync-types) trait) go a step further than the filter predicate above. You can build your own storage row directly from the borrowed `HistoryMsgSecretRecordRef`. This ensures the intermediate owned `HistoryMsgSecretRecord` is never allocated at all. Bench (allocator-instrumented synthetic history extraction): 20.20 MiB → 14.43 MiB allocated (-28.6%); CodSpeed history stream-drain memory: 243.8 KB → 115.2 KB (2.1× less) * **Borrowed JID parsing** — [`jid::parse_jid_ref`](#wacore-binary) parses common protocol JIDs into a `JidRef` with zero allocation, falling back to `Jid`'s owned parser only for edge cases. Cut allocations 4,093 → 97 in the history-sync task's JID classification path * **In-place buffered media decrypt** — non-streaming `HttpClient` downloads now authenticate the already-buffered response body and decrypt AES-256-CBC in place (`DownloadUtils::verify_and_decrypt_in_place`, see [download](/api/download#downloadutils)), truncating the MAC/padding tail instead of allocating a second file-sized output buffer. Streaming clients are unaffected. Buffered download/decrypt span: 4.355 MiB → 2.146 MiB ### CI integration The repository includes GitHub Actions workflows for both benchmark types: **Protocol benchmarks** (`.github/workflows/benchmark.yml`): * Runs on every push to `main` and on pull requests * Uses [github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark) to store baselines and generate trend charts on GitHub Pages * A custom parser script (`.github/scripts/iai-to-benchmark-json.py`) converts iai-callgrind instruction counts into the `customSmallerIsBetter` JSON format * Pull requests receive a collapsed PR comment (via `.github/scripts/bench-comment.py`) grouping benchmarks into **regressions** (>5% slower), **improvements** (>5% faster), and **unchanged** * The PR comment is idempotent — subsequent benchmark runs update the existing comment instead of creating duplicates **Integration benchmarks** (`.github/workflows/bench-integration.yml`): * Runs on every push to `main` and on pull requests * Spins up a Bartender mock server as a Docker service container * Measures allocation counts, allocation bytes, and wall-clock time for each scenario * Pushes to `main` store the baseline for trend tracking * Pull requests compare against the baseline Raw benchmark output for both suites is uploaded as artifacts (retained for 30 days) for debugging. ## Next steps Protocol Buffers message definitions Type-safe protocol node pattern IqSpec request/response pairing Device state and commands