Overview
whatsapp-rust uses a trait-based storage system to persist device state, cryptographic keys, and protocol metadata. The storage layer is split into five domain-specific traits:
- SignalStore - Signal protocol cryptographic operations (identity keys, sessions, pre-keys, sender keys)
- AppSyncStore - WhatsApp app state synchronization (sync keys, versions, mutation MACs)
- ProtocolStore - WhatsApp protocol alignment (SKDM tracking, LID-PN mapping, device registry)
- MsgSecretStore -
messageSecret persistence for poll/edit/bot-reply decryption (added in v0.6)
- DeviceStore - Device persistence operations
All five traits are combined into the Backend trait for convenience.
The backend trait
Any type implementing all five domain traits automatically implements Backend:
pub trait Backend:
SignalStore + AppSyncStore + ProtocolStore + MsgSecretStore + DeviceStore + Send + Sync {}
impl<T> Backend for T
where
T: SignalStore + AppSyncStore + ProtocolStore + MsgSecretStore + DeviceStore + Send + Sync
{}
MsgSecretStore became a required member of Backend in v0.6, so a custom backend must implement it (the bundled SqliteStore already does). Its methods have defaults that keep the surface small — see MsgSecretStore for which methods you actually need to write.
SignalStore Trait
Handles Signal protocol cryptographic storage for end-to-end encryption.
Identity Operations
/// Store an identity key for a remote address
async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()>;
/// Load an identity key for a remote address
async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>>;
/// Delete an identity key
async fn delete_identity(&self, address: &str) -> Result<()>;
Session Operations
/// Get an encrypted session for an address
async fn get_session(&self, address: &str) -> Result<Option<Vec<u8>>>;
/// Store an encrypted session
async fn put_session(&self, address: &str, session: &[u8]) -> Result<()>;
/// Delete a session
async fn delete_session(&self, address: &str) -> Result<()>;
/// Check if a session exists (default implementation uses get_session)
async fn has_session(&self, address: &str) -> Result<bool>;
PreKey Operations
/// Store a pre-key
async fn store_prekey(&self, id: u32, record: &[u8], uploaded: bool) -> Result<()>;
/// Store multiple pre-keys in a single batch operation (default loops over store_prekey)
async fn store_prekeys_batch(&self, keys: &[(u32, Vec<u8>)], uploaded: bool) -> Result<()>;
/// Load a pre-key by ID
async fn load_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;
/// Load multiple pre-keys by ID in a single batch operation (default loops over load_prekey)
async fn load_prekeys_batch(&self, ids: &[u32]) -> Result<Vec<(u32, Vec<u8>)>>;
/// Remove a pre-key
async fn remove_prekey(&self, id: u32) -> Result<()>;
/// Get the highest pre-key ID currently stored
async fn get_max_prekey_id(&self) -> Result<u32>;
Signed PreKey operations
/// Store a signed pre-key
async fn store_signed_prekey(&self, id: u32, record: &[u8]) -> Result<()>;
/// Load a signed pre-key by ID
async fn load_signed_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;
/// Load all signed pre-keys (returns id, record pairs)
async fn load_all_signed_prekeys(&self) -> Result<Vec<(u32, Vec<u8>)>>;
/// Remove a signed pre-key
async fn remove_signed_prekey(&self, id: u32) -> Result<()>;
Sender key operations
For group messaging encryption:
/// Store a sender key for group messaging
async fn put_sender_key(&self, address: &str, record: &[u8]) -> Result<()>;
/// Get a sender key
async fn get_sender_key(&self, address: &str) -> Result<Option<Vec<u8>>>;
/// Delete a sender key
async fn delete_sender_key(&self, address: &str) -> Result<()>;
AppSyncStore Trait
Handles WhatsApp app state synchronization storage.
Sync key operations
/// Get an app state sync key by ID
async fn get_sync_key(&self, key_id: &[u8]) -> Result<Option<AppStateSyncKey>>;
/// Set an app state sync key
async fn set_sync_key(&self, key_id: &[u8], key: AppStateSyncKey) -> Result<()>;
/// Get the latest sync key ID
async fn get_latest_sync_key_id(&self) -> Result<Option<Vec<u8>>>;
Version Tracking
/// Get the app state version for a collection
async fn get_version(&self, name: &str) -> Result<HashState>;
/// Set the app state version for a collection
async fn set_version(&self, name: &str, state: HashState) -> Result<()>;
Mutation MAC Operations
/// Store mutation MACs for a version
async fn put_mutation_macs(
&self,
name: &str,
version: u64,
mutations: &[AppStateMutationMAC],
) -> Result<()>;
/// Get a mutation MAC by index
async fn get_mutation_mac(&self, name: &str, index_mac: &[u8]) -> Result<Option<Vec<u8>>>;
/// Batch variant of get_mutation_mac: fetch many previous-MAC values in one
/// call, returning index_mac -> value_mac. The default loops over
/// get_mutation_mac; backends with set-membership queries (SQL `IN (...)`)
/// should override to avoid an N+1 (one round-trip per mutation) during sync.
async fn get_mutation_macs(
&self,
name: &str,
index_macs: &[Vec<u8>],
) -> Result<HashMap<Vec<u8>, Vec<u8>>>;
/// Delete mutation MACs by their index MACs
async fn delete_mutation_macs(&self, name: &str, index_macs: &[Vec<u8>]) -> Result<()>;
get_mutation_macs was added in v0.6 to collapse the app-state sync’s per-mutation previous-MAC lookups (which were N+1) into a single batched query. It has a default implementation, so existing custom backends keep working — override it with a WHERE index_mac IN (…) query for the performance win. The SQLite store chunks the IN list at 500 entries.
ProtocolStore Trait
Handles WhatsApp protocol alignment and tracking.
Per-device sender key tracking
Tracks sender key distribution status per device in groups, matching WhatsApp Web’s participant.senderKey Map<deviceJid, boolean> model. Each device has a boolean indicating whether it holds a valid sender key (true) or needs a fresh SKDM (false).
/// Get sender key distribution status for all known devices in a group.
/// Returns (device_jid_string, has_key) pairs.
async fn get_sender_key_devices(&self, group_jid: &str) -> Result<Vec<(String, bool)>>;
/// Set sender key status for devices. Use has_key=true after successful
/// SKDM distribution (WA Web: markHasSenderKey), or has_key=false to mark
/// devices as needing fresh SKDM (WA Web: markForgetSenderKey).
async fn set_sender_key_status(&self, group_jid: &str, entries: &[(&str, bool)]) -> Result<()>;
/// Clear all sender key device tracking for a group (on sender key rotation).
async fn clear_sender_key_devices(&self, group_jid: &str) -> Result<()>;
LID-PN Mapping
Manages mappings between LID (Locally Indexed Device) and phone numbers:
/// Get a mapping by LID
async fn get_lid_mapping(&self, lid: &str) -> Result<Option<LidPnMappingEntry>>;
/// Get a mapping by phone number (returns the most recent LID)
async fn get_pn_mapping(&self, phone: &str) -> Result<Option<LidPnMappingEntry>>;
/// Store or update a LID-PN mapping
async fn put_lid_mapping(&self, entry: &LidPnMappingEntry) -> Result<()>;
/// Get all LID-PN mappings (for cache warm-up)
async fn get_all_lid_mappings(&self) -> Result<Vec<LidPnMappingEntry>>;
Base key collision detection
/// Save the base key for a session address during retry collision detection
async fn save_base_key(&self, address: &str, message_id: &str, base_key: &[u8]) -> Result<()>;
/// Check if the current session has the same base key as the saved one
async fn has_same_base_key(
&self,
address: &str,
message_id: &str,
current_base_key: &[u8],
) -> Result<bool>;
/// Delete a base key entry
async fn delete_base_key(&self, address: &str, message_id: &str) -> Result<()>;
Device Registry
/// Update the device list for a user (called after usync responses)
async fn update_device_list(&self, record: DeviceListRecord) -> Result<()>;
/// Batched variant — update the device list for many users in one call,
/// used by the parallelized group encrypt fan-out so the device-registry
/// write doesn't serialize per-recipient.
async fn update_device_lists(&self, records: Vec<DeviceListRecord>) -> Result<()>;
/// Get all known devices for a user
async fn get_devices(&self, user: &str) -> Result<Option<DeviceListRecord>>;
TcToken Storage
Trusted contact privacy tokens:
/// Get a trusted contact token for a JID (stored under LID)
async fn get_tc_token(&self, jid: &str) -> Result<Option<TcTokenEntry>>;
/// Store or update a trusted contact token for a JID
async fn put_tc_token(&self, jid: &str, entry: &TcTokenEntry) -> Result<()>;
/// Delete a trusted contact token for a JID
async fn delete_tc_token(&self, jid: &str) -> Result<()>;
/// Get all JIDs that have stored tc tokens
async fn get_all_tc_token_jids(&self) -> Result<Vec<String>>;
/// Delete tc tokens with token_timestamp older than cutoff (returns count deleted)
async fn delete_expired_tc_tokens(&self, cutoff_timestamp: i64) -> Result<u32>;
Sent message store
Persists sent message payloads for retry handling. Matches WhatsApp Web’s getMessageTable pattern where retry receipts look up the original message from storage.
/// Store a sent message's serialized payload for retry handling.
/// Called after each send_message(); the payload is the protobuf-encoded Message.
async fn store_sent_message(
&self,
chat_jid: &str,
message_id: &str,
payload: &[u8],
) -> Result<()>;
/// Retrieve and delete a sent message (atomic take). Returns serialized payload.
/// Called when a retry receipt arrives; consuming prevents double-retry.
async fn take_sent_message(
&self,
chat_jid: &str,
message_id: &str,
) -> Result<Option<Vec<u8>>>;
/// Delete sent messages older than cutoff (unix timestamp seconds).
/// Returns count deleted.
async fn delete_expired_sent_messages(
&self,
cutoff_timestamp: i64,
) -> Result<u32>;
The take_sent_message method is an atomic read-and-delete operation. Once a message payload is taken for retry, it is removed from storage to prevent double-retry. For status broadcasts where multiple devices may retry, the client re-adds the message after taking it.
MsgSecretStore
The fifth required member of Backend. It persists the 32-byte messageSecret values needed to decrypt later add-ons keyed off an original message: poll votes, poll/event edits, message edits (secret_encrypted_message), and Meta AI / fbid bot replies (<enc type="msmsg">). Secrets are keyed by (chat, sender, msg_id) and carry an absolute expiry so they can be pruned by policy (see messageSecret retention).
/// Store a single secret with no expiry (expires_at = 0, kept forever).
async fn put_msg_secret(
&self,
chat: &str,
sender: &str,
msg_id: &str,
secret: &[u8],
) -> Result<()>;
/// Batch upsert. Each MsgSecretEntry carries its own absolute expires_at
/// deadline and parent message_ts. On conflict the later deadline wins
/// (0 = never) and the later non-zero message_ts wins. Returns count stored.
async fn put_msg_secrets(&self, entries: Vec<MsgSecretEntry>) -> Result<usize>;
/// Look up a secret for decryption.
async fn get_msg_secret(
&self,
chat: &str,
sender: &str,
msg_id: &str,
) -> Result<Option<Vec<u8>>>;
/// Look up a secret together with the parent message's event time, used to
/// enforce the per-add-on edit window.
async fn get_msg_secret_with_ts(
&self,
chat: &str,
sender: &str,
msg_id: &str,
) -> Result<Option<(Vec<u8>, i64)>>;
/// Delete secrets whose expires_at <= cutoff. Rows with expires_at = 0 are
/// kept forever. Returns count removed.
async fn delete_expired_msg_secrets(&self, cutoff_timestamp: i64) -> Result<u32>;
MsgSecretEntry is { chat, sender, msg_id, secret, expires_at, message_ts }. The SQLite table is:
CREATE TABLE msg_secrets (
chat TEXT NOT NULL,
sender TEXT NOT NULL,
msg_id TEXT NOT NULL,
secret BLOB NOT NULL,
device_id INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now')),
expires_at INTEGER NOT NULL DEFAULT 0, -- absolute unix-seconds deadline (0 = never)
message_ts INTEGER NOT NULL DEFAULT 0, -- parent message event time (edit-window check)
PRIMARY KEY (chat, sender, msg_id, device_id)
);
CREATE INDEX idx_msg_secrets_expires ON msg_secrets (device_id, expires_at);
A custom backend only needs to implement three methods: put_msg_secrets, get_msg_secret, and delete_expired_msg_secrets. The other two have defaults — put_msg_secret delegates to put_msg_secrets with expires_at = 0, and get_msg_secret_with_ts pairs get_msg_secret with a 0 timestamp. Override get_msg_secret_with_ts only if your store persists message_ts and you want the edit-window enforced.
DeviceStore Trait
Handles device data persistence:
/// Save device data
async fn save(&self, device: &Device) -> Result<()>;
/// Load device data
async fn load(&self) -> Result<Option<Device>>;
/// Check if a device exists
async fn exists(&self) -> Result<bool>;
/// Create a new device row and return its generated device_id
async fn create(&self) -> Result<i32>;
/// Create a snapshot of the database state
/// Optional: label with name, save extra_content (e.g. failing message)
async fn snapshot_db(&self, name: &str, extra_content: Option<&[u8]>) -> Result<()>;
SqliteStore implementation
The default storage implementation using SQLite with Diesel ORM. SQLite is bundled by default — you don’t need it installed on your system.
Bundled SQLite
The whatsapp-rust-sqlite-storage crate enables the bundled-sqlite feature by default, which compiles SQLite from source and statically links it. To use a system-installed SQLite instead:
[dependencies]
whatsapp-rust-sqlite-storage = { version = "0.6", default-features = false }
Creating a store
use whatsapp_rust::store::SqliteStore;
// Basic usage - creates/opens database at path
let store = SqliteStore::new("whatsapp.db").await?;
// With device_id for multi-device support
let store = SqliteStore::new_for_device("whatsapp.db", 1).await?;
// Using sqlite:// URL format
let store = SqliteStore::new("sqlite://path/to/db.sqlite").await?;
Features
- Connection pooling - Uses Diesel r2d2 with pool size of 2
- WAL mode - Write-Ahead Logging for better concurrency
- Automatic migrations - Runs embedded migrations on startup
- Semaphore-based locking - Prevents concurrent writes
- Retry logic - Automatic retry with exponential backoff for locked database
- Multi-device support - Single database can store multiple device sessions
Database Configuration
SqliteStore automatically configures connections with:
PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 30000;
PRAGMA synchronous = NORMAL;
PRAGMA cache_size = 512;
PRAGMA temp_store = memory;
PRAGMA foreign_keys = ON;
Usage Example
use whatsapp_rust_sqlite_storage::SqliteStore;
use whatsapp_rust::Bot;
use std::sync::Arc;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Create store
let backend = Arc::new(SqliteStore::new("whatsapp.db").await?);
// Use with Bot builder
let mut bot = Bot::builder()
.with_backend(backend.clone())
.with_transport_factory(transport_factory)
.with_http_client(http_client)
.with_runtime(TokioRuntime)
.on_event(|event, client| async move { /* handle events */ })
.build()
.await?;
// Store implements all traits
// SignalStore
backend.put_identity("address@s.whatsapp.net", [0u8; 32]).await?;
let identity = backend.load_identity("address@s.whatsapp.net").await?;
// AppSyncStore
let version = backend.get_version("regular").await?;
// ProtocolStore
let devices = backend.get_sender_key_devices("group@g.us").await?;
// DeviceStore
if backend.exists().await? {
let device = backend.load().await?;
}
Ok(())
}
CacheStore Trait
The CacheStore trait enables pluggable cache backends for the client’s data caches. By default, caches use in-process moka; implementing this trait lets you use Redis, Memcached, or any other external cache.
Location: wacore/src/store/cache.rs
#[async_trait]
pub trait CacheStore: Send + Sync + 'static {
/// Retrieve a cached value by namespace and key.
async fn get(&self, namespace: &str, key: &str) -> anyhow::Result<Option<Vec<u8>>>;
/// Store a value with an optional TTL.
/// When `ttl` is `None`, the entry persists until explicitly deleted.
async fn set(
&self,
namespace: &str,
key: &str,
value: &[u8],
ttl: Option<Duration>,
) -> anyhow::Result<()>;
/// Delete a single key from the given namespace.
async fn delete(&self, namespace: &str, key: &str) -> anyhow::Result<()>;
/// Delete all keys in a namespace.
async fn clear(&self, namespace: &str) -> anyhow::Result<()>;
/// Approximate entry count (diagnostics only). Default returns 0.
async fn entry_count(&self, _namespace: &str) -> anyhow::Result<u64> {
Ok(0)
}
}
Namespaces
Each logical cache uses a unique namespace string. Implementations should partition keys by namespace (e.g., prefix as {namespace}:{key} in Redis).
| Namespace | Cache | Description |
|---|
"group" | group_cache | Group metadata |
"device_registry" | device_registry_cache | Device registry entries |
"lid_pn_by_lid" | lid_pn_cache | LID-to-phone bidirectional mappings |
Error handling
Cache operations are best-effort. The client treats read failures as cache misses and logs warnings on write failures. Implementations should still return errors for observability.
CacheStores configuration
pub struct CacheStores {
pub group_cache: Option<Arc<dyn CacheStore>>,
pub device_registry_cache: Option<Arc<dyn CacheStore>>,
pub lid_pn_cache: Option<Arc<dyn CacheStore>>,
}
Set individual caches or use CacheStores::all(store) to route all pluggable caches to the same backend:
use whatsapp_rust::{CacheConfig, CacheStores};
let redis = Arc::new(MyRedisCacheStore::new("redis://localhost:6379"));
let config = CacheConfig {
cache_stores: CacheStores::all(redis),
..Default::default()
};
See Custom backends — cache store for a full implementation example.
TypedCache
TypedCache<K, V> is a generic wrapper that dispatches to either moka or a custom CacheStore backend.
Location: src/cache_store.rs
| Method | Signature | Description |
|---|
from_moka | fn from_moka(cache: Cache<K, V>) -> Self | Wrap an existing moka cache (zero overhead) |
from_store | fn from_store(store: Arc<dyn CacheStore>, namespace: &'static str, ttl: Option<Duration>) -> Self | Create a cache backed by a custom store |
get | async fn get<Q>(&self, key: &Q) -> Option<V> | Look up a value. Misses and deserialization failures return None |
insert | async fn insert(&self, key: K, value: V) | Insert or update a value |
invalidate | async fn invalidate<Q>(&self, key: &Q) | Remove a single key |
invalidate_all | fn invalidate_all(&self) | Remove all entries (sync). Requires tokio-runtime for custom backends |
clear | async fn clear(&self) | Remove all entries (async). Awaits completion for custom backends |
run_pending_tasks | async fn run_pending_tasks(&self) | Run internal housekeeping (moka only) |
entry_count | fn entry_count(&self) -> u64 | Approximate entry count (sync). Returns 0 for custom backends |
entry_count_async | async fn entry_count_async(&self) -> u64 | Approximate entry count, delegating to custom backend if available |
invalidate_all() on custom CacheStore backends requires the tokio-runtime feature. Without it, the clear is silently skipped. Use the async clear() method as an alternative.
Implementing custom storage
To implement a custom storage backend:
- Implement all four domain traits
- The
Backend trait is automatically implemented
- All methods must be
async and thread-safe (Send + Sync)
Example: Redis store
use async_trait::async_trait;
use redis::aio::ConnectionManager;
use wacore::store::traits::*;
use wacore::store::error::Result;
pub struct RedisStore {
client: ConnectionManager,
device_id: i32,
}
impl RedisStore {
pub async fn new(redis_url: &str) -> Result<Self> {
let client = redis::Client::open(redis_url)
.map_err(|e| StoreError::Connection(Box::new(e)))?;
let conn = client.get_connection_manager().await
.map_err(|e| StoreError::Connection(Box::new(e)))?;
Ok(Self {
client: conn,
device_id: 1,
})
}
}
#[async_trait]
impl SignalStore for RedisStore {
async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()> {
let mut conn = self.client.clone();
let key_name = format!("identity:{}:{}", self.device_id, address);
redis::cmd("SET")
.arg(key_name)
.arg(&key[..])
.query_async(&mut conn)
.await
.map_err(|e| StoreError::Database(Box::new(e)))?;
Ok(())
}
async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>> {
let mut conn = self.client.clone();
let key_name = format!("identity:{}:{}", self.device_id, address);
let result: Option<Vec<u8>> = redis::cmd("GET")
.arg(key_name)
.query_async(&mut conn)
.await
.map_err(|e| StoreError::Database(Box::new(e)))?;
Ok(result)
}
// Implement remaining SignalStore methods...
}
#[async_trait]
impl AppSyncStore for RedisStore {
// Implement all AppSyncStore methods...
}
#[async_trait]
impl ProtocolStore for RedisStore {
// Implement all ProtocolStore methods...
}
#[async_trait]
impl DeviceStore for RedisStore {
// Implement all DeviceStore methods...
}
// Backend is automatically implemented!
Best Practices
- Thread Safety - Use
Arc for shared state, Mutex for mutable state
- Error Handling - Convert backend errors to
StoreError variants
- Transactions - Use database transactions for atomic operations
- Retries - Implement retry logic for transient failures
- Connection Pooling - Reuse connections when possible
- Blocking Operations - Wrap blocking I/O in
tokio::task::spawn_blocking
Data Structures
AppStateSyncKey
pub struct AppStateSyncKey {
pub key_data: Vec<u8>,
pub fingerprint: Vec<u8>,
pub timestamp: i64,
}
LidPnMappingEntry
pub struct LidPnMappingEntry {
pub lid: String, // LID user part
pub phone_number: String, // Phone number user part
pub created_at: i64, // Unix timestamp
pub updated_at: i64, // Unix timestamp
pub learning_source: String, // e.g. "usync", "peer_pn_message"
}
TcTokenEntry
pub struct TcTokenEntry {
pub token: Vec<u8>, // Raw token bytes
pub token_timestamp: i64, // When token was received
pub sender_timestamp: Option<i64>, // When we sent our token
}
DeviceListRecord
pub struct DeviceListRecord {
pub user: String, // User part of JID
pub devices: Vec<DeviceInfo>, // Known devices
pub timestamp: i64, // Last update timestamp
pub phash: Option<String>, // Participant hash from usync
pub raw_id: Option<u32>, // ADV raw_id for identity change detection
}
pub struct DeviceInfo {
pub device_id: u32, // 0 = primary, 1+ = companions
pub key_index: Option<u32>, // Key index if known
}
The raw_id field stores the ADV (Account Device Verification) key index list raw_id from device notifications. When this value changes for a user, it indicates an identity change (e.g., the user reinstalled WhatsApp). The client uses this to detect identity changes and clear Signal sessions for that user’s non-primary devices. Per-device sender key tracking is not wiped globally on identity change — that would empty the tracker too aggressively and feed the no-distribution path on the next group send. SKDM redistribution is instead driven per-group/per-device by retry receipts (matching WhatsApp Web’s WAWebUpdateLocalSignalSession/markForgetSenderKey behavior).
Error Handling
All storage operations return Result<T> from wacore::store::error. Each variant preserves the underlying typed error as its source() so callers can downcast to the original backend error when needed:
pub enum StoreError {
Io(#[from] std::io::Error),
Serialization(#[source] Box<dyn std::error::Error + Send + Sync>),
Validation(String),
Connection(#[source] Box<dyn std::error::Error + Send + Sync>),
Database(#[source] Box<dyn std::error::Error + Send + Sync>),
RetriesExhausted { op: String },
Migration(#[source] Box<dyn std::error::Error + Send + Sync>),
InvalidConfig(String),
DeviceNotFound(i32),
}
pub type Result<T> = std::result::Result<T, StoreError>;
StoreError exposes a helper is_database_busy_or_locked() that walks the source chain looking for SQLite BUSY/LOCKED markers. Retry layers use it to decide whether a database error is transient without depending on a specific backend crate.
See Also