Storage Traits - WhatsApp Rust

Overview

whatsapp-rust uses a trait-based storage system to persist device state, cryptographic keys, and protocol metadata. The storage layer is split into four domain-specific traits:

SignalStore - Signal protocol cryptographic operations (identity keys, sessions, pre-keys, sender keys)
AppSyncStore - WhatsApp app state synchronization (sync keys, versions, mutation MACs)
ProtocolStore - WhatsApp protocol alignment (SKDM tracking, LID-PN mapping, device registry)
DeviceStore - Device persistence operations

All four traits are combined into the Backend trait for convenience.

The Backend Trait

Any type implementing all four domain traits automatically implements Backend:

pub trait Backend: SignalStore + AppSyncStore + ProtocolStore + DeviceStore + Send + Sync {}

impl<T> Backend for T 
where 
    T: SignalStore + AppSyncStore + ProtocolStore + DeviceStore + Send + Sync
{}

SignalStore Trait

Handles Signal protocol cryptographic storage for end-to-end encryption.

Identity Operations

/// Store an identity key for a remote address
async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()>;

/// Load an identity key for a remote address
async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Delete an identity key
async fn delete_identity(&self, address: &str) -> Result<()>;

Session Operations

/// Get an encrypted session for an address
async fn get_session(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Store an encrypted session
async fn put_session(&self, address: &str, session: &[u8]) -> Result<()>;

/// Delete a session
async fn delete_session(&self, address: &str) -> Result<()>;

/// Check if a session exists (default implementation uses get_session)
async fn has_session(&self, address: &str) -> Result<bool>;

PreKey Operations

/// Store a pre-key
async fn store_prekey(&self, id: u32, record: &[u8], uploaded: bool) -> Result<()>;

/// Store multiple pre-keys in a single batch operation (default loops over store_prekey)
async fn store_prekeys_batch(&self, keys: &[(u32, Vec<u8>)], uploaded: bool) -> Result<()>;

/// Load a pre-key by ID
async fn load_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;

/// Load multiple pre-keys by ID in a single batch operation (default loops over load_prekey)
async fn load_prekeys_batch(&self, ids: &[u32]) -> Result<Vec<(u32, Vec<u8>)>>;

/// Remove a pre-key
async fn remove_prekey(&self, id: u32) -> Result<()>;

/// Get the highest pre-key ID currently stored
async fn get_max_prekey_id(&self) -> Result<u32>;

Signed PreKey Operations

/// Store a signed pre-key
async fn store_signed_prekey(&self, id: u32, record: &[u8]) -> Result<()>;

/// Load a signed pre-key by ID
async fn load_signed_prekey(&self, id: u32) -> Result<Option<Vec<u8>>>;

/// Load all signed pre-keys (returns id, record pairs)
async fn load_all_signed_prekeys(&self) -> Result<Vec<(u32, Vec<u8>)>>;

/// Remove a signed pre-key
async fn remove_signed_prekey(&self, id: u32) -> Result<()>;

Sender Key Operations

For group messaging encryption:

/// Store a sender key for group messaging
async fn put_sender_key(&self, address: &str, record: &[u8]) -> Result<()>;

/// Get a sender key
async fn get_sender_key(&self, address: &str) -> Result<Option<Vec<u8>>>;

/// Delete a sender key
async fn delete_sender_key(&self, address: &str) -> Result<()>;

AppSyncStore Trait

Handles WhatsApp app state synchronization storage.

Sync Key Operations

/// Get an app state sync key by ID
async fn get_sync_key(&self, key_id: &[u8]) -> Result<Option<AppStateSyncKey>>;

/// Set an app state sync key
async fn set_sync_key(&self, key_id: &[u8], key: AppStateSyncKey) -> Result<()>;

/// Get the latest sync key ID
async fn get_latest_sync_key_id(&self) -> Result<Option<Vec<u8>>>;

Version Tracking

/// Get the app state version for a collection
async fn get_version(&self, name: &str) -> Result<HashState>;

/// Set the app state version for a collection
async fn set_version(&self, name: &str, state: HashState) -> Result<()>;

Mutation MAC Operations

/// Store mutation MACs for a version
async fn put_mutation_macs(
    &self,
    name: &str,
    version: u64,
    mutations: &[AppStateMutationMAC],
) -> Result<()>;

/// Get a mutation MAC by index
async fn get_mutation_mac(&self, name: &str, index_mac: &[u8]) -> Result<Option<Vec<u8>>>;

/// Delete mutation MACs by their index MACs
async fn delete_mutation_macs(&self, name: &str, index_macs: &[Vec<u8>]) -> Result<()>;

ProtocolStore Trait

Handles WhatsApp protocol alignment and tracking.

Per-device sender key tracking

Tracks sender key distribution status per device in groups, matching WhatsApp Web’s participant.senderKey Map<deviceJid, boolean> model. Each device has a boolean indicating whether it holds a valid sender key (true) or needs a fresh SKDM (false).

/// Get sender key distribution status for all known devices in a group.
/// Returns (device_jid_string, has_key) pairs.
async fn get_sender_key_devices(&self, group_jid: &str) -> Result<Vec<(String, bool)>>;

/// Set sender key status for devices. Use has_key=true after successful
/// SKDM distribution (WA Web: markHasSenderKey), or has_key=false to mark
/// devices as needing fresh SKDM (WA Web: markForgetSenderKey).
async fn set_sender_key_status(&self, group_jid: &str, entries: &[(&str, bool)]) -> Result<()>;

/// Clear all sender key device tracking for a group (on sender key rotation).
async fn clear_sender_key_devices(&self, group_jid: &str) -> Result<()>;

LID-PN Mapping

Manages mappings between LID (Locally Indexed Device) and phone numbers:

/// Get a mapping by LID
async fn get_lid_mapping(&self, lid: &str) -> Result<Option<LidPnMappingEntry>>;

/// Get a mapping by phone number (returns the most recent LID)
async fn get_pn_mapping(&self, phone: &str) -> Result<Option<LidPnMappingEntry>>;

/// Store or update a LID-PN mapping
async fn put_lid_mapping(&self, entry: &LidPnMappingEntry) -> Result<()>;

/// Get all LID-PN mappings (for cache warm-up)
async fn get_all_lid_mappings(&self) -> Result<Vec<LidPnMappingEntry>>;

Base Key Collision Detection

/// Save the base key for a session address during retry collision detection
async fn save_base_key(&self, address: &str, message_id: &str, base_key: &[u8]) -> Result<()>;

/// Check if the current session has the same base key as the saved one
async fn has_same_base_key(
    &self,
    address: &str,
    message_id: &str,
    current_base_key: &[u8],
) -> Result<bool>;

/// Delete a base key entry
async fn delete_base_key(&self, address: &str, message_id: &str) -> Result<()>;

Device Registry

/// Update the device list for a user (called after usync responses)
async fn update_device_list(&self, record: DeviceListRecord) -> Result<()>;

/// Get all known devices for a user
async fn get_devices(&self, user: &str) -> Result<Option<DeviceListRecord>>;

TcToken Storage

Trusted contact privacy tokens:

/// Get a trusted contact token for a JID (stored under LID)
async fn get_tc_token(&self, jid: &str) -> Result<Option<TcTokenEntry>>;

/// Store or update a trusted contact token for a JID
async fn put_tc_token(&self, jid: &str, entry: &TcTokenEntry) -> Result<()>;

/// Delete a trusted contact token for a JID
async fn delete_tc_token(&self, jid: &str) -> Result<()>;

/// Get all JIDs that have stored tc tokens
async fn get_all_tc_token_jids(&self) -> Result<Vec<String>>;

/// Delete tc tokens with token_timestamp older than cutoff (returns count deleted)
async fn delete_expired_tc_tokens(&self, cutoff_timestamp: i64) -> Result<u32>;

Sent message store

Persists sent message payloads for retry handling. Matches WhatsApp Web’s getMessageTable pattern where retry receipts look up the original message from storage.

/// Store a sent message's serialized payload for retry handling.
/// Called after each send_message(); the payload is the protobuf-encoded Message.
async fn store_sent_message(
    &self,
    chat_jid: &str,
    message_id: &str,
    payload: &[u8],
) -> Result<()>;

/// Retrieve and delete a sent message (atomic take). Returns serialized payload.
/// Called when a retry receipt arrives; consuming prevents double-retry.
async fn take_sent_message(
    &self,
    chat_jid: &str,
    message_id: &str,
) -> Result<Option<Vec<u8>>>;

/// Delete sent messages older than cutoff (unix timestamp seconds).
/// Returns count deleted.
async fn delete_expired_sent_messages(
    &self,
    cutoff_timestamp: i64,
) -> Result<u32>;

The take_sent_message method is an atomic read-and-delete operation. Once a message payload is taken for retry, it is removed from storage to prevent double-retry. For status broadcasts where multiple devices may retry, the client re-adds the message after taking it.

DeviceStore Trait

Handles device data persistence:

/// Save device data
async fn save(&self, device: &Device) -> Result<()>;

/// Load device data
async fn load(&self) -> Result<Option<Device>>;

/// Check if a device exists
async fn exists(&self) -> Result<bool>;

/// Create a new device row and return its generated device_id
async fn create(&self) -> Result<i32>;

/// Create a snapshot of the database state
/// Optional: label with name, save extra_content (e.g. failing message)
async fn snapshot_db(&self, name: &str, extra_content: Option<&[u8]>) -> Result<()>;

SqliteStore implementation

The default storage implementation using SQLite with Diesel ORM. SQLite is bundled by default — you don’t need it installed on your system.

Bundled SQLite

The whatsapp-rust-sqlite-storage crate enables the bundled-sqlite feature by default, which compiles SQLite from source and statically links it. To use a system-installed SQLite instead:

Cargo.toml

[dependencies]
whatsapp-rust-sqlite-storage = { version = "0.5", default-features = false }

Creating a store

use whatsapp_rust::store::SqliteStore;

// Basic usage - creates/opens database at path
let store = SqliteStore::new("whatsapp.db").await?;

// With device_id for multi-device support
let store = SqliteStore::new_for_device("whatsapp.db", 1).await?;

// Using sqlite:// URL format
let store = SqliteStore::new("sqlite://path/to/db.sqlite").await?;

Features

Connection pooling - Uses Diesel r2d2 with pool size of 2
WAL mode - Write-Ahead Logging for better concurrency
Automatic migrations - Runs embedded migrations on startup
Semaphore-based locking - Prevents concurrent writes
Retry logic - Automatic retry with exponential backoff for locked database
Multi-device support - Single database can store multiple device sessions

Database Configuration

SqliteStore automatically configures connections with:

PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 30000;
PRAGMA synchronous = NORMAL;
PRAGMA cache_size = 512;
PRAGMA temp_store = memory;
PRAGMA foreign_keys = ON;

Usage Example

use whatsapp_rust_sqlite_storage::SqliteStore;
use whatsapp_rust::Bot;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create store
    let backend = Arc::new(SqliteStore::new("whatsapp.db").await?);
    
    // Use with Bot builder
    let mut bot = Bot::builder()
        .with_backend(backend.clone())
        .with_transport_factory(transport_factory)
        .with_http_client(http_client)
        .with_runtime(TokioRuntime)
        .on_event(|event, client| async move { /* handle events */ })
        .build()
        .await?;
    
    // Store implements all traits
    // SignalStore
    backend.put_identity("address@s.whatsapp.net", [0u8; 32]).await?;
    let identity = backend.load_identity("address@s.whatsapp.net").await?;
    
    // AppSyncStore
    let version = backend.get_version("regular").await?;
    
    // ProtocolStore
    let devices = backend.get_sender_key_devices("group@g.us").await?;
    
    // DeviceStore
    if backend.exists().await? {
        let device = backend.load().await?;
    }
    
    Ok(())
}

CacheStore Trait

The CacheStore trait enables pluggable cache backends for the client’s data caches. By default, caches use in-process moka; implementing this trait lets you use Redis, Memcached, or any other external cache. Location: wacore/src/store/cache.rs

#[async_trait]
pub trait CacheStore: Send + Sync + 'static {
    /// Retrieve a cached value by namespace and key.
    async fn get(&self, namespace: &str, key: &str) -> anyhow::Result<Option<Vec<u8>>>;

    /// Store a value with an optional TTL.
    /// When `ttl` is `None`, the entry persists until explicitly deleted.
    async fn set(
        &self,
        namespace: &str,
        key: &str,
        value: &[u8],
        ttl: Option<Duration>,
    ) -> anyhow::Result<()>;

    /// Delete a single key from the given namespace.
    async fn delete(&self, namespace: &str, key: &str) -> anyhow::Result<()>;

    /// Delete all keys in a namespace.
    async fn clear(&self, namespace: &str) -> anyhow::Result<()>;

    /// Approximate entry count (diagnostics only). Default returns 0.
    async fn entry_count(&self, _namespace: &str) -> anyhow::Result<u64> {
        Ok(0)
    }
}

Namespaces

Each logical cache uses a unique namespace string. Implementations should partition keys by namespace (e.g., prefix as {namespace}:{key} in Redis).

Namespace	Cache	Description
`"group"`	`group_cache`	Group metadata
`"device_registry"`	`device_registry_cache`	Device registry entries
`"lid_pn_by_lid"`	`lid_pn_cache`	LID-to-phone bidirectional mappings

Error handling

Cache operations are best-effort. The client treats read failures as cache misses and logs warnings on write failures. Implementations should still return errors for observability.

CacheStores configuration

pub struct CacheStores {
    pub group_cache: Option<Arc<dyn CacheStore>>,
    pub device_registry_cache: Option<Arc<dyn CacheStore>>,
    pub lid_pn_cache: Option<Arc<dyn CacheStore>>,
}

Set individual caches or use CacheStores::all(store) to route all pluggable caches to the same backend:

use whatsapp_rust::{CacheConfig, CacheStores};

let redis = Arc::new(MyRedisCacheStore::new("redis://localhost:6379"));
let config = CacheConfig {
    cache_stores: CacheStores::all(redis),
    ..Default::default()
};

See Custom backends — cache store for a full implementation example.

TypedCache

TypedCache<K, V> is a generic wrapper that dispatches to either moka or a custom CacheStore backend. Location: src/cache_store.rs

Method	Signature	Description
`from_moka`	`fn from_moka(cache: Cache<K, V>) -> Self`	Wrap an existing moka cache (zero overhead)
`from_store`	`fn from_store(store: Arc<dyn CacheStore>, namespace: &'static str, ttl: Option<Duration>) -> Self`	Create a cache backed by a custom store
`get`	`async fn get<Q>(&self, key: &Q) -> Option<V>`	Look up a value. Misses and deserialization failures return `None`
`insert`	`async fn insert(&self, key: K, value: V)`	Insert or update a value
`invalidate`	`async fn invalidate<Q>(&self, key: &Q)`	Remove a single key
`invalidate_all`	`fn invalidate_all(&self)`	Remove all entries (sync). Requires `tokio-runtime` for custom backends
`clear`	`async fn clear(&self)`	Remove all entries (async). Awaits completion for custom backends
`run_pending_tasks`	`async fn run_pending_tasks(&self)`	Run internal housekeeping (moka only)
`entry_count`	`fn entry_count(&self) -> u64`	Approximate entry count (sync). Returns `0` for custom backends
`entry_count_async`	`async fn entry_count_async(&self) -> u64`	Approximate entry count, delegating to custom backend if available

invalidate_all() on custom CacheStore backends requires the tokio-runtime feature. Without it, the clear is silently skipped. Use the async clear() method as an alternative.

Implementing Custom Storage

To implement a custom storage backend:

Implement all four domain traits
The Backend trait is automatically implemented
All methods must be async and thread-safe (Send + Sync)

Example: Redis Store

use async_trait::async_trait;
use redis::aio::ConnectionManager;
use wacore::store::traits::*;
use wacore::store::error::Result;

pub struct RedisStore {
    client: ConnectionManager,
    device_id: i32,
}

impl RedisStore {
    pub async fn new(redis_url: &str) -> Result<Self> {
        let client = redis::Client::open(redis_url)
            .map_err(|e| StoreError::Connection(e.to_string()))?;
        let conn = client.get_connection_manager().await
            .map_err(|e| StoreError::Connection(e.to_string()))?;
        
        Ok(Self {
            client: conn,
            device_id: 1,
        })
    }
}

#[async_trait]
impl SignalStore for RedisStore {
    async fn put_identity(&self, address: &str, key: [u8; 32]) -> Result<()> {
        let mut conn = self.client.clone();
        let key_name = format!("identity:{}:{}", self.device_id, address);
        redis::cmd("SET")
            .arg(key_name)
            .arg(&key[..])
            .query_async(&mut conn)
            .await
            .map_err(|e| StoreError::Database(e.to_string()))?;
        Ok(())
    }
    
    async fn load_identity(&self, address: &str) -> Result<Option<Vec<u8>>> {
        let mut conn = self.client.clone();
        let key_name = format!("identity:{}:{}", self.device_id, address);
        let result: Option<Vec<u8>> = redis::cmd("GET")
            .arg(key_name)
            .query_async(&mut conn)
            .await
            .map_err(|e| StoreError::Database(e.to_string()))?;
        Ok(result)
    }
    
    // Implement remaining SignalStore methods...
}

#[async_trait]
impl AppSyncStore for RedisStore {
    // Implement all AppSyncStore methods...
}

#[async_trait]
impl ProtocolStore for RedisStore {
    // Implement all ProtocolStore methods...
}

#[async_trait]
impl DeviceStore for RedisStore {
    // Implement all DeviceStore methods...
}

// Backend is automatically implemented!

Best Practices

Thread Safety - Use Arc for shared state, Mutex for mutable state
Error Handling - Convert backend errors to StoreError variants
Transactions - Use database transactions for atomic operations
Retries - Implement retry logic for transient failures
Connection Pooling - Reuse connections when possible
Blocking Operations - Wrap blocking I/O in tokio::task::spawn_blocking

Data Structures

AppStateSyncKey

pub struct AppStateSyncKey {
    pub key_data: Vec<u8>,
    pub fingerprint: Vec<u8>,
    pub timestamp: i64,
}

LidPnMappingEntry

pub struct LidPnMappingEntry {
    pub lid: String,              // LID user part
    pub phone_number: String,     // Phone number user part
    pub created_at: i64,          // Unix timestamp
    pub updated_at: i64,          // Unix timestamp
    pub learning_source: String,  // e.g. "usync", "peer_pn_message"
}

TcTokenEntry

pub struct TcTokenEntry {
    pub token: Vec<u8>,                    // Raw token bytes
    pub token_timestamp: i64,              // When token was received
    pub sender_timestamp: Option<i64>,     // When we sent our token
}

DeviceListRecord

pub struct DeviceListRecord {
    pub user: String,               // User part of JID
    pub devices: Vec<DeviceInfo>,   // Known devices
    pub timestamp: i64,             // Last update timestamp
    pub phash: Option<String>,      // Participant hash from usync
    pub raw_id: Option<u32>,        // ADV raw_id for identity change detection
}

pub struct DeviceInfo {
    pub device_id: u32,           // 0 = primary, 1+ = companions
    pub key_index: Option<u32>,   // Key index if known
}

The raw_id field stores the ADV (Account Device Verification) key index list raw_id from device notifications. When this value changes for a user, it indicates an identity change (e.g., the user reinstalled WhatsApp). The client uses this to detect identity changes and clear all Signal sessions and sender key device tracking for that user, forcing SKDM redistribution on the next group message.

Error Handling

All storage operations return Result<T> from wacore::store::error:

pub enum StoreError {
    Connection(String),    // Connection failures
    Database(String),      // Database operation errors
    Migration(String),     // Migration errors
    Serialization(String), // Serialization/deserialization errors
    NotFound,             // Resource not found
}

pub type Result<T> = std::result::Result<T, StoreError>;

Core

Messaging

Features

Storage & Backends

Core Modules

​Overview

​The Backend Trait

​SignalStore Trait

​Identity Operations

​Session Operations

​PreKey Operations

​Signed PreKey Operations

​Sender Key Operations

​AppSyncStore Trait

​Sync Key Operations

​Version Tracking

​Mutation MAC Operations

​ProtocolStore Trait

​Per-device sender key tracking

​LID-PN Mapping

​Base Key Collision Detection

​Device Registry

​TcToken Storage

​Sent message store

​DeviceStore Trait

​SqliteStore implementation

​Bundled SQLite

​Creating a store

​Features

​Database Configuration

​Usage Example

​CacheStore Trait

​Namespaces

​Error handling

​CacheStores configuration

​TypedCache

​Implementing Custom Storage

​Example: Redis Store

​Best Practices

​Data Structures

​AppStateSyncKey

​LidPnMappingEntry

​TcTokenEntry

​DeviceListRecord

​Error Handling

​See Also

Overview

The Backend Trait

SignalStore Trait

Identity Operations

Session Operations

PreKey Operations

Signed PreKey Operations

Sender Key Operations

AppSyncStore Trait

Sync Key Operations

Version Tracking

Mutation MAC Operations

ProtocolStore Trait

Per-device sender key tracking

LID-PN Mapping

Base Key Collision Detection

Device Registry

TcToken Storage

Sent message store

DeviceStore Trait

SqliteStore implementation

Bundled SQLite

Creating a store

Features

Database Configuration

Usage Example

CacheStore Trait

Namespaces

Error handling

CacheStores configuration

TypedCache

Implementing Custom Storage

Example: Redis Store

Best Practices

Data Structures

AppStateSyncKey

LidPnMappingEntry

TcTokenEntry

DeviceListRecord

Error Handling

See Also