Overview
whatsapp-rust-chat-store is an opt-in crate that materializes the client’s event stream into queryable chat/message history tables, so a UI or stateful bot survives a restart without re-syncing from WhatsApp. It has no UI dependencies of its own.
- Event-sourced. Register
chat_store.handler()on the client; a single write-behind writer task applies batched events in one transaction per drained batch, preserving event order. Covers messages, receipts, server acks/nacks, edits, revokes, reactions, history sync, app-state chat updates (pin/mute/archive/read/star/delete/clear), push names, and contact actions. Sends and locally-originated edits/revokes/reactions — WhatsApp doesn’t fan these back to the sending device — are recorded explicitly; see Recording outgoing messages and Recording local amendments. - Proto as source of truth. Each message row stores the encoded
wa::Messageplus denormalized columns (kind,text,status). New proto fields never require a schema migration. - Query + invalidation, not a second cache. Reads are async queries (keyset pagination for message and chat-list pages — stable under concurrent inserts, never
OFFSET); consumers subscribe to aStoreChangebroadcast and re-query what they display. - Shares the device store’s database file. Writes go through
SqliteStore::shared(), so the chat tables live in the same file as device/session state without a second connection pool contending for the WAL lock. Backing up the full session stays “copy one file”. Multi-account ready (device_idon every row).
This is a separate, opt-in crate — adding it does not change any behavior for existing
whatsapp-rust consumers who don’t depend on it. It has no integration point in the client beyond the existing register_handler mechanism.Installation
ChatStore::new takes a &whatsapp_rust_sqlite_storage::SqliteStore directly (not the whatsapp_rust::store::SqliteStore re-export), so add that crate alongside whatsapp-rust-chat-store:
Cargo.toml
whatsapp-rust-chat-store is not published to crates.io (its schema and query surface are still moving), so it comes from the git source. Pin whatsapp-rust and whatsapp-rust-sqlite-storage to the same revision: a registry copy and a git copy of whatsapp-rust-sqlite-storage are two distinct crates to Cargo, and ChatStore::new only accepts the SqliteStore from the copy it was compiled against.Setting up the store
ChatStore binds to an existing SqliteStore — it runs its own migrations on first use and shares that store’s connection pool and write semaphore via SqliteStore::shared():
ChatStore::new returns Arc<ChatStore> — the intended usage is one long-lived shared handle, held alongside the client for the process lifetime.
Skipping batches your durability hook already committed
If the client also has an inbound durability hook registered, and that hook persists into this sameChatStore, the store materializes each batch twice by default — once from the hook’s write, once from its own handler(). A batch’s hook_committed flag only says a hook committed it, not that it committed it here, so the store never infers this on its own — opt in explicitly:
Recording outgoing messages
Incoming events (messages, receipts, acks) are captured automatically once the handler is registered. Messages the client itself sends need to be recorded explicitly, since sends don’t go through the inbound event pipeline:record_outgoing is synchronous — it only enqueues onto the writer channel and returns, it does not wait for the write to commit. The row starts at MessageStatus::Pending and is lifted by the server ack/receipts that follow. Because the enqueue happens on the same writer queue as inbound events, it cannot race the ack that immediately follows a send. Call flush() if you need to wait for the row to actually land before reading it back.
timestamp is only the optimistic display time shown before the server responds. Once a positive message ack arrives with a server timestamp, the store overwrites the row’s timestamp with that authoritative value and re-sorts the thread and the chat list accordingly — see Outgoing timestamp reconciliation.
chat may be either of a 1:1 peer’s identities (phone number or LID) — see PN/LID identity aliasing for how the store routes and reads across the two.
Outgoing timestamp reconciliation
An outgoing row’s timestamp starts as the local clock value passed torecord_outgoing. When the server’s positive message ack for that id arrives with a timestamp, the store replaces it with the ack’s own t — the same clock inbound messages already use — so a reply that lands quickly no longer sorts above the message it’s answering. An ack that omits t leaves the row’s timestamp untouched.
- Independent of status. The timestamp fix applies whether or not a delivery/read receipt reached the writer first; an earlier receipt advancing the row’s status doesn’t suppress the correction.
- Reorders the thread and the chat list. The row’s new timestamp can change its position in
messages(). The chat list’slast_message_atfollows it too, but only when the reconciled row was already the chat’s most recent activity. If some other message — deleted or not — is already newer than the corrected timestamp,last_message_atstays put and onlylast_message_preview/last_message_kindare recomputed from the newest surviving row. - A nack never reconciles the timestamp.
ack.errorbeing set means the send failed; the row keeps its optimistic local timestamp. As with status transitions generally, a nack only moves status toErrorwhen the row is stillPending— a nack arriving after a delivery/read receipt changes neither the status nor the timestamp. - A reused message id across chats is left alone. If the ack can’t be resolved to one chat (no usable
from, and the id matches an outgoing row in more than one chat for this device) neither row’s timestamp is touched, and the store logs a warning under theChatStore/Acktarget instead of guessing.
An ack for a message row that hasn’t landed yet is no longer silently dropped: it waits in a bounded, TTL’d queue and re-applies as soon as the matching
record_outgoing/inbound insert commits, correcting the row’s status and timestamp retroactively. Every discarded (expired or overflowed) ack still logs a warning under the ChatStore/Ack target, so a loss is never silent.Recording local amendments
WhatsApp fans an edit, sender revoke, or reaction out to the account’s other linked devices, but not back to the device that sent it — so, like sends, these need to be recorded explicitly for the local thread to reflect them immediately instead of waiting for a history sync:MESSAGE_EDIT, a sender revoke, a reaction) and reuses the same apply logic, so a local write keeps the same ordering, tombstone, and PN/LID routing semantics described in Semantics worth knowing — including offline-drain reordering (an edit or revoke recorded before its target has landed still applies once the target arrives) and the monotonic rule that a stale local write can’t undo a newer one.
record_editandrecord_revoketarget the message bytarget_id, the same id passed torecord_outgoing. Atarget_idthat collides with a differently-authored message (a peer happened to reuse the id) is left untouched — only this client’s own message is amended.record_reactiontakes the samewa::MessageKeyshape passed toClient::send_reactionand requirestarget.idto be set. An emptyemojiremoves this client’s own reaction, matching the inbound removal event. The target’s full identity (from_me,participant) is matched, not just its id, with the same PN/LID alias resolution as companion-device identities — a reaction recorded for a message stored under the peer’s other identity, or with a device-suffixedparticipant, still finds it.
record_outgoing — call flush() to await completion.
Waiting for writes to land
ChatStoreError::WriteBatchFailed if the batch containing one of those writes rolled back. Mainly useful in tests, or right before reading your own just-sent message back out.
PN/LID identity aliasing
A 1:1 peer has two interchangeable wire identities — phone number (@s.whatsapp.net) and LID (@lid) — and inbound traffic for the same thread can arrive addressed under either one, regardless of which key the thread’s rows already live under.
- An existing thread keeps its key. Whichever identity addresses it, live traffic (messages, receipts, reactions, app-state updates) routes to the thread that already exists, matching WA Web’s
selectChatForOneOnOneMessage. - A brand-new chat is keyed by LID when the peer already has a known PN↔LID mapping; otherwise it’s keyed by whichever identity first addressed it.
- Every read resolves the alias.
messages,message,reactionsandreceiptsall accept either of the peer’s identities as thechatargument and match rows stored under either key — so a caller that only ever addresses a peer by phone number keeps working even if some rows ended up under the LID key (or vice versa). - Splits heal automatically. If a peer’s rows are split across both keys (for example, from receipts that arrived under the wrong identity before this resolution existed), the next piece of live traffic for that peer merges the pair into one thread — advance-only status/star/revoke/edit conflict resolution, union of reactions and per-state receipts, sticky metadata (pin/mute/archive/name/ephemeral) kept, badge recounted. A state both sides already recorded keeps the earlier of the two timestamps, rather than either side winning arbitrarily. Ties go to the LID side.
record_outgoing, this goes through the writer queue and returns immediately; call flush() to await completion.
Companion-device identities
A peer’s linked device (WhatsApp Web/Desktop) addresses traffic under a device-suffixed JID —10203040506070:48@lid rather than the peer’s bare 10203040506070@lid. Every row the store keys on identity (chats, contacts, message receipts) uses the bare form, so a companion device’s traffic is normalized rather than filed under a key nothing else reads:
- Receipts from a companion device count. Multi-device delivery/read semantics are any-device — WhatsApp emits the receipt once, from whichever of the peer’s devices acted first, and never re-sends it from the primary — so
messages.statusadvances the same whether the ack came from the peer’s phone or their linked device. - One row per participant per state, not per device. A group member reading on their phone and again on Web still produces a single
Readrow inreceipts(), not two. Each reported state gets its own row instead of overwriting the previous one. contact()resolves either form. A caller holding a message’ssender(which keeps its device by design) finds the sameContactEntryas a caller holding the peer’s bare identity — both look up the same row.
Querying
All query methods areasync and run on the shared pool’s blocking thread pool.
chats_page with no cursor — a convenience for a caller that only ever wants the first page.
chats_page is keyset-paginated: pass the ChatCursor of the last chat you already hold (ChatCursor::from(&chat_entry)) to fetch the page after it. The list is two ordered runs concatenated — pinned chats by pin time, then everything else by activity — so both runs stay a plain indexed range scan even across a page boundary, rather than the full scan plus temp B-tree a single combined sort would need.
chat is the point lookup the primary key always supported — resolving an addressed JID back to its store key, or folding one chat’s unread count, without paging through the whole list. It accepts either of a 1:1 peer’s identities the same way messages does, and never synthesizes a merged row across a still-unreconciled PN/LID pair: while a pair is split, sticky metadata can sit on the side this doesn’t return, exactly as chats/chats_page list such a pair as two entries. reconcile_chat is what unions them.
MessageCursor of the oldest message you already hold (MessageCursor::from(&stored_message)) to fetch the page before it. There is no OFFSET-based paging — keyset pagination stays stable under concurrent inserts (e.g. a new message arriving while a chat history is being scrolled).
chat accepts either of a 1:1 peer’s identities (phone number or LID) — see PN/LID identity aliasing.
chats plus messages can otherwise only answer that by paging every thread. messages_by_arrival is messages_by_arrival_in_range with no bounds; the ranged form additionally restricts the feed to a half-open wall-clock window, since <= timestamp < until, so two windows queried at the same instant tile without double-counting or dropping a row. Either bound may be None. A limit of zero, or negative (which SQLite reads as unbounded), returns nothing rather than the whole table.
That tiling guarantee holds for a single scan, not across two queries taken at different times: an outgoing row’s timestamp_ms can be corrected after insert (see Outgoing timestamp reconciliation), independently of seq, so a message can cross a window boundary between an earlier and a later query — missed by both queries, or counted by both, depending on which way it moved. Page by seq with messages_by_arrival instead of chaining wall-clock windows if you need a guarantee that survives a timestamp changing underneath you.
Keyset-paginated by ArrivalCursor, not MessageCursor — the feed sorts by seq alone rather than (timestamp_ms, seq), so a cursor from messages() can’t page it. The intended usage is a loop that re-enters at the head every pass (after: None) and walks down until it recognizes rows it already has, comparing by content — (chat_jid, id) — never by stopping at a remembered seq:
chat_jid through the same PN/LID alias resolution messages/message use before comparing (chat_jid, id) — a stored row’s chat_jid itself is not stable. PN/LID reconciliation merges a split pair by rewriting the losing side’s rows onto the surviving key in place, and like every other mutation, that rewrite leaves seq untouched. A message you’ve already walked past can therefore reappear later under a different chat_jid; comparing the raw column reads that as a new message and reprocesses it.
Stopping at a saved watermark instead skips messages, silently. SQLite hands out the rowid backing seq as max(rowid) + 1: deleting whichever row currently holds the table’s highest rowid frees that number for the next arrival to reuse. delete-for-me and clear-chat — both routine, since they go through this store — delete rows scoped to one chat, not the whole per-device table, so a clear only resets the counter all the way to 1 when the chat it empties happened to hold every remaining row; with other chats still populated, it simply frees whatever rowids that chat held, which is already enough for a later insert to land at or below a remembered watermark. Either way, a message that does so reads as already seen and never surfaces again under a watermark comparison.
Ordered by arrival rather than timestamp_ms for the same reason MessageCursor isn’t reused here: history-sync backfill inserts old conversations at new arrival positions, so a poller keyed on timestamp files that backfill behind its watermark and never looks at it again, while an arrival-keyed one sees it at the head on its next pull.
A revoke, edit, star, or status change rewrites a row in place and leaves its seq untouched — seq is assigned once, by the insert — so a message this feed has already walked past never resurfaces no matter what happens to it afterward. A tombstone or an undecryptable placeholder is a row like any other and does appear, since it’s inserted like one. A consumer that needs to react to mutations, not just arrivals, subscribes to StoreChange::Messages instead.
The wall-clock window is a filter over the arrival scan, not a seek, so cost tracks rows walked rather than rows returned — a narrow window over an old part of a large store still reads everything newer than it before yielding anything.
message, reactions and receipts accept either of a 1:1 peer’s identities the same way messages does. receipts returns one row per participant per state their receipts have reported, not one row per participant. Delivered, Read, and Played are each recorded as a separate row when reported — a Read receipt with no prior Delivered receipt produces only a Read row, not both. Each row’s timestamp is the instant that state was first reported. unread_total sums only positive unread counters, ignoring the -1 manually-marked-unread sentinel on individual chats.
Breaking change:
receipts used to return rows only for group chats. A 1:1’s delivery/read state lived solely in StoredMessage.status, with no per-state timestamp available. It now records the same per-state rows for 1:1 chats too, so a caller can render “Delivered hh:mm” above “Read hh:mm” the way WA Web’s contact message-info drawer does. A caller that assumed at most one row per participant should now expect up to three: Delivered, Read, and Played. They come back oldest state first.put_media_ref after downloading media, then use media_ref to check whether a file with the same file_sha256 was already downloaded before fetching it again.
Subscribing to changes
StoreChange per committed write batch (deduplicated per kind). The channel buffers 256 messages; a Lagged receiver should treat it as “something changed” and re-query whatever it currently displays rather than trying to replay the gap.
StoreChange is a pure invalidation signal — it never carries row data.
Full-text search
With thesearch feature enabled, ChatStore maintains a SQLite FTS5 index over message text and exposes:
ChatStoreError::InvalidSearchQuery.
search_messages_in_chat is the same search restricted to one chat. Scoping happens inside the FTS join rather than filtering a global result afterwards, so a chat that ranks sparsely in the whole store still returns its hits. chat accepts either of a 1:1 peer’s identities (phone number or LID) — see PN/LID identity aliasing.
A query with any token shorter than three characters is ordered newest-first instead of by relevance, for the whole query: ORDER BY rank has to score every row a short prefix matches before limit can discard any, which on a real store means most of it, so ranking on a short prefix would turn one keystroke into a multi-second query. Only a query where every token is three characters or longer ranks by relevance.
The FTS5 virtual table and its sync triggers are created lazily and idempotently in ChatStore::new() when the feature is enabled, not via a migration — a build without search leaves no FTS objects behind.
The FTS triggers use a single delete+insert per row change rather than a
WHEN-guarded variant — the guarded form looks equivalent but corrupts FTS5 rank queries under certain update patterns. Don’t “simplify” the trigger bodies if you’re forking this crate.Types
ChatEntry
ChatCursor
chats_page(). pinned_at_ms is Some when the cursor sits in the pinned run and None for the activity run — the chat list is two ordered runs concatenated rather than one column SQLite can sort on directly, so the cursor records which run it’s in as well as where in that run. Build one with ChatCursor::from(&chat_entry).
StoredMessage
MessageCursor
messages(). seq is the row’s SQLite rowid — the order the socket delivered it — not a value you construct by hand.
Breaking change:
MessageCursor.msg_id: String is now seq: i64. The previous tiebreak sorted same-second messages (the server’s timestamp is whole seconds, so a live back-and-forth often lands several on one value) by comparing message ids, which put a peer’s message above a same-second reply most of the time — outgoing ids carry a fixed generated prefix that isn’t comparable to a peer’s. Same-second messages now resolve in arrival order instead. Always build a cursor with MessageCursor::from(&stored_message) rather than constructing the fields directly, and this change needs no code update on your side.ArrivalCursor
messages_by_arrival() and messages_by_arrival_in_range(). Separate from MessageCursor because the two feeds sort by different keys — a per-chat page orders by (timestamp_ms, seq), the session-wide feed by seq alone — so a cursor from one cannot page the other; a type that silently ignored the timestamp half would be worse than a compile error. Build one with ArrivalCursor::from(&stored_message).
Same non-durability caveat as MessageCursor.seq, and one more besides: seq is good for a live paging session and must not be persisted across restarts, and must never be compared against a remembered value as a watermark — a new message can legitimately land at or below a seq you’ve already seen. See Querying for why.
MessageKind
Template/Buttons/List/Interactive are WABA business content — a template notification, a message with quick-reply buttons, a list picker, or a flow/native-flow interactive message. Each has a *Reply/*Response counterpart for what a user’s tap materializes as. text is extracted per type mirroring WA Web’s parsers (e.g. a template’s hydrated_content_text, a list’s description, a response’s selected_display_text); footer text and button labels stay in the stored proto rather than text. A non-hydrated template still classifies as Template with text: None.
Undecryptable is a placeholder for a message that could not be decrypted yet — a retry or a PDO placeholder-resend may still fill it in, at which point the row is replaced with the recovered content under the same id. ViewOnce, Hosted, and Bot are also fanouts the server reported as unavailable, but permanently: a view-once photo/video/voice note is never shared with a companion device by design, and the same holds for hosted-content and bot-message fanouts. They previously all collapsed into Undecryptable, which gave a UI no way to distinguish “may still arrive” from “never will” — surfaced by a chat’s last_message_kind too, so a chat list can render the right chip without opening the thread.
Other(String) round-trips a forward-compatible label for message types added after this crate was released.
MessageStatus
WebMessageInfo.Status. Status transitions are enforced monotonic by the writer — a late Delivered receipt can never downgrade a row already at Read, and a server nack only fails a still-Pending row.
ReactionEntry / ReceiptEntry
ReceiptEntry rows for the same user_jid — one per state (Delivered, Read, Played) that peer’s receipts have reported. A state with no matching receipt has no row; states aren’t backfilled just because a later one arrived. Each row’s timestamp is the earliest instant that state was reported. This applies to both 1:1 and group chats; see Querying and Companion-device identities.
ContactEntry
.display_name() resolves the name to show, in the same precedence order as WA Web: full_name → first_name → push_name → business_name.
business_name is learned live from inbound messages: a business sender’s <verified_name> cert (decoded into MessageInfo.verified_name) is written to the contact row the same way push_name already is. A message without a cert never clobbers a name learned from an earlier one.MediaRef
StoreChange
Error handling
WriteBatchFailed carries the underlying error as text rather than the original typed error, since one failed batch fans out to every pending flush() caller and StoreError isn’t Clone.
Semantics worth knowing
- Monotonic status; receipt rows track a minimum timestamp.
StoredMessage.statusonly ever moves forward — a late-arrivingdeliveredcan’t downgrade aread. AReceiptEntryrow works differently: a state (Delivered/Read/Played) gets at most one row per participant. That row is created the first time the state is reported and is never deleted afterward. Itstimestampholds the earliest instant the state was reported so far — a later report of the same state can only lower that timestamp, never raise it, and can never remove the row. - Outgoing timestamps converge on the server clock. A positive message ack that carries a server timestamp replaces the optimistic local timestamp set by
record_outgoing, reordering the thread and chat list as needed — see Outgoing timestamp reconciliation. - Monotonic read state. Self-read state is tracked as a keyed cursor (watermark + boundary message id), so same-second siblings resolve deterministically and a stale replayed read/receipt can’t resurrect an unread badge. A no-op read still clears a manual-unread marker.
- Offline-drain reordering is handled. If a revoke or edit arrives before the message it targets (common when draining a backlog), the target is materialized as already-revoked/edited up front — the original content’s later arrival can’t resurrect revoked content or show pre-edit text, and never double-counts unread.
- History sync never clobbers live rows (
ON CONFLICT DO NOTHING). Live redeliveries and PDO recovery replace content in place instead — anUndecryptableplaceholder becomes the real message under the same id once it’s recovered. - Content refreshes are sender-scoped. Message ids are sender-chosen, so a different sender reusing an id can’t rewrite someone else’s message.
- A receipt for a message no chat holds is dropped, not parked. A receipt’s message id is chosen by the original sender and echoed back by the peer, not assigned by the server. When that id doesn’t match any stored row, the store can’t tell an unrecorded send apart from a message the user already deleted — both look identical: an addressed id with nothing behind it. The second reading is the common case, since a peer’s receipt costs a round trip and typically arrives well after the send it answers. A receipt is only ever recorded once the message it names is found under the addressed chat key or its PN/LID counterpart.
- Reaction removal is a tombstone, not a delete. Removing a reaction (an empty-emoji event, or
record_reactionwithemoji: "") keeps the row so a stale, older reaction arriving later — e.g. from a history chunk — can’t resurrect it.reactions()hides tombstoned rows. - PN/LID splits heal, they don’t recur. Once live traffic (or
reconcile_chat) merges a peer’s phone-number- and LID-keyed rows into one thread, later traffic under either identity keeps routing to that same thread — it can’t re-split. - Companion-device traffic never forks a thread. A device-suffixed identity is normalized to the bare peer before it reaches routing, so a linked device can’t materialize a chat, contact, or receipt row of its own — see Companion-device identities.
- The arrival feed tracks insertion, not mutation.
messages_by_arrival/messages_by_arrival_in_rangeorder byseq, assigned once at insert and left untouched by any later edit, revoke, star, or status change — so a row the feed has already walked past never resurfaces there no matter what happens to it afterward. Subscribe toStoreChange::Messagesfor that; the feed answers “what has arrived”, not “what has changed”.
See also
- Storage Traits -
SqliteStoreand the underlying backend traits - Custom Backends - Implementing your own storage backend
- Events - The event system
ChatStore’s handler listens to - Inbound Durability Hook - Combine with
skip_hook_committed_batchesto avoid double-materializing a hook-fed batch