About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.

This research is published for educational and defensive purposes. The target sample is anonymized throughout.

---

Reimplementing WhatsApp End-to-End Encryption: Signal Protocol, Curve25519, and the Double Ratchet

I spent a stretch reverse-engineering WhatsApp’s private-messaging path — both the Android and desktop clients — and reimplemented the send side of its end-to-end encryption. This is the record of that process, kept for technical reference.

Environment

• WhatsApp v2.22
• IDA 7.5, Frida 14.2.2, Gda 3.86, JEB, jadx-gui, unidbg
• LineageOS 17.1 (Android 10) on a Xiaomi Mi 8

Background

WhatsApp’s messaging is end-to-end encrypted: only the two parties can decrypt a message, and the server merely relays ciphertext. I started by reading the encryption overview whitepaper (the Signal protocol) and a few secondary writeups. Honest admission: at first the key concepts — identity keys, pre-keys, the ratchet — just spun my head. They only made sense once I was inside the app, matching each concept to a concrete function call. The whitepaper is correct; it’s just not an operational walkthrough.

Transport and the key modules

First, pin down the transport. With Wireshark plus hooking and breakpoints, I confirmed it’s plain TCP:

The peer resolves to a Facebook/WhatsApp endpoint (157.240.199.61 region Hong Kong). From there, the JNI function names surface the modules that matter — libwhatsapp.so and, of course, libcurve25519.so.

Curve25519 is the Diffie-Hellman function at the heart of this: an elliptic-curve scheme offering 128-bit security, designed for ECDH key agreement, one of the fastest ECC curves, and unencumbered by known patents.

libcurve25519.so boolean org.whispersystems.curve25519.NativeCurve25519Provider.smokeCheck(int) 0x9d782548 func: 0x78b8406288 0x0  iOffset: 4288
libcurve25519.so byte[] org.whispersystems.curve25519.NativeCurve25519Provider.generatePrivateKey(byte[]) 0x9d782638 func: 0x78b8405a2c 0x0  iOffset: 3a2c
libcurve25519.so byte[] org.whispersystems.curve25519.NativeCurve25519Provider.calculateAgreement(byte[], byte[]) 0x9d7825c0 func: 0x78b8405b68 0x0  iOffset: 3b68

Generating a key:

retval: [object Object]
java.lang.Exception
    at org.whispersystems.curve25519.NativeCurve25519Provider.generatePublicKey(Native Method)
    at org.whispersystems.curve25519.OpportunisticCurve25519Provider.generatePublicKey(:750206)

And the first plaintext, "11", in protobuf form:

[MI 10::com.whatsapp]-> byteArray,byte src : [10,2,49,49]        protobuf
byteArray,md5str:
11
java.lang.Exception
    at X.1FH.A02(Native Method)
    at com.whatsapp.jobqueue.job.SendE2EMessageJob.writeObject(:271863)
    at java.lang.reflect.Method.invoke(Native Method)

Following the flow shows the usual crypto suspects — java.security.MessageDigest, javax.crypto.Mac, javax.crypto.Cipher — and hooking them reveals the data transform at each stage.

The two-layer encryption

There are two distinct layers:

1. Transport: AES-256-GCM between client and server, with a fresh IV per packet.
2. Content: AES-256-CBC for the message body itself. This key is derived from a Diffie-Hellman against the recipient’s public key, so only the recipient can decrypt it — the server never sees the plaintext. Each message gets its own content key.

Every app launch opens a fresh TCP connection (once you’re past the SMS-code login). That handshake initializes a key pair, and the public key goes out in the very first packet to the server. Several other keys are involved — your own identity_key, a session token, the server’s public key — combined via calculateAgreement and HKDF to produce the intermediate material and keys that encrypt everything downstream.

That first packet also carries device environment info:

Once the server validates it, the connection is up and private messages can flow.

The Message Key and the ratchet

The content layer uses a Message Key — an 80-byte value:

• 32 bytes → the AES-256 key
• 32 bytes → the HMAC-SHA256 key
• 16 bytes → the IV

The HMAC-SHA256 key computes a MAC over the AES ciphertext, of which only the first 8 bytes are kept. And the Message Key itself advances through a ratchet: after each message, a SHA-256 combination derives the next message’s key.

Messaging also has a session concept. To talk end-to-end, both sides first establish a session — agreeing on keys — and then save the context. From that point, you derive each message’s Message Key from the message sequence number alone. The session survives app restarts, so I started the analysis on an already-established session: hook the current Message Key, derive the one for the target sequence number, encrypt, send.

Establishing a session

Then I implemented session setup from scratch, following the whitepaper:

The initiator requests the recipient’s public Identity Key, Signed Pre Key, and a One-Time Pre Key. The server returns the requested public keys; the One-Time Pre Key is single-use and deleted after the request. The initiator stores the recipient’s Identity Key as Irecipient, the Signed Pre Key as Srecipient, and the One-Time Pre Key as Orecipient. The initiator generates an ephemeral Curve25519 key pair Einitiator and loads its own Identity Key as Iinitiator. The initiator computes master_secret = ECDH(Iinitiator, Srecipient) || ECDH(Einitiator, Irecipient) || ECDH(Einitiator, Srecipient) || ECDH(Einitiator, Orecipient) — omitting the final ECDH if no One-Time Pre Key was available — then uses HKDF to derive a Root Key and Chain Keys.

The algorithms are simple enough; the hard part is the key-transition flow. Session establishment also needs two key pairs up front — OurBaseKey and ourRatchetKey:

I implemented all of it and tested: the recipient received my message but couldn’t read the content.

The One-Time Pre Key that wouldn’t line up

Some guides said a sender switching devices can produce this, but that wasn’t my case. I re-verified the crypto repeatedly — my encryption matched the hooked parameters and results exactly. Exhausted, I stepped back and thought about how E2E must work, and found the gap: the One-Time Pre Key.

The recipient uploads a batch of One-Time Pre Keys to the server, each used once and deleted. If my session didn’t establish correctly, the server wouldn’t sync it back to the recipient. And the recipient needs to know which of their pre-keys I used. That implies an identifier. I’d already noticed a few uncertain fields in the session-establishment packet, but I couldn’t find a value common to both the returned recipient keys and my sent data — because, it turned out, the formats differed.

Stuck, I opened a second front: the desktop client.

The desktop side-quest

WhatsApp on Windows runs seven processes. I again hunted for the send/recv boundary and found the process handling network I/O (its command line gives it away):

The desktop client sends over WebSocket. I pulled the SSL source and annotated the functions:

Found the data send/recv points:

But the received data had no further logic applied — it sometimes just cleared memory — so I suspected it was forwarded to another process. The references to MojoMessages confirmed it: Mojom is Chromium’s cross-platform IPC framework. Not my focus, but it meant this process only handled the network layer; the messaging logic lived elsewhere:

From there I traced the encryption flow, and once it was clear I could hook the desktop client directly (frida or an injected DLL):

Cross-checking the data confirmed it: there’s an ID marking the shared pre-key.

With that fixed, the recipient could finally read my message:

Template messages work through the same path:

(Side note: the first-time SMS-code login runs over HTTP, with its own key-pair generation feeding the encryption.)

What this exercised

1. End-to-end messaging end to end — the transport layer, the session handshake, the double ratchet, and the Message Key derivation.
2. Curve25519 in practice — where ECDH and HKDF actually fit in a real protocol.

The thing worth internalizing for anyone doing this kind of work: the crypto primitives are never the hard part. It’s the key lifecycle — which key derives from which, which identifier ties a session to a pre-key, what survives a restart. Get the plumbing right and the encryption is mechanical; miss one identifier and the recipient sees garbage forever.

---

Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).

Get in touch