Building an Open-Source ARM64 VMP from Scratch: Virtualizing All 121 A64 Base Instructions
About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.
This project is open source and shared for educational and research purposes.
I’ve been building VMPacker, an open-source ARM64 Linux ELF code-protection system — a full instruction-level VM that protects binaries the way commercial VMP tooling does, except here you can read every line of it.
Commercial VM protection easily runs into five figures. VMPacker is fully open and built for anyone who wants to understand how VMP protection actually works under the hood.
The 2.0 milestone: all 121 ARM64 instructions
After a couple of days of deep porting work, version 2.0 now theoretically covers the entire ARM64 A64 base instruction set — all 121 instructions:
| Category | Instructions |
|---|---|
| Arithmetic / logic | ADD, SUB, MUL, AND, ORR, EOR, LSL, LSR, ASR, MVN, BIC, ORN, EON… |
| Multiply-extended | MADD, MSUB, SMADDL, SMSUBL, UMADDL, UMSUBL, SMULH, UMULH, UDIV, SDIV |
| Data movement | MOV, MOVZ, MOVK, MOVN |
| Memory access | LDR, STR, LDP, STP, LDPSW, LDADD, CAS, LDAR, STLR, LDAXR, STLXR (multiple widths + addressing modes) |
| Branch control | B, BL, BR, BLR, RET, B.cond, CBZ/CBNZ, TBZ/TBNZ |
| Conditional select | CSEL, CSINC, CSINV, CSNEG, CCMP, CCMN |
| Bitfield / bit ops | UBFM, SBFM, BFM, EXTR, CLZ, CLS, RBIT, REV, REV16, REV32 |
| Carry arithmetic | ADC, ADCS, SBC, SBCS |
| SIMD | LD1, ST1 |
| System / barriers | SVC, MRS, MSR, ADRP/ADR, DMB, DSB, ISB, HLT, BRK, PRFM |
In practice this means most ARM64 functions produced by a C/C++ compiler can be protected directly, without the toolchain dying on an “unsupported instruction.”
Why I built it
My first version only used a modified UPX packer, and the protection was weak — the binaries kept getting cracked, and I was stuck.
When I went looking for an open-source ARM64 VM-based protector, there was almost nothing usable. The commercial options were priced out of reach. So I built one from scratch: VMPacker, aimed at raising the anti-reversing and anti-cracking bar from the root rather than papering over it with a packer.
Architecture
The system is built around three core modules:
1. Instruction decoder (Go)
- Table-driven pattern matching based on the Arm Architecture Reference Manual.
- Supports the DP-IMM / DP-REG / Branch / Load-Store instruction families.
- Emits a unified intermediate representation (IR).
2. Bytecode translator (Go)
- Translates the ARM64 IR into 63 custom VM instructions.
- Handles PC-relative relocation (ADRP/ADR → absolute address computation).
- Fixes branch target addresses and resolves label references.
3. VM interpreter stub (C → flat binary)
- Compiled to position-independent code (PIC) as a flat binary.
- Injected into the ELF through a PT_NOTE segment.
- Builds the indirect dispatch table on the stack at runtime.
- CRC32 integrity check.
Five layers of protection
| Layer | Technique | Effect |
|---|---|---|
| Randomized opcode mapping | A different instruction encoding per build | Reversers can’t recognize semantics at a glance |
| OpcodeCryptor | enc[pc] = op[pc] ^ (key ^ (pc * 0x9E3779B9)) |
Makes static analysis extremely difficult |
| Reversed bytecode | Instructions stored in reverse, interpreter walks backwards | Increases analysis complexity |
| Token entry | A 3-instruction trampoline replaces the original function | Hides the real bytecode location |
| Indirect dispatch | Function-pointer table filled on the stack at runtime | Breaks IDA cross-reference analysis |
Extensibility
The project is interface-driven — Decoder, Translator, and Packer are fully decoupled — so in theory it can be extended to:
- Other ISAs: x86_64, RISC-V
- Other binary formats: PE (Windows), Mach-O (macOS)
Project
GitHub: https://github.com/LeoChen-CoreMind/VMPacker
Licensed AGPL-3.0 — free to use for learning and research.
Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).
Get in touch