Analyzing the VMP Behind a Hardened App's onCreate
About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.
This research is published for educational and defensive purposes. The target sample is anonymized throughout.
Analyzing the VMP Behind a Hardened App’s onCreate
Preface
After reproducing oacia’s article on the 360 Jiagu dex decryption flow, I wanted to keep going and look at how onCreate is implemented, continuing with the same hardened app oacia used.
Locating onCreate
onCreate is registered as a native function.
The Init function and the attachBaseContext function don’t yield any usable clues. But among the static functions there’s a JNI function, interface11.
Before attachBaseContext runs, all static {} blocks execute at class-load time (before the Application class’s constructor).
Using Yang’s script to intercept the JNI registration function RegisterNatives to find the function’s address:
[RegisterNatives] java_class: com.stub.StubApp name: interface11 sig: (I)V fnPtr: 0x73fd0bb9d8 module_name: libjiagu_64.so module_base: 0x73fcf82000 offset: 0x1399d8Looking straight at the assembly: the front part is thread detection and register operations; at the end it jumps to the value in x8, which can’t be directly decompiled.
Use Stalker to track changes to x8 and print the jumps:
function hookTargetFunc() {
var baseAddr = Module.findBaseAddress(TargetLibName)
//console.log(TargetLibName + " Module Base: " + baseAddr);
if (baseAddr == null) {
console.log('Please makesure ' + TargetLibName + ' is Loaded, by setting extractNativeLibs true.');
return;
}
// hook the given function address
Interceptor.attach(baseAddr.add(TargetFuncOffset), {
onEnter: function (args) {
console.log('\nCall ' + TargetFuncOffset.toString(16).toUpperCase() + ' In')
this.tid = Process.getCurrentThreadId();
var so_addr = Module.findBaseAddress(so_name);
var so_size = Process.getModuleByName(so_name).size;
Stalker.follow(this.tid, {
events: {
call: true, // CALL instructions: yes please
// Other events:
ret: false, // RET instructions
exec: false, // all instructions: not recommended as it's
// a lot of data
block: false, // block executed: coarse execution trace
compile: false // block compiled: useful for coverage
},
onReceive: function () {
},
transform: function (iterator) {
var instruction = iterator.next();
const startAddress = instruction.address;
if (startAddress){
do{
if (instruction.mnemonic.startsWith('br')||instruction.mnemonic.startsWith('blr')||instruction.mnemonic.startsWith('bl')) {
try {
iterator.putCallout(function (context) {
var pc = context.pc; // get the pc register, the current address
var lr = context.lr;
var x8 = context.x8; // get the x8 register
var module = Process.findModuleByAddress(pc);
if (module) {
try{
console.log(module.name + "!" +Memory.readCString(ptr(x8)));
}
catch (e){
}
console.log("x8 ====" + DebugSymbol.fromAddress(ptr(x8)));
}
});
} catch (e) {
console.log("error", e)
}
}
iterator.keep();
} while ((instruction = iterator.next()) !== null);
}
},
onCallSummary(summary) {
}
})
}, onLeave: function (retVal) {
console.log('Call ' + TargetFuncOffset.toString(16).toUpperCase() + ' Out\n')
Stalker.unfollow(this.tid)
}
})
}From the trace I found two onCreate sites. Let’s look at what the latter address does.
x8 ====0x7bfee7fe0c libjiagu_64.so!0x139e0c
x8 ====0x7bfee7fe64 libjiagu_64.so!0x139e64
x8 ====0x7bfee7fe7c libjiagu_64.so!0x139e7c// seems to be the key part
x8 ====0x7bfee80460 libjiagu_64.so!0x13a460// copy operation
x8 ====0x7bfee7fe8c libjiagu_64.so!0x139e8c
x8 ====0x7bfee7ff60 libjiagu_64.so!0x139f60 // enters the complex function sub_19DCA4
x8 ====0x7bfee7ffa0 libjiagu_64.so!0x139fa0// copy
x8 ====0x7bfee7ffc4 libjiagu_64.so!0x139fc4
x8 ====0x7bfee8010c libjiagu_64.so!0x13a10c
x8 ====0x7bfee80140 libjiagu_64.so!0x13a140
x8 ====0x7bfee80170 libjiagu_64.so!0x13a170
// onCreate
x8 ====0x7bfee80194 libjiagu_64.so!0x13a194
x8 ====0x7bfee801fc libjiagu_64.so!0x13a1fc
x8 ====0x7bfee804e4 libjiagu_64.so!0x13a4e4
x8 ====0x7bfee8024c libjiagu_64.so!0x13a24c
x8 ====0x7bfee80264 libjiagu_64.so!0x13a264// process detection, mutex_lock
// before this seems to be some stack operations, then registers change
x8 ====0x7bfee802e8 libjiagu_64.so!0x13a2e8//
x8 ====0x7bfee80328 libjiagu_64.so!0x13a328// memory
x8 ====0x7bfee7fe64 libjiagu_64.so!0x139e64
x8 ====0x7bfee80600 libjiagu_64.so!0x13a600
x8 ====0x7bfee8061c libjiagu_64.so!0x13a61c
// then the next onCreate appears
The latter onCreate is all memory and stack operations.
x8 ====0x7bfee807ac libjiagu_64.so!0x13a7ac// next jump
x8 ====0x7bfee807b8 libjiagu_64.so!0x13a7b8
x8 ====0x7bfee807f4 libjiagu_64.so!0x13a7f4
x8 ====0x7bfee80808 libjiagu_64.so!0x13a808
x8 ====0x7bfee80810 libjiagu_64.so!0x13a810
x8 ====0x7bfee8082c libjiagu_64.so!0x13a82c
x8 ====0x7bfee80854 libjiagu_64.so!0x13a854
x8 ====0x7bfee80854 libjiagu_64.so!0x13a854
x8 ====0x7bfee80838 libjiagu_64.so!0x13a838
x8 ====0x7bfee807e0 libjiagu_64.so!0x13a7e0
x8 ====0x7bfee807f4 libjiagu_64.so!0x13a7f4
x8 ====0x7bfee80870 libjiagu_64.so!0x13a870
x8 ====0x7bfee8087c libjiagu_64.so!0x13a87c
x8 ====0x7bfee80894 libjiagu_64.so!0x13a894
x8 ====0x7bfee80894 libjiagu_64.so!0x13a894
x8 ====0x7bfee80894 libjiagu_64.so!0x13a894
x8 ====0x7bfee80894 libjiagu_64.so!0x13a894
x8 ====0x7bfee808c4 libjiagu_64.so!0x13a8c4
x8 ====0x7bfee808dc libjiagu_64.so!0x13a8dc
x8 ====0x7bfee808e8 libjiagu_64.so!0x13a8e8After 0x13a170 the first onCreate appears, and blr x8 enters the key part.
Use jnitrace to look at the flow:
jnitrace -l libjiagu_64.so com.oacia.apk_protect
x8 ====0x7c166e43c0 libart.so!_ZN3art3JNI11GetMethodIDEP7_JNIEnvP7_jclassPKcS6_//JNIEnv->GetMethodID()
x8 ====0x7c164d6a1c libart.so!_ZN3art11ClassLinker15InitializeClassEPNS_6ThreadENS_6HandleINS_6mirror5ClassEEEbb
libart.so!<clinit> // runs during class loading
libart.so!<clinit>
libart.so!<init>
libart.so!<init>
bytesToHex
bytesToHex
libart.so!enc2// seems to be key data for process detection
libart.so!enc2
libart.so!onCreate
libart.so!onCreate
libart.so!Landroid/os/Bundle;
libart.so!Landroid/os/Bundle;Next, analyzing the key part: after 0x13a61c the second onCreate appears, again blr x8, entering a JNI function.
x8 ====0x7c1671e4d4 libart.so!_ZN3art3JNI15RegisterNativesEP7_JNIEnvP7_jclassPK15JNINativeMethodi//JNIEnv->RegisterNatives()
libart.so!enc2
libart.so!enc2
libart.so!onCreate
libart.so!onCreate
libart.so!Landroid/os/Bundle;
libart.so!Landroid/os/Bundle;
x8 ====0x7c169d0160 libart.so!_ZN3art7Runtime9instance_E
x8 ====0x7c169d0160 libart.so!_ZN3art7Runtime9instance_E
x8 ====0x7c16723374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv// JNIEnv->ExceptionCheck()So it’s confirmed: the interface11 method registers onCreate onto some native method.
Let’s hook _ZN3art3JNI15RegisterNativesEP7_JNIEnvP7_jclassPK15JNINativeMethodi to find the address:
function traceRegisterNatives() {
var RegisterNativesaddr = null;
var libartmodule = Process.getModuleByName("libart.so");
libartmodule.enumerateSymbols().forEach(function (symbol) {
if (symbol.name == "_ZN3art3JNI15RegisterNativesEP7_JNIEnvP7_jclassPK15JNINativeMethodi") {
RegisterNativesaddr = symbol.address;
}
})
console.log("RegisterNativesaddr->" + RegisterNativesaddr);
if (RegisterNativesaddr != null) {
Interceptor.attach(RegisterNativesaddr, {
onEnter: function (args) {
console.log("go into RegisterNativesaddr");
var methodsptr = ptr(args[2]);
var method_count = args[3];
var i = 0;
console.log("[" + Process.getCurrentThreadId() + "]RegisterNatives->" + method_count);
for (i = 0; i < method_count; i++) {
var name = methodsptr.add(i * Process.pointerSize * 3 + 0).readPointer().readUtf8String();
var signature = methodsptr.add(i * Process.pointerSize * 3 + 1 * Process.pointerSize).readPointer().readUtf8String();
var fnPtr = methodsptr.add(i * Process.pointerSize * 3 + 2 * Process.pointerSize).readPointer();
var find_module = Process.findModuleByAddress(fnPtr);
console.log("[" + Process.getCurrentThreadId() + "]RegisterNatives->name:" + name + ",sig:" + signature + ",addr:" + fnPtr);
}
if (name == "onCreate") {
console.log(hexdump(fnPtr));
var code = Instruction.parse(fnPtr);
var next_code = code.next;
console.log(code.address, ":", code);
for (var i = 0; i < 10; i++) {
var next_c = Instruction.parse(next_code);
console.log(next_c.address, ":", next_c);
next_code = next_c.next;
}
var onCreate_fun_ptr = next_c.next.add(0xf);
var onCreate_fun_addr = Memory.readPointer(onCreate_fun_ptr);
console.log("onCreate_function_address:",onCreate_fun_addr,"offset:",ptr(onCreate_fun_addr));
}
}, onLeave: function (retval) {
}
})
}
}
function main() {
var module = Module.findBaseAddress("libjiagu_64.so");
console.log("libjiagu_64.so base address: " + module);
traceRegisterNatives();
}
setImmediate(main);It outputs onCreate:
But it’s dynamically registered, so there’s no module name and the address can’t be printed directly. Let’s search for the bytecode in the SO directly. At 0x178c50 + 0xe7000 = 0x26FC50.
Find the onCreate address in IDA:
Locating the VMP interpreter
Cross-reference onCreate — it’s called by sub_1A1908.
Look at sub_1A1908’s cross-references.
It leads to sub_12DFE4, which is likely where the function binding happens.
Argument off_265F20 points to sub_137978. Enter it.
sub_137978 is the key operation. You can fix it up with Edit → Other → Specify switch idiom.
sub_137978’s logic: save the stack frame, allocate new space, then jump as before. Keep printing x8 changes and track the function:
Call 137978 In
x8 ====0x7164d7d568 libjiagu_64.so!0x139568
x8 ====0x7164d7b9c0 libjiagu_64.so!0x1379c0
x8 ====0x7164d7b9e0 libjiagu_64.so!0x1379e0
x8 ====0x7164d7ba04 libjiagu_64.so!0x137a04
x8 ====0x7164d7ba7c libjiagu_64.so!0x137a7c
x8 ====0x717c4e9ec4 libart.so!_ZN3art3JNI14PushLocalFrameEP7_JNIEnvi
x8 ====0x7164d7bbe8 libjiagu_64.so!0x137be8 // there's an XOR here
x8 ====0x7164d7cd58 libjiagu_64.so!0x138d58
x8 ====0x7164d7cd68 libjiagu_64.so!0x138d68
x8 ====0x7164d7cdf4 libjiagu_64.so!0x138df4
x8 ====0x7164d7cf04 libjiagu_64.so!0x138f04
x8 ====0x7164d7cf0c libjiagu_64.so!0x138f0c
x8 ====0x717c4eb8ec libart.so!_ZN3art3JNI11NewLocalRefEP7_JNIEnvP8_jobject
x8 ====0x7164d7cde8 libjiagu_64.so!0x138de8
x8 ====0x7164d7cdf4 libjiagu_64.so!0x138df4
x8 ====0x7164d7d70c libjiagu_64.so!0x13970c // enters sub_144EF0
x8 ====0x7164d7d70c libjiagu_64.so!0x13970c // enters sub_144EF0
x8 ====0x7164d88f4c libjiagu_64.so!0x144f4c // assigns "sj8uwp\}{}r" to a register
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x7164d89018 libjiagu_64.so!0x145018
x8 ====0x7164da3a54 libjiagu_64.so!0x15fa54 // register assignment
x8 ====0x717c4eaa68 libart.so!_ZN3art3JNI12NewGlobalRefEP7_JNIEnvP8_jobject
x8 ====0x7164d89024 libjiagu_64.so!0x145024
x8 ====0x7164d89040 libjiagu_64.so!0x145040
x8 ====0x7164d89058 libjiagu_64.so!0x145058
x8 ====0x7164d890b8 libjiagu_64.so!0x1450b8
x8 ====0x7164d890fc libjiagu_64.so!0x1450fc // there's EOR W8, W8, #31, stored at X0 + 8. x0's value goes to x28
x8 ====0x7164d89148 libjiagu_64.so!0x145148
x8 ====0x7164d9e630 libjiagu_64.so!0x15a630// jumps to sub_16CE0C; has junk instructions, seems important, calls sub_15FC34
x8 ====0x7164d89154 libjiagu_64.so!0x145154//CMP W0, #129
x8 ====0x7164d8916c libjiagu_64.so!0x14516c/CMP W0, #191
x8 ====0x7164d89184 libjiagu_64.so!0x145184//CMP W0, #224
x8 ====0x7164d89508 libjiagu_64.so!0x145508// enters the switch dispatcher, CMP W0, #206
x8 ====0x7164d89520 libjiagu_64.so!0x145520 //CMP W0, #215
x8 ====0x7164d8a274 libjiagu_64.so!0x146274//CMP W0, #210
x8 ====0x7164d8a28c libjiagu_64.so!0x14628c//CMP W0, #212
x8 ====0x7164d8cca4 libjiagu_64.so!0x148ca4 //CMP W0, #211 looks like binary search, doesn't it
x8 ====0x7164d8f3cc libjiagu_64.so!0x14b3cc // enters the sub_160D7C interpreter; the earlier x28 goes to x7, i.e. argument a8
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x717c4eaa68 libart.so!_ZN3art3JNI12NewGlobalRefEP7_JNIEnvP8_jobject
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x717c4ee3c0 libart.so!_ZN3art3JNI11GetMethodIDEP7_JNIEnvP7_jclassPKcS6_
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
x8 ====0x717c52d374 libart.so!_ZN3art3JNI14ExceptionCheckEP7_JNIEnv
libjiagu_64.so!p:}|q
libjiagu_64.so!p:}|q
libjiagu_64.so!p:}|q
libjiagu_64.so!p:}|q
x8 ====0x7164da9f8c libjiagu_64.so!0x165f8c
x8 ====0x7164da5df0 libjiagu_64.so!0x161df0
x8 ====0x717c5093dc libart.so!_ZN3art3JNI25CallNonvirtualVoidMethodAEP7_JNIEnvP8_jobjectP7_jclassP10_jmethodIDP6jvalue
// crashThat locates our interpreter: sub_160D7C.
Locating and decrypting the opcodes
Use IDA 8.3 for dynamic debugging.
The key part starts at 0x1450fc. Hooking from here hits rtld_db_dlactivity anti-debugging, and it leads us into the wrong branch.
The rtld_db_dlactivity function is an empty function by default; when a debugger is present it gets replaced with a breakpoint instruction. In Android anti-debugging, the SIGTRAP anti-debug can detect the debugged state by inspecting this function’s value.
We hook 0x15a630 directly and debug from there — that’s where we find the encrypted opcodes.
The pointer to its address is stored in the x0 register.
First decryption stage
sub_16CE0C is the first decryption function.
There are junk instructions; nop them out, redefine it as a function, and it XORs the data.
Step in and follow to the code after the junk instructions:
loc_16CF9C
ADR X8, loc_16D09C
SUB X8, X8, #236
LDR X30, [SP,#0xA0+var_A0]
MOV X30, X8
RETADR (Address of Label) is an ARM64-specific pseudo-instruction used to load a PC-relative address into a register.
Here it loads the address of loc_16D09C into X8, then subtracts 236 (0xEC) from it.
Then it loads the saved x30 from the stack — X30 is normally the return address register — and assigns X8 to X30, so the ret returns to the address we set.
Entering the real decryption. The logic:
Read the first byte of the encrypted opcode into w8, then read the eighth byte from where the encrypted-opcode address is stored into w13 (c5).
w9 holds the second byte of the encrypted opcode, then subtracts w12 (the eighth byte read from the encrypted-opcode address location, equal to w13), giving 7.
Assign, compare, and take the low eight bits of W9 (07); subtract, and if EQ (ZF=1, the earlier TST result is 0) W8 = W11 (0xE2), otherwise W8 = W10 (0xE7).
Then XOR gives w9=e5, keep the low 8 bits of W13, go to the left branch, XOR gives w10=6B.
Then the same operation as before leads to another part of the decryption: w8 restores to c5, w9 is inserted into the high bits of w10, then the low bits of w8 are copied to the high bits.
Then w8 XOR w10 gives the decrypted opcode 20ae.
Second decryption stage
Pass the just-decrypted opcode and the encrypted opcode into sub_15FC34 — this should be the next decryption. Look at the key part:
x8 is set to the main dex file’s address; w9 is the value at encrypted-opcode-address + 18 (A0AF), then it loads w10 w8 from the main dex file.
W11 = W9 / W10 (integer division)
W9 = W9 - (W11 * W10) (remainder)
W9 = W9 << 8 (left shift 8)Then the low 8 bits of X9 are replaced with the low 8 bits of X19 to get the final value 23AE, the offset.
Finally, looking it up gives the value D2, assigned to w0.
Then it binary-searches based on w0’s value. This value changes — if you start debugging from the earlier function, you get the wrong value; the correct lookup lands at the interpreter.
The interpreter’s arguments are as follows. Focus on a5 a6 (encrypted opcode) and a7 (already-decrypted opcode); a4 here is the decrypted main dex file address we got earlier.
x0 MOV W0, #3 ; n4
1 MOV W1, WZR ; a2
2 MOV X2, X20 ; JNIEnv
3 MOV X3, X21 ; a4
4 MOV X4, X19 ; a5
5 MOV X5, X22 ; a6
6 MOV W6, W25 ; a7
7 MOV X7, X28 ; a8
X0 0000000000000003
X1 0000000000000000
X2 0000007FC4CEE0A0 -> 0000007971A27800 -> 000000797C933118 (libart.so ! _ZN3art3JNI14NewObjectArrayEP7_JNIEnviP7_jclassP8_jobject)
X3 000000797CC43680 -> 0000007962CE6000 (debug003) -> ("dex\n038")
X4 0000007971A26600 ([anon:libc_malloc]) -> 0000000000000000
X5 000000797CD00380 -> 00000079632ACEE0 (debug003) -> EA743718C721CC4E
X6 00000000000020AE
*X7 000000797CCE6C40 -> 0000007FC4CEE0A0 -> 0000007971A27800 -> [...] (libart.so ! _ZN3art3JNI14NewObjectArrayEP7_JNIEnviP7_jclassP8_jobject)Third decryption stage
sub_16CB4C has the same junk instructions; remove them. The function is the same pattern as sub_16CE0C; start the analysis from the last piece of real logic.
X8 = X20 << 1 (2), read the byte at X19+8 into W13 (c5), read 1 byte from X26 + X8 into W8 (the third encrypted opcode, 21).
W9 = W9 - W12 (c7-c5=2); assign; test whether bit 5 of W9 is 1; W12 = W9 & 0xFF; W13 = W8 - W13; W8 = (Z == 1) ? W11 : W10.
W9 = W8 XOR W12; W8 = W13 & 0xFF; W10 = W8 XOR W11 (8E).
Then it’s roughly the same logic as before; finally returns 0x154b = 5451.
And 5451 corresponds to the method_id of onCreate.
Fourth opcode decryption
sub_16C918 also has an XOR. Analyze the key part:
w8 reads the fifth and sixth encrypted opcode (3718); w9 takes its high bits.
Here w12 is still the earlier c5 — all four decryptions use the same key.
W9 = W9 - W12; W12 = W8 - W12; W8 = W9 & 0xFF; W9 = (Z == 1) ? W11 : W10; W8 = W9 XOR W8.
W9 = W12 & 0xFF; the logic after is basically the same as before; finally returns 0x0021.
Wrapping up
Above we got the final decrypted opcodes; reverse them:
ae20 (d2) 4b152100 — here d2 is the key for the binary search.
Compared against the source APK’s bytecode, only the operands differ. The rest follows the same pattern.
Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).
Get in touch