Colin XiaoReverse Engineering & Security Research

Key Techniques for Dynamic .NET Unpacking

About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.

This research is published for educational and defensive purposes. The target sample is anonymized throughout.

Key Techniques for Dynamic .NET Unpacking

The essential problem and the essential strategy of .NET unpacking, in one place.

The core of .NET packing

A .NET executable stores IL bytecode as its primary content. When a .NET binary is packed, that IL bytecode is hidden or shielded. Unpacking is therefore about one thing above all: recovering the IL bytecode, which gets you close to the original source.

What the JIT is actually for

A .NET program typically resolves some or all of its IL at JIT time. If the packer doesn’t encrypt the tokens, complete IL shows up at JIT time. That’s the window.

The JIT flow

The rough call chain of the .NET JIT:

clr!ThePreStub
clr!PreStubWorker
clr!MethodDesc::DoPrestub
clr!MethodDesc::MakeJitWorker
clr!UnsafeJitFunction
clr!CallCompileMethodWithSEHWrapper
clr!invokeCompileMethod
clr!invokeCompileMethodHelper
clrjit!CILJit::compileMethod

At runtime, when a method actually executes, the CLR generates a stub for each method, then chains through a pile of JIT functions to produce machine code. Two JIT functions matter here: DoPrestub and compileMethod, with compileMethod the lower-level of the two.

The key to recovering IL through the JIT

Where does the IL show up?

For an unpacked .NET binary, the IL is available right in DoPrestub — one of its parameters is the MethodDesc (method descriptor), and from that you can get the IL.

For a packed .NET binary, the MethodDesc you get inside DoPrestub points at IL that’s either encrypted or wrong. A shell generally hijacks some JIT function, calls its own decrypt/recover routine to obtain the real IL, then hands it back to the hijacked JIT function to generate machine code and execute the method.

Shells almost never reimplement the lower-level work themselves — turning every IL instruction into machine code, plus the optimization work — because that’s a huge amount of effort, rarely beats the original JIT, and can introduce runtime errors.

Strategy for getting the IL

One strategy is to enumerate every method, grab its MethodDesc, and manually trigger DoPrestub, which in turn triggers the shell’s internal IL decrypt/recover routine. But there are five problems:

  1. Once a method triggers DoPreStub, it executes immediately.
  2. Once a method triggers DoPreStub, it won’t trigger again — so what if the shell needs multiple triggers to recover the IL?
  3. UnboxingStub methods are not JIT-compiled directly.
  4. Inline methods get inlined into their callers.
  5. When a .NET method executes, the CLR runs checks — the preconditions that must hold for the method to run:
    • Has the class’s static constructor (type initializer) run?
    • Has the generic class been resolved?

How do you actually get the IL?

The question, restated: after packing, how do you trigger only the shell’s IL decrypt/recover routine without letting the actual method execute?

Wrap the JIT call into a CallJit helper (see wwh1004)

public static void CallJit(void* methodHandle) {

    Logger.Instance.LogInfo("即将执行：Reset");
    RuntimeFunctions.Reset(methodHandle);

    Logger.Instance.LogInfo("即将执行：DoPrestub");
    RuntimeFunctions.DoPrestub(methodHandle);
    if (RuntimeFunctions.IsUnboxingStub(methodHandle)) {
        Logger.Instance.LogInfo("IsUnboxingStub：True");
        // CLR内部存在UnboxingStub的方法不会被JIT直接编译，所以需要对UnboxingStub进行编译
        void* pUnboxingStub;
        Logger.Instance.LogInfo("即将执行：GetWrappedMethodDesc");
        pUnboxingStub = RuntimeFunctions.GetWrappedMethodDesc(methodHandle);

        Logger.Instance.LogInfo("即将执行：Reset");
        RuntimeFunctions.Reset(pUnboxingStub);

        Logger.Instance.LogInfo("即将执行：DoPrestub");
        RuntimeFunctions.DoPrestub(pUnboxingStub);

        Logger.Instance.LogInfo("即将执行：Reset");
        RuntimeFunctions.Reset(pUnboxingStub);
    }

    Logger.Instance.LogInfo("即将执行：Reset");
    RuntimeFunctions.Reset(methodHandle);

Patch the relevant JIT functions

  • Fix the inline-method handling. Follow wwh1004’s approach, but note the x86 vs x64 differences.
  • Disable the static-constructor check inside DoPreStub.

The static-constructor check isn’t exposed directly in the DoPreStub source, but it looks like this:

//DoPreStub 源代码里没有直接给出
// 检查该方法所属的类型是否需要运行静态构造函数
    if (!pMD->CheckRunClassInitThrowing()) {
        // 初始化失败（如抛出异常）
        return;
    }

It’s likely inlined — the sub-function’s body has been pulled into the caller. The same applies to calling MakeJitWorker.

//伪代码
 if ( ((*(_DWORD *)(v89 + 0x38) & 0x100000) == 0// if (IsClassConstructorTriggeredViaPrestub())
             || (unsigned int)CORProfilerDisableOptimizations()
             || (*((_DWORD *)MethodTable::GetModule(v88) + 8) & 0x800) == 0
             && ((g_CORDebuggerControlFlags & 8) == 0 || (*((_DWORD *)MethodTable::GetModule(v88) + 8) & 0x400) != 0))
            && !*(_DWORD *)(*((_QWORD *)MethodTable::GetModule(v88) + 6) + 0x114i64) )
          {//【Patch，直接nop掉，不执行该调用】
            MethodTable::CheckRunClassInitThrowing(v6);// 
// 触发静态构造器的执行
// pMD->GetMethodTable()->CheckRunClassInitThrowing();
          }
  • Disable the generics check inside DoPreStub.

Original source:

//源代码
if (ContainsGenericVariables())
    {
        COMPlusThrow(kInvalidOperationException, IDS_EE_CODEEXECUTION_CONTAINSGENERICVAR);
    }

Pseudo-code, with five patch points:

//伪代码：需Patch的代码有5处。
  if ( ((unsigned __int8)v13 & 2) != 0 )        // if (ContainsGenericVariables())函数的内联？
  {
    if ( (unsigned int)TypeHandle::IsGenericVariable((TypeHandle *)&v167) )
      goto LABEL_271;//【Patch，不跳转】
    if ( (unsigned int)TypeHandle::HasTypeParam((TypeHandle *)&v167) )
    {
      TypeParam = (MethodTable **)TypeHandle::GetTypeParam(&v167, v186);
      if ( (unsigned int)TypeHandle::ContainsGenericVariables(TypeParam, 0) )
        goto LABEL_271;//【Patch，不跳转】
    }
  }
  else if ( v13 && *(int *)v13 >= 0 && (*(_DWORD *)v13 & 0x30) != 0 && MethodTable::ContainsGenericVariables(v13, 0) )
  {
    goto LABEL_271;//【Patch，不跳转】
  }
  if ( g_IBCLogger )
    IBCLogger::LogMethodDescAccessStatic(this);
  v14 = *((_WORD *)this + 3);
  if ( (v14 & 7) == 5 && (*((_BYTE *)this + 0x18) & 7) == 0 )
  {//【Patch，不跳转】
LABEL_271:
    v157 = 0i64;
    v156 = 0i64;
    v155 = 0i64;
    v154 = 0i64;
    RealCOMPlusThrow(0x21i64, 0x1ABBi64, 0i64); //Patch目的是不让其执行到此处。  COMPlusThrow(kInvalidOperationException, IDS_EE_CODEEXECUTION_CONTAINSGENERICVAR);
  }
  if ( g_IBCLogger )
  {
    IBCLogger::LogMethodDescAccessStatic(this);
    v14 = *((_WORD *)this + 3);
  }
  if ( (v14 & 7) == 5 && *((_WORD *)this + 0xD) )
  {
    MethodDesc::GetMethodInstantiation(this, &v180);
    v103 = 0;
    v104 = v181;
    if ( v181 )
    {
      v105 = v180;
      do
      {
        v106 = *(MethodTable **)(v105 + 8i64 * v103);
        if ( ((unsigned __int8)v106 & 1) != 0 )
          v106 = *(MethodTable **)((char *)v106 - 1);
        v177 = v106;
        if ( (unsigned int)TypeHandle::ContainsGenericVariables(&v177, 0) )
          goto LABEL_271;//【Patch，不跳转】
      }
      while ( ++v103 < v104 );
    }
    pThread = v204;
  }                                             // ContainsGenericVariables()判断结束

That’s the shape of it: enumerate the methods, wrap the JIT trigger so only the shell’s recover routine runs, and patch out the static-constructor and generics checks so the method never actually executes. Recover the IL, and the packed .NET binary is effectively unpacked.

Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).

Get in touch