Colin XiaoReverse Engineering & Security Research

A Windows 7 Loader Bug Caused by Static DLL Dependency Order - Debugging a Packed Application Crash

About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.

This research is published for educational and defensive purposes. The target sample is anonymized throughout.

Introduction

In software protection and license-verification work, wrapping a target program in an encrypted shell is common. But the shell can introduce OS-specific compatibility problems. This is a real case: a Windows application crashed at startup on Windows 7 after the license-verification shell was added, while the un-wrapped version ran fine, and the problem couldn’t be reproduced on Windows 10 or later. Step-by-step debugging traced it to a Windows 7 loader flaw in how static DLL dependencies are ordered — the entry point of the system DLL setupapi.dll was skipped, which made a later API call fail. This post reconstructs the investigation, the debugging techniques, and the final fix.

The symptom

  • OS: Windows 7 SP1 (reproduced on both 32-bit and 64-bit).
  • Wrapped build: the executable crashed immediately at startup with a Windows Error Reporting (APPCRASH) dialog, faulting module ntdll.dll, exception code 0xc0000005 (access violation).
  • Control build: the un-wrapped program ran fine in the same environment.
  • Environment difference: the same wrapped build ran fine on Windows 10 and Windows 11.

Initial investigation

Reproducing and the debugging dead-end

The crash reproduced reliably in a Windows 7 VM. Attaching Visual Studio to the process gave a broken stack: the faulting instruction address didn’t belong to any loaded module. The reason is that the license-verification module was anonymously loaded — the shell used VirtualAlloc to allocate memory, wrote machine code into it, and patched the entry point, never calling LoadLibrary or any other standard API to register the module. Its code segment therefore has no backing image file in the kernel, so the debugger can’t map it to module symbols, the stack shows “unknown address”, and the call chain can’t be traced.

Narrowing the scope

The client provided more detail: the crash only happened when specific user DLLs (ModuleA.dll, ModuleB.dll) were encrypted, and every potentially interfering extra feature (compression, anti-debug) had already been disabled to rule them out. Those DLLs are exactly the modules the license-verification logic depends on, pointing at the license code as the trigger.

Getting a valid call stack

To get a clean call chain, we temporarily changed the shell to load the license module via LoadLibrary instead of anonymous loading. The change only alters how the module is loaded — the execution path and crash behavior were unchanged — so it’s a valid way to recover symbol information. With symbols resolved, we got the full crash stack:

ntdll.dll!RtlAllocateHeap
setupapi.dll!AllocateDeviceInfoSet
setupapi.dll!SetupDiCreateDeviceInfoListExW
setupapi.dll!SetupDiGetClassDevsExW
setupapi.dll!SetupDiGetClassDevsA
授权模块!授权函数
...

Note the crash is inside RtlAllocateHeap, with its first argument HeapHandle equal to NULL — the caller passed an invalid heap handle to the allocator.

Root cause

Locating the suspicious global

Reverse engineering setupapi.dll’s AllocateDeviceInfoSet showed it calls RtlAllocateHeap, and the heap handle it uses comes from a global in the DLL’s data segment (g_hSetupAPIHeap below), located in .data with an initial value of 0.

A data breakpoint reveals the missing initialization

To find when g_hSetupAPIHeap gets written, we set a hardware write breakpoint on it in x64dbg and ran both the control and the wrapped builds:

  • Control build: the breakpoint hit, with this write stack:
setupapi.dll!ProcessAttach()  - 0x1171 bytes
setupapi.dll!DllMain()  + 0x1616 bytes
setupapi.dll!_CRT_INIT()  - 0x42 bytes
ntdll.dll!LdrpRunInitializeRoutines()  + 0x1fe bytes
ntdll.dll!LdrpLoadDll()  + 0x594 bytes
ntdll.dll!LdrLoadDll()  + 0xed bytes
KernelBase.dll!LoadLibraryExW()  + 0xea bytes
stub.exe!start(void * hModule=0x0000000000000000, unsigned long ul_reason_for_call=0x00000000, void * lpReserved=0x0000000000000000)  Line 20  C++
kernel32.dll!BaseThreadInitThunk()  + 0xd bytes
ntdll.dll!RtlUserThreadStart()  + 0x1d bytes

The variable is initialized in setupapi.dll’s DllMain (the ProcessAttach branch), and DllMain is invoked by the system loader via LdrpRunInitializeRoutines.

  • Wrapped build: the same breakpoint never hit. The program ran straight into RtlAllocateHeap(NULL, ...) and crashed, with g_hSetupAPIHeap staying 0 the whole time. So setupapi.dll’s initialization code was never called at all.

An accidental breakthrough

Stuck, we tried a few unconventional things — one was manually reordering the PE import table to move system DLLs (like SETUPAPI.dll) ahead of user DLLs. That change made the Windows 7 crash go away.

That pointed at the EXE’s static DLL dependency order in the import table. In the crashing build, the user DLLs (ModuleA.dll) appeared before the system DLL (SETUPAPI.dll); reordering them fixed it.

Analysis: the Windows 7 loader’s dependency-order flaw

When the Windows loader creates a process, it parses the main module’s import table and loads the required DLLs in import order. After all DLLs are loaded, the loader calls each DLL’s entry point (DllMain) via LdrpRunInitializeRoutines, in load order. Ideally each DLL’s DllMain runs after the DLLs it depends on are initialized.

But Windows 7’s loader has a flaw in handling static dependency order: if a system DLL appears later in the import table than some user DLL, and that user DLL’s DllMain directly or indirectly calls a function in the system DLL, the system DLL’s entry point may not have run yet and its globals are still uninitialized. As Windows Internals (7th ed., ch. 3) describes, when import order creates complex dependencies, the loader can omit some DLLs’ entry points from the initialization list when building it, so they never run. That’s what happened to setupapi.dll here.

In this case the license module lives in a user DLL, whose DllMain calls setupapi.dll!SetupDiGetClassDevsA, which needs g_hSetupAPIHeap. Since setupapi.dll’s DllMain never ran, that variable stayed 0 and the call died in RtlAllocateHeap(NULL, ...).

The fix: reorder the import table

Principle

Moving all system DLLs ahead of user DLLs in the EXE’s import table ensures system DLLs load and initialize first, avoiding the dependency-order flaw. This only changes the order of import entries — no code logic changes — so it shouldn’t introduce new problems.

Tooling

I wrote a small tool, reorder_imports, to reorder a PE’s import table automatically. Core steps:

  1. Parse the PE: read the input, validate the DOS and NT headers, locate the import table’s RVA in the data directory.
  2. Find the import section: find the section containing the import table via the section table, and compute its file offset (the tool assumes the import table sits in a single section, which holds for the vast majority of PEs).
  3. Walk the import descriptors: for each imported DLL name, decide whether it’s a system DLL. The check: append the DLL name to GetSystemDirectory and try LoadLibrary; success means system DLL — reliable as long as the system directory hasn’t been tampered with.
  4. Stable reorder: std::stable_partition the import descriptor array so all system DLL descriptors move to the front, and user DLLs keep their relative order.
  5. Write back: write the modified buffer to the output file.

Source is in Appendix A. Compile under a Visual Studio developer prompt or any C++11 compiler, then run reorder_imports <input.exe> <output.exe>.

Verification

The client ran the wrapped EXE through the tool, tested on Windows 7, and it started normally with no crash. Windows 10 and 11 also passed with full functionality and no side effects.

Why newer Windows doesn’t have this bug

It can’t be reproduced on Windows 8, 10, or 11, which matches Microsoft’s later loader improvements. From Windows Internals and related docs:

  • Loader init-order optimization: from Windows 8, the loader topologically sorts DLL dependencies so every DLL’s DllMain runs after all its dependencies are initialized.
  • API Sets redirection finalized: the API Set mechanism was solidified after Windows 8; the system resolves real implementations from the system directory first, preventing same-named DLLs in the app directory from interfering.
  • KnownDLLs hardening: core system DLLs are better protected, always loaded from the system directory, with guaranteed initialization order.
  • Loader lock management: DllMain timing is managed more strictly, reducing order-related entry-point omissions.

Together these eliminate the Windows 7 static dependency-order flaw.

Takeaways

  • Debugging technique: for crashes caused by uninitialized variables, set a data write breakpoint on the suspect global and compare against the control build — it quickly pinpoints where initialization is missing. Data breakpoints are a first-choice tool for uninitialized-variable bugs.
  • The Windows 7 loader flaw: Windows 7’s loader can skip a system DLL’s DllMain under certain static dependency orderings. Verify the final import order, and post-process it if needed.
  • The value of accidents: in hard investigations, a chance experiment (like reordering the import table) can break things open — but back it with theory to confirm it’s sound.
  • Cross-version testing matters: loader behavior differs subtly across OS versions.
  • Build vs. post-processing: controlling import order at the linker level is unreliable on modern Visual Studio (2015+); use a post-processing tool for precise adjustment.

Appendix A: import-reordering tool source

// reorder_imports.cpp
// 编译：cl /EHsc reorder_imports.cpp
#include <Windows.h>
#include <algorithm>
#include <fstream>
#include <iostream>
#include <iterator>
#include <vector>

// 判断 DLL 是否为系统 DLL：尝试从系统目录加载
bool is_system_dll(const char* name) {
    char sysPath[MAX_PATH];
    GetSystemDirectoryA(sysPath, MAX_PATH);
    strcat_s(sysPath, "\\");
    strcat_s(sysPath, name);
    HMODULE handle = LoadLibraryA(sysPath);
    if (handle) {
        FreeLibrary(handle);
        return true;
    }
    return false;
}

int parse_import(const IMAGE_SECTION_HEADER& section, char* base, std::size_t vaddr) {
    auto pred = [&](const IMAGE_IMPORT_DESCRIPTOR& desc) {
        return is_system_dll(&base[desc.Name - section.VirtualAddress]);
    };
    auto first = reinterpret_cast<IMAGE_IMPORT_DESCRIPTOR*>(&base[vaddr - section.VirtualAddress]);
    auto last = first;
    for (auto iter = first; iter->Name; ++iter, ++last)
        std::cout << "name: " << &base[iter->Name - section.VirtualAddress] << '\n';
    std::stable_partition(first, last, pred);
    std::cout << "\nAfter reordering:\n";
    for (auto iter = first; iter->Name; ++iter)
        std::cout << "name: " << &base[iter->Name - section.VirtualAddress] << '\n';
    return 0;
}

int main(int argc, char* argv[]) try {
    if (argc < 3) {
        std::cout << "Usage: " << argv[0] << " <input> <output>\n";
        return 1;
    }
    std::ifstream is{argv[1], std::ios::binary | std::ios::ate};
    is.exceptions(std::ifstream::failbit);
    const auto& count = is.tellg();
    if (!count) throw std::runtime_error{"empty file"};
    std::vector<char> buff(count);
    is.seekg(0, std::ios::beg).read(&buff[0], count);

    auto& Dos = reinterpret_cast<IMAGE_DOS_HEADER&>(buff[0]);
    if (Dos.e_magic != IMAGE_DOS_SIGNATURE) throw std::runtime_error{"invalid DOS signature"};
    auto& Nt = reinterpret_cast<IMAGE_NT_HEADERS&>(buff[Dos.e_lfanew]);
    if (Nt.Signature != IMAGE_NT_SIGNATURE) throw std::runtime_error{"invalid NT signature"};

    std::size_t vaddr;
    if (Nt.OptionalHeader.Magic == IMAGE_NT_OPTIONAL_HDR64_MAGIC) {
        auto& Nt64 = reinterpret_cast<IMAGE_NT_HEADERS64&>(buff[Dos.e_lfanew]);
        vaddr = Nt64.OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_IMPORT].VirtualAddress;
    } else {
        auto& Nt32 = reinterpret_cast<IMAGE_NT_HEADERS32&>(buff[Dos.e_lfanew]);
        vaddr = Nt32.OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_IMPORT].VirtualAddress;
    }

    auto sections = IMAGE_FIRST_SECTION(&Nt);
    for (std::size_t i = 0; i < Nt.FileHeader.NumberOfSections; ++i) {
        auto offset = vaddr - sections[i].VirtualAddress;
        if (!vaddr || vaddr < sections[i].VirtualAddress || offset > sections[i].Misc.VirtualSize)
            continue;
        parse_import(sections[i], &buff[sections[i].PointerToRawData], vaddr);
        break; // 假设导入表位于单个节内
    }

    std::ofstream os{argv[2], std::ios::binary};
    os.exceptions(std::ofstream::failbit);
    os.write(&buff[0], buff.size());
    std::cout << "Successfully reordered imports and wrote to " << argv[2] << "\n";
    return 0;
} catch (const std::exception& e) {
    std::cerr << "error: " << e.what() << '\n';
    return 1;
}

References

  1. Stack Overflow. Changing Windows DLL load order? (load order, not search order). https://stackoverflow.com/questions/6371779/changing-windows-dll-load-order-load-order-not-search-order
  2. Alibaba Cloud Developer Community. Why can’t you call LoadLibrary and FreeLibrary inside DllMain? https://developer.aliyun.com/article/246619
  3. Russinovich, M., Solomon, D. A., & Ionescu, A. Windows Internals, Part 1 (7th ed.). Microsoft Press, 2017.

Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).

Get in touch