Recovering the Encryption Seed in FlexNet Publisher ≤ V11.16
About the author. I’m Colin Xiao — an independent reverse engineer, mostly working solo on contract jobs. I dig into compiled code of every kind, including but not limited to: Android and iOS apps, x86/x64/ARM native binaries, packers and obfuscators, DRM and license checks, network protocols and APIs, crypto and signature algorithms, malware, game binaries, and embedded/IoT firmware. I also handle large-scale data collection and vulnerability research. If you have any project in mind, I’d be glad to take it on — whatever it is, as long as it’s within my wheelhouse. Just reach out. You can reach me on X at @colinxiao_lab or on Telegram at https://t.me/ColinXiao666.
This research is published for educational and defensive purposes. The target sample is anonymized throughout.
Recovering the Encryption Seed in FlexNet Publisher ≤ V11.16
The problem
FlexNet Publisher V11.16 (the FlexLM licensing toolkit) made a change: it hides the 3D4DA1D6 marker that was previously used to recover the encryption seed, so the old quick-localization trick no longer works.
Two methods remain: the generic method, and the 0F42400h marker method.
Background on the change
Starting from some version after v11.4 (the exact version is hard to pin down), FlexLM can drop non-ECC verification at link time. Even a correct 12-bit or 20-bit value will then be judged illegal. The default makefile in the V11.14 SDK uses the lmgr_trl library, which is exactly the “non-ECC dropped” variant.
- Daemons not linked against
lmgr_trlcan use the0F42400hmarker in place of3D4DA1D6to locate the seed and recover it directly. - Daemons linked against
lmgr_trlcan only use the generic method. But even then, recovering the seed is meaningless: the module doesn’t validate non-ECC at all, and any module recognized as that format is immediately treated as illegal. As the flow below shows, execution never reaches the left-hand path.
Colin Xiao — I take on contract reverse-engineering work, and writeups like this are how I show what I can do. If you need help with anything in this space — including but not limited to the areas I cover above — I’m available. No project is too big or too small — if it’s reverse engineering and I can do it, I’ll take it. DM me on X (@colinxiao_lab) or Telegram (@ColinXiao666).
Get in touch