Static UPX Unpacking: Rebuilding the Original PE Without Running It

UPX is probably the most common executable packer in the malware ecosystem. It is open source, free, fast, and it produces a working executable that is often half the size of the original. For an analyst, it also means that what lands in the triage queue is not the malware itself but a small decompression stub sitting on top of an opaque, high-entropy blob.

Malva.RE now handles UPX-packed Windows executables end to end: detection of x86 and x64 UPX binaries, static decompression, and reconstruction of the original PE file, which is then fed back into the rest of the analysis pipeline. Nothing is executed, emulated or sandboxed at any point: the packed file is treated purely as data.

This article explains how the UPX format works on Windows, what has to be undone to get the original program back, which pitfalls a static unpacker has to deal with, and how malware authors modify UPX to break the official upx -d.

Why this matters for analysts

When a sample is UPX-packed, almost every static signal is hidden:

• the strings of the real program (URLs, mutexes, registry keys, user agents) are compressed;
• the import table only lists the handful of functions the stub needs (typically LoadLibraryA, GetProcAddress, VirtualProtect, ExitProcess);
• YARA rules written against the payload never match, because the bytes they look for do not exist in the file;
• configuration extractors for RATs and stealers find nothing to parse;
• family classification falls back to “packed executable”, which tells you very little.

The usual workaround is to run the sample in a sandbox and dump it from memory, or to run upx -d locally. The first option requires execution infrastructure and carries its own evasion problems. The second often fails on malware, precisely because attackers tamper with the UPX metadata to make it fail.

With static unpacking, the reconstructed PE becomes a child file in the Malva.RE report tree, typed as a Windows executable or DLL. From there it goes through exactly the same analysis as any uploaded binary: PE parsing, YARA, family-specific collectors, configuration extraction, IOC aggregation. The packed file and the unpacked payload are both visible, with the relationship between them.

Why malware uses UPX (and modifies it)

UPX was designed as a legitimate size optimiser, and it is still widely used by legitimate software, which is part of why it is attractive to attackers:

• Size: smaller droppers and payloads are easier to deliver and embed in documents, archives or other loaders.
• Cheap obfuscation: compression removes plaintext strings and the real import table from naive static inspection.
• Plausible deniability: a UPX-packed binary is not malicious by itself, so “packed with UPX” is a weak detection signal on its own.
• Toolchain friendliness: UPX supports EXE and DLL, 32 and 64 bits, and works with the output of most compilers.

The obvious drawback for an attacker is that upx -d restores the original file in one command. So a large share of UPX-packed malware is deliberately modified after packing, with changes that keep the stub working (the stub does not care about most metadata) but break the official decompressor:

• the UPX! magic is erased or overwritten;
• sections UPX0 / UPX1 are renamed to something innocuous (.text, .data, random names);
• fields of the pack header (sizes, checksums) are altered so that integrity checks fail;
• the whole pack header is wiped.

The packed program still runs, because the stub only uses constants embedded in its own code. But the official tool relies on that metadata and refuses to unpack. A static unpacker that wants to be useful on real malware has to cope with this.

Anatomy of a UPX-packed PE

Sections before and after

UPX rewrites the section table completely. The original sections are merged into a single virtual region that the stub fills at runtime.

 Original PE                         UPX-packed PE
 ┌──────────────────────┐            ┌──────────────────────────────┐
 │ DOS header + stub    │            │ DOS header + stub            │
 │ NT headers           │            │ NT headers (rewritten)       │
 │ Section table        │            │ Section table (3 entries)    │
 │                      │            │ "x.yz" version + PackHeader  │
 ├──────────────────────┤            ├──────────────────────────────┤
 │ .text                │            │ UPX0  raw size = 0           │
 │ .rdata               │            │       virtual size = size of │
 │ .data                │   ─────►   │       all original sections  │
 │ .rsrc                │            ├──────────────────────────────┤
 │ .reloc               │            │ UPX1  compressed image       │
 │ ...                  │            │       + decompression stub   │
 └──────────────────────┘            │       (entry point is here)  │
                                     ├──────────────────────────────┤
                                     │ UPX2 / .rsrc  stub imports,  │
                                     │       uncompressed resources │
                                     └──────────────────────────────┘

Key properties of this layout:

• UPX0 has a raw size of zero and a large virtual size. It reserves the address space where the original sections will be decompressed.
• UPX1 contains the compressed data followed by the decompression stub. The entry point of the packed file points into this section.
• A third section (named UPX2, or .rsrc when resources are kept) holds the stub’s own minimal import table and whatever resources UPX chose not to compress: typically the main icon, the version information and the manifest, so that Explorer and the loader still see them.

The PackHeader

In a Windows PE, UPX stores its metadata in a fixed 32-byte structure called the PackHeader, placed in the header area just after the section table and usually preceded by a plain-text version string. Its layout for the revisions found in Win32/Win64 files is:

A few remarks that matter in practice:

• The revision byte is not the UPX version. Several UPX releases share the same revision, and the version string in front of the header is trivially editable. The unpacker records the header revision as structural evidence, but does not claim a UPX release number it cannot prove.
• Our reader handles the header revisions used by the 32-byte layout (10 to 14). Other revisions are not interpreted by guessing field positions; they go through the recovery path described further below.
• The two Adler-32 checksums and the header checksum are what allow a decompressor to tell a genuine header from random bytes, and a correct decompression from a corrupted one.

What about l_info and p_info?

UPX documentation and source often mention the l_info and p_info structures. These belong to the Unix-like formats (ELF and related loaders), where they describe the loader and the program blocks. They are not part of the Windows PE packing format: PE files rely on the PackHeader plus a restoration trailer inside the compressed image, described below. Since this work targets Windows PE files, l_info / p_info are not involved.

Step 1: detecting UPX on x86 and x64

The gap we had to close

Before this work, the UPX signatures loaded in Malva.RE’s YARA set targeted ARM decompression stubs (Windows CE flavours). They never matched an x86 or x64 Windows executable, which means that, in practice, standard UPX-packed PE files were not tagged as UPX at all.

The new detection rule combines independent indicators, any of which is enough on a valid PE with at least two sections:

• a section named UPX0, UPX1, .UPX0 or .UPX1;
• the UPX! magic within the first 4 KiB of the file (the header area);
• a known entry-point stub prologue, matched as bytes at the entry point:
  • x86 EXE: pushad; mov esi, imm32; lea edi, [esi+imm32]; push edi;
  • x86 DLL: cmp byte [esp+8], 1; jnz rel32; pushad; mov esi, imm32 (the stub only unpacks on DLL_PROCESS_ATTACH);
  • x64 EXE: push rbx; push rsi; push rdi; push rbp; lea rsi, [rip+rel32]; lea rdi, [rsi+imm32];
  • x64 DLL: a register-saving prologue that is too generic on its own, so it is only accepted together with the structural signature of UPX (first section with zero raw size and a non-zero virtual size).

The entry-point patterns are what make detection resilient to renamed sections and erased magic: an attacker can rewrite metadata freely, but the stub has to keep its code to work.

Structural scoring

On top of the YARA rule, the unpacker performs its own structural analysis before trying anything. It looks for:

1. The UPX layout: an empty virtual section (no raw data, virtual size of at least one page) immediately followed by a stored section that contains the entry point.
2. Conventional UPX section names.
3. A stub-like byte pattern at the entry point.
4. High entropy in the stored section (only counted together with the layout, since entropy alone means nothing).
5. A PackHeader, searched in bounded windows: the beginning of the file, the end of the file, and the start of each section. The file is never scanned byte by byte across hundreds of megabytes of overlay.

Each indicator contributes to a score, and inconsistencies are recorded as anomalies: magic erased, header checksum mismatch, declared lengths impossible for the file size, UPX-like layout with renamed sections, UPX-like layout with no readable header. These anomalies are surfaced in the report, because a tampered UPX header is a meaningful signal in itself: legitimate software has no reason to do it. When such tampering is found, the sample is also tagged accordingly.

Step 2: decompressing the image

Locating the compressed stream

The PackHeader gives the compressed and uncompressed lengths, but not the file offset of the compressed stream: in a PE, the header lives in the header area while the data lives in UPX1, and the stub simply knows where to look. The unpacker therefore builds a short, bounded list of candidate offsets:

• the byte right after each PackHeader found;
• addresses embedded as immediate operands in the stub: on x86, the mov esi, imm32 that loads the source pointer; on x64, the RIP-relative lea rsi, [rip+rel32]. These are read as constants and converted to file offsets; no instruction is decoded or executed;
• the start of each stored section, plus a few small offsets.

The candidate list is deduplicated and capped, so an adversarial file cannot make the unpacker try an unbounded number of positions.

Compression methods

UPX encodes the compression algorithm in the method byte of the PackHeader:

Any other method identifier is rejected explicitly. In particular, an unknown method byte is never silently interpreted as another algorithm.

NRV2B, NRV2D, NRV2E

The NRV family (from the UCL library) consists of simple LZ77 variants with a bit-oriented control stream. Control bits are fetched in words of 8, 16 or 32 bits depending on the variant, most significant bit first, and interleaved with literal bytes:

• a control bit of 1 means “copy the next byte as a literal”;
• a 0 starts a match. The match offset is encoded with a variable-length (Elias-gamma-like) prefix followed by one extra byte. A special prefix value means “reuse the previous offset”, which is very effective on repetitive code;
• the decoded offset value 0xFFFFFFFF is the end-of-stream marker;
• the match length is encoded with a few direct bits, or with another variable-length prefix for long matches. NRV2D and NRV2E steal the low bit of the offset to carry part of the length, and NRV2E has a slightly different short-length scheme;
• very distant matches get an implicit extra byte of length (beyond 0xD00 for NRV2B, 0x500 for NRV2D/E).

Because the stream is self-terminated, a decoder can check that it consumed exactly c_len bytes and produced exactly u_len bytes. Both checks are enforced.

LZMA

For LZMA, UPX does not use the standard 13-byte .lzma header. It prepends a compact 2-byte property header:

• first byte: pb in the low 3 bits, and lc + lp in the upper bits;
• second byte: lc in the low nibble, lp in the high nibble.

The range-coded stream follows immediately, without an end marker; the expected output size comes from u_len. Property values are validated (bounds on lc, lp, pb and consistency between the two bytes) before any model is allocated. The decoder is a full LZMA implementation (literal coding with match-byte prediction, rep0 to rep3 distances, short reps, length coders, distance slots and align bits).

Integrity checks

When the header is intact, the decompression is accepted only if:

1. the Adler-32 of the compressed bytes matches c_adler;
2. the decoder consumes the declared compressed length (for NRV methods);
3. the output has exactly the declared length u_len;
4. the Adler-32 of the decompressed bytes matches u_adler.

An important subtlety: u_adler is computed on the still-filtered image, that is, before the call/jump transform is reversed. Verifying it after undoing the filter fails on every filtered sample.

Step 3: reversing the call/jump filters

Why UPX filters code

x86 call rel32 (E8) and jmp rel32 (E9) instructions store a relative displacement. Two calls to the same function from different places have different operands, which compresses poorly. Before compression, UPX rewrites these operands into absolute targets, so repeated calls to the same function become identical byte sequences. The stub reverses the transform after decompression.

The family of filters used on Windows PE files is often referred to as the CTO filters, after the filter_cto parameter stored in the header.

How the reversal works

For each opcode of interest in the code range, the first byte of the operand is compared with the marker (filter_cto). If it matches, the operand is a transformed one: the remaining 24 bits are read as a big-endian absolute position, converted back to a displacement relative to the end of the instruction, and written back in little-endian form.

The filter is applied only to the original code range, taken from BaseOfCode and SizeOfCode in the original optional header, which UPX saves inside the compressed image.

A classic pitfall: the marker is not always zero

A tempting shortcut is to assume the marker byte is 0. It is not. UPX chooses the marker per file, to minimise false positives on data that happens to look like E8/E9. Some binaries (Visual Basic 6 executables are a common example) end up with a non-zero marker. With the wrong marker, two things go wrong at once: real calls stay transformed, and unrelated data bytes are “un-filtered” into garbage. The result is a PE that looks plausible but contains broken code, which silently degrades every downstream analysis.

On our reference corpus, UPX selected filter 0x49 for x64 DLLs with branches regardless of the filter option requested, which illustrates why the unpacker must read the effective filter from the file and never assume it from the command line.

Filter 0x11 and any other unknown filter identifier are refused explicitly.

Step 4: rebuilding the original PE

The decompressed buffer is not a PE file yet. It is the memory image of the original sections, followed by metadata that UPX needs to restore everything else. Rebuilding a valid file requires several distinct operations.

The restoration trailer

Two layout facts are essential:

• byte zero of the decompressed buffer corresponds to the lowest original section RVA, not to RVA 0. The original headers are not part of the mapped image;
• the last four bytes of the buffer hold the offset, inside the buffer, of a copy of the original NT headers and section table.

Right after that saved section table, UPX writes compact records used to restore imports, relocations and resources. Which records are present depends on what the original file contained. The unpacker reads exactly the records it expects, and requires the trailer to end where it should: any unexplained trailing bytes mean an unfamiliar layout, and the file is refused rather than reconstructed approximately.

Headers and section layout

The output file is assembled as follows:

• the DOS header and DOS stub of the packed file are kept;
• the saved original NT headers and section table are written at the packed file’s e_lfanew;
• each original section is copied from the decompressed image to its original PointerToRawData, after checking the alignment against the original FileAlignment and verifying that the bytes actually exist in the decompressed buffer;
• SizeOfImage is recomputed from the section table and SectionAlignment;
• the PE CheckSum is set to zero, and the data directories that no longer describe anything valid are cleared: certificate table, debug directory, bound imports and IAT directory.

Imports

The packed file only imports what the stub needs. The original import table is replaced, inside the compressed image, by a compact import record stream. For each imported DLL it stores:

• the location of the DLL name (the names are kept in the packed file’s own import area, so the stub can pass them to LoadLibraryA);
• the offset of the original IAT;
• a list of tagged entries: import by name (a zero-terminated string), import by 16-bit ordinal, or by 32-bit ordinal (which must still fit the 16-bit PE ordinal field), terminated by a zero tag.

Two producer profiles exist and both are handled:

1. Rebuilt names: the record gives a destination where DLL names must be written. The unpacker lays out the DLL names, then the hint/name entries (two-byte aligned, with the zero hint UPX uses), fills each IAT thunk with either a hint/name RVA or an ordinal with the high bit set, and writes the import descriptors.
2. Preserved addresses: later UPX versions keep the original thunks and hint/name addresses in the image. The unpacker restores DLL names and function names at their original locations, and checks that every preserved ordinal matches the compact record.

The number of rebuilt descriptors must fit the original import directory size. Every count (modules, functions, string lengths) is bounded.

Relocations

Relocations are also stored in compressed form:

• a stream of position deltas: one byte for small deltas, an extended 20-bit form introduced by bytes 0xF0 and above, and a full 32-bit form as a last resort, terminated by a zero;
• at each relocated position, UPX stores the pointer in big-endian, relative form (this also compresses better).

The unpacker parses the whole stream first, then converts each target back into a little-endian absolute address based on the original ImageBase. Finally, it regenerates a standard .reloc table: entries grouped by 4 KiB page into base relocation blocks, typed IMAGE_REL_BASED_HIGHLOW for PE32 or IMAGE_REL_BASED_DIR64 for PE32+, each block padded to a 4-byte boundary. The rebuilt table must have exactly the size declared by the original relocation directory. Files with stripped relocations (IMAGE_FILE_RELOCS_STRIPPED) carry no relocation record and need none. Only the relocation record flavour observed on our corpus (flags 0) is accepted.

Exports and TLS

For DLLs, UPX may move the export directory out of the compressed area so that the loader can still resolve exports before the stub runs. In that case, the export directory, its function table, name table and ordinal table are copied back to their original location and their RVAs rebased. When exports were left in place, they are taken from the decompressed image.

TLS data and callbacks are validated, never invoked: the template range, index address and each callback address must lie within the rebuilt image.

Resources

Resources are the most intricate part. UPX keeps a resource directory in the packed file because Windows needs some resources (icon, version information, manifest) without running the program. That directory can describe a mix of payloads:

• payloads that were compressed with the rest of the image, and are now back in the decompressed buffer at their original RVA;
• payloads left uncompressed in the packed file, each preceded by four bytes holding its original RVA.

The unpacker walks the stored resource tree (with cycle detection and depth and node limits), copies directory nodes, names and data entries to the original resource location, moves the uncompressed payloads back to their original addresses, and fixes the data entries accordingly.

Icons need special care. Depending on the --compress-icons setting, UPX may expose only the first image of the first icon group, so that Explorer still shows an icon, and lower the image count in the group directory. The original count is stored in the trailer; the unpacker restores it in the first GRPICONDIR. All four icon settings (0 to 3) are covered by our tests. A resource tree that is itself only partially compressed, and would need merging, is refused.

Final validation

Before a reconstructed file is accepted, it is parsed again as a PE and validated: every non-empty data directory must map into the file, import descriptors must be terminated and their thunks valid, relocation blocks well formed with targets inside the image, export counts and ordinals consistent, TLS addresses inside the image, and the resource tree free of cycles. If any check fails, the candidate is discarded.

Nested packing

Some samples are packed several times. After a successful layer, the result is examined again, and further UPX layers are unpacked up to a depth of three. Deeper nesting is reported as a failure rather than silently stopping halfway.

Step 5: recovering tampered samples

This is where a static unpacker earns its keep on real malware. When the PackHeader is damaged or missing, the process becomes a bounded search, always ending with the full reconstruction and validation above.

Damaged but present header

A header with an erased magic is still recognisable: the revision byte, format, method, level and lengths must all be plausible, and the method must be one we support. Such headers are used with the same integrity checks as intact ones, when those checks still hold.

Header lengths that lie

If decoding with the declared lengths fails, the unpacker stops trusting them and tries the supported methods (header method first) at each candidate offset, letting the self-terminated NRV streams define their own length. A note records the method, offset and candidate number used, and a mismatch between the declared and actual decompressed size is reported as an anomaly.

LZMA without any size

LZMA streams in UPX carry no end marker, so without u_len the decoder does not know where to stop. During recovery only, the decoder stops at a point where the range coder is fully flushed and the output ends with a plausible restoration trailer: a valid saved NT header referenced by the final four bytes. This is only a candidate size; the full reconstruction and validation still have to succeed.

Header wiped entirely

Without a header, the filter type and marker are unknown, and getting them wrong produces silently broken code. The unpacker therefore reads them from the stub itself: the end of each known decompression routine (NRV and LZMA, 32 and 64 bits) is followed by the filter-reversal code, whose shape identifies the filter and whose embedded constant is the marker. These are matched as bounded byte profiles, collected from files produced by the official UPX tools, for older and newer stub generations. No instruction is decoded, emulated or executed.

If the stub profile is unknown and the code section contains branch opcodes that might have been filtered, the unpacker refuses to produce an output rather than guessing. A wrong guess would produce a plausible but corrupted PE, which is worse than no PE.

What we test against

Every packed reference file is also unpacked after four in-memory alterations, mirroring what is seen in the wild:

1. the four UPX! bytes erased;
2. all section names replaced;
3. the uncompressed length replaced by an absurd value, which also invalidates the header checksum;
4. the 32 bytes of the PackHeader wiped.

An additional mutation checks that an unknown filter profile is refused even when the data decompresses correctly.

Safety and resource limits

The unpacker processes hostile input inside a shared analysis worker, so it is designed defensively:

• pure static processing: input is a byte array; no process is started, no PE or assembly is loaded, no native call, emulation, disk or network access takes place;
• input size and output size are capped, and output buffers grow progressively instead of trusting a size declared by the header;
• a global timeout with cooperative cancellation in every loop; when the timeout fires, the unpacking task is cancelled and awaited, so no orphaned computation keeps consuming CPU;
• every read, back-reference distance, length and address is bounds-checked, and every PE structure walk is bounded in count and depth;
• the number of recovery candidates and the nesting depth are capped;
• an unpacking failure never fails the scan: the UPX tag stays, and the failure reason is shown in the report.

Validation and current scope

The reconstruction is validated against the official upx -d (with relocation stripping disabled) on a corpus of benign test programs we compiled and packed with the official UPX 3.91 and 3.95 tools: EXE and DLL, x86 and x64, all four method families, five filter options, resources kept or mixed, and the four icon settings. Success requires the same architecture, file type, entry point, SizeOfImage, section count and section attributes, and byte-identical raw data for every section, padding included. Accepted differences are limited to headers: retained DOS stub, zeroed checksum, cleared certificate, debug, bound import and IAT directories. The decoders are written from format descriptions; no UPX or UCL source code is reused.

Known limits, stated plainly:

• only UPX 3.91 and 3.95 output has been compared against an independent reference so far. Other 3.x releases, as well as 1.x, 2.x, 4.x and 5.x, may work where the formats are shared, but this is not yet demonstrated;
• the NRV 8-bit and 16-bit variants are validated with test vectors only, as no official PE using them is in the corpus;
• the overlay and any Authenticode signature are not restored;
• an original section table that does not start with the lowest section RVA is refused;
• unknown filters (including 0x11), unknown relocation record flags, partially compressed resource trees needing a merge, and unrecognised stub profiles with missing filter parameters are refused;
• .NET assemblies are not claimed: our .NET test input was refused by the UPX tools themselves;
• only Windows PE files (x86 and x64) are in scope: UPX for ELF, Mach-O or other formats is not handled by this component.

In the Malva.RE report

When a UPX-packed sample is submitted, the report now shows:

• the UPX tag, now raised for x86/x64 files as well;
• the compression method and filter found, and a confidence value (an evidence score, not a calibrated probability);
• any anomalies: erased magic, renamed sections, inconsistent lengths, missing header, heuristic recovery details. A dedicated tag flags tampered UPX headers;
• the reconstructed PE as a child node in the file tree, locked as a Windows executable or DLL, and analysed like any other binary: PE structure, YARA matches on the real program, family detection, configuration extraction and IOCs.

This closes a frustrating blind spot: a commodity stealer or RAT wrapped in UPX, with its header damaged to defeat upx -d, is now analysed as the stealer or RAT it actually is, without ever being run.

Try it on your own samples

You can submit UPX-packed samples to Malva.RE and inspect the reconstructed payload directly in the report tree. Start a free trial. Please keep in mind that reports produced on the free plan are public.