David Fifield <david@bamsoftware.com>
Last updated:
git clone https://www.bamsoftware.com/git/exepack.git
exepack is a program to compress and decompress 16-bit DOS executables with EXEPACK, a format for self-extracting executables.
exepack unpacked.exe packed.exe
exepack -d packed.exe unpacked.exe
unzip -p comic.zip comic.exe | exepack -d /dev/stdin unpacked.exe
I wanted to reverse engineer some old DOS games like Mega Man and Captain Comic. These games' executables are packed with EXEPACK; you need to unpack them in order for the disassembly to make sense. I decided to write my own unpacker after encountering a file compressed using a variant of EXEPACK that other tools couldn't handle at the time.
Goals:
exepack is written in Rust. You need rustc and cargo to compile it.
If you have a DOS EXE file that contains the string
Packed file is corrupt,
it is most likely packed with EXEPACK.
The most prominent documentation of EXEPACK online is at the DOS Game Modding Wiki: http://www.shikadi.net/moddingwiki/Microsoft_EXEPACK#File_Format. As of this writing, the format described there is just one of several slightly incompatible EXEPACK formats. The formats differ in the size of the EXEPACK metadata header, whether they support an optional padding block, the size of their executable decompression stub, the localization of an error message, and the presence of certain bugs.
The general structure of an EXEPACK-packed file is:
┌──────────────────────────────────────┐ runtime address │ EXE header │ ├──────────────────────────────────────┤ ← ds:0100 = es:0100 │ compressed data │ ├──────────────────────────────────────┤ │ optional skip_len padding │ │ (only if EXEPACK header is 18 bytes) │ ├──────────────────────────────────────┤ ← cs:0000 │ EXEPACK header (16 or 18 bytes) │ ├──────────────────────────────────────┤ ← cs:ip │ EXEPACK decompression stub │ ├──────────────────────────────────────┤ │ packed relocation table │ ├──────────────────────────────────────┤ ← cs:exepack_size ┆ possible trailing garbage ┆
Pointers use segment:offset notation:
cs:ip means, as a linear address,
16×cs+ip.
The cs and ip fields in the
EXE header
tell us where to find the EXEPACK header and how big it is.
There are two possible EXEPACK headers,
a 16-byte one and an 18-byte one.
They differ in the presence of a skip_len field.
| 16-byte header | 18-byte header |
|---|---|
uint16_t real_ip uint16_t real_cs uint16_t mem_start uint16_t exepack_size uint16_t real_sp uint16_t real_ss uint16_t dest_len uint16_t signature "RB" |
uint16_t real_ip uint16_t real_cs uint16_t mem_start uint16_t exepack_size uint16_t real_sp uint16_t real_ss uint16_t dest_len uint16_t skip_len uint16_t signature "RB" |
The header field names are from ModdingWiki.
mem_start is not an actual meaningful header field;
it is just temporary storage used by the decompression stub.
exepack_size is the size of the entire EXEPACK block:
header, stub, and packed relocation table.
dest_len should perhaps instead be called
uncompressed_len:
it's the size (in 16-byte paragraphs) of the uncompressed data.
Similarly, cs could also be called
compressed_len, because the compressed data
ends just before the EXEPACK header.
The only exception is when skip_len is present;
in that case, uncompressed_len
and compressed_len both get reduced by
16×(skip_len − 1).
With the 16-byte header, it is as if skip_len
always has the value 1; i.e., no skip_len padding.
(Aside: apart from complicating the unpacking algorithm,
skip_len doesn't seem to serve any purpose.
w4kfu found
many executables with skip_len > 1,
but they would work just as well with skip_len = 1.)
The decompression stub immediately follows the EXEPACK header.
As it is located at cs:ip, it is the code that DOS
will jump to as soon as the compressed executable is loaded.
The stub is responsible for copying itself out of the way,
decompressing the compressed data,
and jumping to the entry point of the original uncompressed program.
There have been several different decompression stubs over the years.
The following table shows the characteristics of the ones that are known to me.
See doc/README.stubs and doc/*.asm in the exepack source code
for commented disassembly.
The one with size 283
is the format documented at ModdingWiki.
This program uses its own custom stub,
designed to fix the problems of the other stubs,
while keeping a size of 283 for compatibility with other external unpackers.
| size | skip_len? |
restores ax? |
A20 bug? | relocation 0xffff bug? | allows expansion? | error string | producer |
|---|---|---|---|---|---|---|---|
| 258 | no | no | yes | yes | no | Packed file is corrupt | EXEPACK 4.00; or LINK /EXEPACK 3.02, 3.05, or 3.06 |
| 258 | no | no | yes | yes | no | Fichero corrompido | ? |
| 279 | no | no | yes | no | no | Packed file is corrupt | EXEPACK 4.03, LINK /EXEPACK 3.51, or IBM Linker/2 1.0 |
| 277 | no | no | yes | no | no | Packed file is corrupt | LINK /EXEPACK 3.10, 3.60, 3.61, 3.64, 3.65, 5.01.20, 5.01.21 |
| 283 | yes | no | yes | no | no | Packed file is corrupt | EXEPACK 4.05 or 4.06 |
| 290 | no | yes | no | no | no | Packed file is corrupt | LINK /EXEPACK 3.69, 5.05, 5.10, 5.13, 5.15, 5.31.009, 5.60, 5.60.220, or 5.60.339 |
| 283 | yes | yes | no | no | yes | Packed file is corrupt | exepack (this program) |
skip_len?skip_len;
"yes" means an 18-byte EXEPACK header with skip_len.
ax?ax has a meaning.
The decompression stub should restore the original value of ax
before jumping to the decompressed code, but most versions
do not.
An external unpacker like this one doesn't care about the contents
of the decompression stub, but it has to know its length
in order to locate the packed relocation table.
There is no field that indicates where the stub ends
and the relocations begin;
it's implicit in the offsets encoded into the instructions of the stub.
The error string Packed file is corrupt
is a fairly reliable indicator:
it always appears right at the end of the stub.
However the message may be localized (Fichero corrompido ),
so it's not completely foolproof.
I initially tried having a table of known stubs,
but later I changed it to instead
search for the byte pattern
that precedes the error message,
cd 21 b8 ff 4c cd 21,
which encodes the instructions
int 0x21; mov ax, 0x4cff; int 0x21,
then seek 22 bytes past the end of it.
Since the error message seems to always be 22 bytes,
this works.
You can always check your guess after reading the packed
relocation table; it should end exepack_size
bytes after the beginning of the EXEPACK header.
After the stub comes the packed relocation table.
Notionally, the relocation table is an array of segment:offset pointers.
EXEPACK compresses the array by normalizing all the pointers to
have a segment that is a multiple of 0x1000, and then storing 16 separate arrays
containing offsets only.
The first uint16_t is the number of offsets in the array
for segment 0000, followed by that many uint16_ts for the offsets themselves;
then a uint16_t for the number of offsets for segment 1000,
followed by that many offsets;
and so on up to segment f000.
The EXEPACK and LINK version numbers come from the source code of
UNP,
and from experimenting with various versions of those programs
sent to me by Dennis Luehring.
The Detect-It-Easy software has signatures for versions of EXEPACK:
EXEPACK.2.sg,
WordPerfect EXEPack.2.sg.
YaraRules has
EXEPACKv405v406 and
EXEPACKLINKv360v364v365or50121 rules.
RGB Classic Games
marks some versions
as "2nd generation", but I don't know what their criteria for that are.
Taking the above observations into consideration,
here is a rough algorithm for EXEPACK unpacking that is compatible with known formats.
An implementation would have to deal with several possible error conditions,
for example skip_len > dest_len.
min_extra_paragraphsheader_paragraphscsipmem_start = 16*header_paragraphs and seek to mem_start + 16*cs.ip bytes; this is the EXEPACK header. Save:
real_ipreal_csreal_spreal_ssdest_lenskip_len (if present; otherwise set to 1)compressed_len = 16*(cs - skip_len + 1) and uncompressed_len = 16*(dest_len - skip_len + 1).
Packed file is corrupt).mem_start and copy the next compressed_len
bytes into a buffer. Enlarge the buffer to length uncompressed_len if necessary.
Decompress the buffer
starting with a source index of compressed_len
and a destination index of uncompressed_len.
Truncate the buffer to length uncompressed_len if necessary.
This is the uncompressed program body.
min_extra_paragraphs = ((16*cs - mem_start)/16 + min_extra_paragraphs) - dest_lenss = real_sssp = real_spcs = real_csip = real_ipchecksum = 0checksum EXE header field with the checksum of everything written so far.
min_extra_paragraphs in an EXE file is something like the
BSS segment
in a Unix executable.
It specifies an amount of additional memory to allocate,
beyond the main program text.
min_extra_paragraphs is effectively part of the program image,
but the EXE file stores only its size, not its contents.
After decompressing, you must adjust min_extra_paragraphs
so that the before and after of the sum
program size + min_extra_paragraphs size
remains constant.
Usually, this will mean decreasing min_extra_paragraphs,
because the decompressed program is usually larger than the compressed program,
but min_extra_paragraphs may also increase.
See an investigation of how Microsoft EXEPACK.EXE and UNP handle this field.
exepack computes the checksum field for files it writes,
but ignores checksum in files it reads.
You may as well leave the field set to zero.
Microsoft KnowledgeBase article Q71971
tells how to compute the EXE checksum,
but also states that checksums are ignored in practice:
Note that Microsoft LINK does not correctly calculate the checksum if the linker command line includes the /CODEVIEW or /EXEPACK option switches. However, because the MS-DOS, Microsoft Windows, and OS/2 versions 1.x do not verify the checksum, this behavior does not present a problem under normal circumstances. Microsoft LINK version 5.3 and later do not compute a 16-bit or 32-bit checksum. The reserved bytes in the .EXE header are set to zero.
Various implementation of DOS I found
do not even examine checksum, let alone verify it:
EXE_Header
struct with a checksum field, but does not refer to it in its
DOS_Execute
function.exe_header
with an exCheckSum field, but does not refer to it in its
DosExeLoader
function.
Strangely, Microsoft EXEPACK.EXE 4.00
always uses a value of 0x1399 in its checksum fields; you
can see an instance of this in Revision 1 of
COMIC.EXE.
Microsoft EXEPACK.EXE 5.00
always writes a checksum of 0x0000.
Jason Summers has done corpus analysis of the EXE checksum.
The decompression algorithm is just as described at ModdingWiki. It runs backwards, and decompresses the buffer into itself.
You need to do the 0xb2 copy operation in reverse as shown,
because the destination region may overlap the source region.
I.e., you can't just memcpy in a forward direction.
You could probably use memmove.
Here is the decompression algorithm, with no error or bounds checking.
// The dst and src indices initially point one byte past the end
// of their respective regions.
decompress(buf, dst, src) {
while (buf[src-1] == 0xff)
src--;
do {
command = buf[--src];
length = buf[--src];
length = (length<<8) + buf[--src];
switch (command & 0xfe) {
case 0xb0:
fill = buf[--src];
for (i = 0; i < length; i++)
buf[--dst] = fill;
break;
case 0xb2:
for (i = 0; i < length; i++)
buf[--dst] = buf[--src];
break;
default:
error(); // Packed file is corrupt
}
} while ((command & 0x01) != 0x01);
}
Send bug reports to david@bamsoftware.com. This is also where you should send files that seem to be EXEPACK-compressed but which this program cannot handle.
The following are not bugs exactly, but rather questions that came up during development that I had to decide one way or another. I'm not sure that what I chose is the best way. If you have an opinion or insight, let me know.
The bytes_in_last_block and blocks_in_file
fields in the
EXE header
encode a total size for the EXE, which may be different (shorter) than
the actual size of the file.
What should we do when there is trailing data after the EXE data?
Currently, exepack honors the EXE header length in all cases:
it truncates the input as if the file were no longer
than the size stated in the EXE header,
and prints a warning:
warning: EXE file size is X; ignoring Y trailing bytes
An alternative would be to remember the trailing data, and append it to the newly compressed or decompressed file. Then there is the question of whether this should happen in compress mode, decompress mode, or both.
As for compress mode, in my tests, Microsoft EXEPACK.EXE 4.00 and 5.00 partially retain, but mangle trailing data. I compiled trail.c using Borland Turbo C 2.01 to produce trail.exe, which has 4656 bytes of EXE data and 2323 trailing bytes (looks like debugging symbols or something). Compressing it with EXEPACK.EXE 5.00 results in trail500.exe, which has 3999 bytes of EXE data and 1956 trailing bytes. The trailing bytes are similar to those of the input, but they have been partially cut off and overwritten with zeroes at the beginning.
As for decompress mode,
UNP retains trailing data by default,
but offers a -r option to discard it.
(UNP calls trailing data "overlay",
which may be what it actually is most of the time.)
But unp.doc cautions:
Compressed EXE files containing an overlay may not work correctly after they have been decompressed. Decompression expands the code size of the EXE file which also means that the overlay moves up. Some programs do not check where the overlay currently is but just use a constant to get the overlay. If this is the case, most anything can happen.
Single source file, written in C. Supports multiple formats. This is the alternative I recommend if exepack doesn't suit your needs.
Single source file, written in C. Only supports one EXEPACK format. Has a bug when processing certain packed relocation tables.
DOS-based unpacker for a ton of self-extracting executable formats. Written in assembly language. No longer maintained. The way it works is cute: it recognizes the input format, sets some breakpoints, runs the executable's own unpacking code, and copies the result out of memory. As a consequence, it really only works inside a real DOS environment. (And isn't safe to run on untrusted files.) I got some of the version numbers for different decompression stubs from UNP's labeled signatures in the source file exe/eexpk.asm.
EXEPACK and LINK.min_extra_paragraphs field.A great series of articles by Michal Necasek that get into the history of EXEPACK and the differences across versions:
Because of the wide deployment of various buggy versions of EXEPACK-packed executables, DOS and Windows dynamically detect and hotpatch EXEPACK code. (The image in the linked article is broken; you can find a copy here.)
My reverse engineering of Mega Man for DOS turned up some interesting unused content, which I documented at The Cutting Room Floor. I learned some tricks that I used to do an RTA speedrun, which was the world record for 5 months in 2018; and a tool-assisted speedrun.
I started a page for Captain Comic at The Cutting Room Floor as well. I did RTA speedruns of Revision 2 and Revision 5; the latter is the world record since . With Kabuto, I did a tool-assisted speedrun of Revision 1.
