Hex probe v12

Runs a hex dump as machine code on three emulated processors, then compares what happens with random data.

The data

Any spacing works. Only hex digits are read, so glued bytes and line breaks are fine.

Does it behave like a program?

Each machine starts at every byte in turn. The best run it manages is compared with the same test on 30 random buffers and 30 shuffles of this data. The data is also tried at all eight bit alignments. Any reading that beats every comparison buffer is retested against up to 300 more of each before it counts as a hit.

Starting…

Run it

Execute the bytes on one machine and see, line by line, what it did.

Recovered program

Turns the data into a program: every defined instruction is kept, undefined opcodes are skipped, and the original addresses are preserved with .org lines so jump and branch targets stay valid. Flip the switch to run the listing itself in the terminal: memory holds only the listed instructions, and a jump into a skipped gap continues at the next listed address.

probe@hex: program listing ListingRun

What the bits look like

Each square is one 28-byte line of the listing.

Bit map

Terms

6502, CHIP-8, 8086
The three processors emulated. The 6502 is an 8-bit chip from 1975 used in early home computers. CHIP-8 is a tiny virtual machine from the 1970s built for simple games. The 8086 is a 16-bit Intel chip from 1978 and the ancestor of PC processors.
Bit alignment
Which bit of the stream counts as the first bit of the first byte. Shifting it by 1 to 7 bits changes every byte, so data that is misaligned looks like noise.
Bytes executed
Bytes that belonged to an instruction the emulator actually ran. A line is "reached" when at least one of its bytes was executed.
Control program
A program of 3,072 bytes by default (any size you set) with a known result and no loops. Every byte is executed once and no address runs twice, so the number of instructions run follows from the bytes: fewer than its byte count, because most instructions are two or three bytes long. A separate model predicts the final registers and every byte it writes, so a correct emulator must match it exactly.
Recovered program
A listing of the instructions a sweep finds in the data. Defined opcodes are kept, undefined ones are skipped, and gaps are marked with a new origin so addresses stay as they were. Instructions the sweep found by entering the middle of another instruction, or that run past the end of the data, are dropped.
Self-test
Before anything else, each emulator runs the control program and must match that model exactly. The badge in the results shows the outcome.
Emulator
A program that imitates a processor: it reads bytes, carries out the instructions they encode, and tracks registers and memory.
Entropy
How unpredictable the bytes are, in bits per byte. 0 means one value repeated; 8 means all 256 values equally likely, as in random data.
Halt
The end of a run: a stop instruction, an undefined opcode, a fault, running past the end of the data, an address about to run a second time, or the instruction limit.
Instruction and opcode
An instruction is one operation for the processor. Its opcode is the first byte, which says which operation; any bytes after it are operands, such as a number or an address.
Instruction limit
The most instructions one run may execute before it is stopped. Default 20,000.
Lag and repeat strength
For a lag of L bits, the share of bits that equal the bit L places later. Random data gives 50%. Repeat strength rescales this so 0 is chance and 1 is a perfect repeat.
Line
28 bytes, the width of a line in the original listing. Line numbers start at 1.
Loose bits
Bits left over after the last whole byte when the total is not a multiple of 8.
Offset
A byte's position counted from the start of the data, starting at 0 and shown in hexadecimal, such as 0x0B0F.
Random buffer and shuffled data
A random buffer has the same length as the data with every byte chosen at random. Shuffled data has the same bytes in a random order, which keeps their frequencies but destroys any sequence. Both use fixed seeds, so results repeat.
Run
One execution from a starting byte until it halts. A run never executes the same address twice, so its instruction count can't exceed the number of distinct addresses it covered.
Stop instruction
An instruction that ends the run on purpose: BRK on the 6502, HLT or INT on the 8086, waiting for a key on CHIP-8.
Strict and lenient
Strict stops at the first undefined opcode or stop instruction. Lenient skips past both and carries on.
Sweep
After each halt, restart at the first byte not yet executed, until every byte has been visited. It reaches every line by construction, so it measures run length, not whether the data is code.
Undefined opcode
A byte that is not a valid instruction on that processor. On the 8086 core the wording is "outside the emulated subset", because the real chip accepts opcodes this emulator does not implement.
Variety
The number of different opcodes a run executed. Best run means the run with the highest variety, counted from a starting byte until it halts or revisits an address.
XOR differential
A bit string that is 1 wherever two files differ and 0 wherever they match. XOR it with File 1 to get File 2.

Limits