BEAM File Format

"Atom" and "AtU8" - Atoms Table

Both tables have same format and same limitations (256 bytes max length) except that bytes in strings are treated either as latin1 or utf8.

• Read U32/big atoms count.

• For each atom: read byte length, followed by characters

Atoms[0] is a module name from -module(M). attribute.

"Code" - Compiled Bytecode

• Read U32/big code version (must match emulator's own version)

• Read U32/big max opcode, U32/big label count, U32/big fun count

• Read the code as a block. Format is discussed at BEAM Code Section Format.

"Abst" - Abstract Syntax Tree

Optional section which contains term_to_binary encoded AST tree.

A quick way to get Abst section (if it exists):

"CatT" - Catch Table

Contains catch labels nicely lined up and marking try/catch blocks. This section description is INCOMPLETE and UNTESTED.

• Read U32/big count

• Read array of count U32/big offsets or labels (not sure).

"FunT" - Function/Lambda Table

Contains pointers to functions in the module.

• Read U32/big count

Until the count do:

• Read U32/big fun_atom_index (name by index from atom table), U32/big arity, U32/big offset (code position), U32/big index, U32/big nfree (frozen values for closures), U32/big ouniq. Sanity check: fun_Atom_index must be in atom table range.

"ExpT" - Exports Table

Encodes exported functions and arity in the -export([]). attribute.

• Read U32/big count

Until the count do:

• Read U32/big export name atom index. Sanity check: atom table range.

• Read U32/big arity, U32/big label (offset in BEAM code section, should be translated into the loaded code offset).

"LitT" - Literals Table

Contains all the constants in file which are larger than 1 machine Word. It is compressed using zip Deflate.

• Read U32/big uncompressed size (prepare output buffer of this size). Run zip inflate (decompression) on the data.

Inside the uncompressed data:

• Read U32/big value count

Until the value count do:

• Skip U32/big

• Read byte ext term format marker (must be 131)

• Read tag byte, ... (follow the documentation)

Values are encoded using the external term format. A better reference is in the standard documentation

"ImpT" - Imports Table

Encodes functions from other modules invoked by the current module.

• Read U32/big count

Until the count do:

• Read U32/big module atom index, U32/big function atom index, U32/big arity

"LocT" - Local Functions

Essentially same as the export table format ExpT for local functions.

• Read U32/big count

Until the count do:

• Read U32/big func atom index, U32/big arity, U32/big location (label)

"Line" - Line Numbers Table

Encodes line numbers mapping to give better error reporting and code navigation for the program user.

• Read U32/big version (must match emulator's own version 0).

• Skip U32/big flags

• Read U32/big line_instr_count, U32/big num_line_refs, U32/big num_filenames

• Store invalid location const as Word[] linerefs first element which points at file #0, line 0.

• Set fname_index = 0, this is index in file name table, empty now

Until the num_line_refs do:

• Parse term at read position (see BEAM Compact Term Encoding)

• If the term is a small integer, push a pair of (fname_index, value) to the linerefs array.

• If the term is an atom, use its numeric value as new fname_index. Sanity check: value must be under num_filenames.

Until the num_filenames do (fill the file names table):

• Read U16/big name size

• Read string of bytes

• Convert string to an atom and push into file names table

"StrT" - Strings Table

This is a huge binary with all concatenated strings from the Erlang parsed AST (syntax tree). Everything {string, X} goes here. There are no size markers or separators between strings, so opcodes that need these values (e.g. bs_put_string) must provide an index and a string length to extract what they need out of this chunk.

Consider compiler application in standard library, files: beam_asm, beam_dict (record #asm{} field strings), and beam_disasm.

"Attr" - Attributes

Contains two parts: a proplist of module attributes, encoded as External Term Format, and a compiler info (options and version) encoded similarly.

Base and Extended Tag

Let's parse the value of tag:

• Read a byte and extract its least 3 bits. This is the base tag. It can be Literal=0, Integer=1, Atom=2, XRegister=3, YRegister=4, Label=5, Character=6, Extended=7.

• If the base tag was Extended=7, then bits 4-5-6-7 PLUS 7 will become the extended tag. It can have values Float=8, List=9, FloatReg=10, AllocList=11, Literal=12.

A badly written and incomplete GitHub example of reading signed word routine used to read signed words later:

A badly written and incomplete GitHub example of parsing a small integer: (used to read SmallInt values later).

Reading the Value

This is the logic, as was decoded from source code of BEAM VM and Ling VM. It looks at the bits in slightly different order.

• Look into the first byte read, bit 3:

  • Bit 3 is 1, so look into bit 4:

    • Bit is 1: Use remaining 3 bits of the byte as byte length (if under 7 - read `N+2` bytes into signed words, if the value is 7 - then length is larger than that and we have to read length first -- it follows as `?tag_u=0` (Literal) nested unsigned value)
    • Bit 4 is 0: use remaining 3 bits + 8 more bits of the following byte

  • Bit #3 = 0: Use remaining 4 bits

Now how to parse an encoded term:

• Read a SmallInt, case tag of:

  • Tag=Integer: use the value (signed?)

  • Tag=Literal: use smallint value as index in LitT table.

  • Tag=Atom: use smallint value MINUS 1 as index in the atom table. 0 smallint means NIL [].

  • Tag=Label: use as label index, or 0 means invalid value.

  • Tag=XRegister, Tag=YRegister: use as register index.

  • Tag=Character (a Unicode symbol): use val as unsigned.

  • Tag=Extended List: contains pairs of terms. Read smallint Size. Create tuple of Size, which will contain Size/2 values. For Size/2 do: read and parse a term (case of value), read a small int (label index), place them into the tuple.