Skip to content

Pack format

A project folder holds what a creator edits: game.toml, maps made in Tiled, OBJ models, PNG pictures, text files, .pfs scripts, tracker modules, PBS data (see project layout). The engine never reads any of that. It reads one binary file, pack/game.pak, which ploom build makes.

project folder ──► converters (pl-assets) ─────────┐
├──► pack/game.pak + pack/game.sym
PBS / Showdown ──► data model ──► compiler (pl-pbs) ┘

Two tools write the one pack, and each keeps the other’s sections as it finds them, its own after them, so the same sources give the same pack whichever ran last (tools/pl-assets/src/pack.rs):

  • The data compiler (tools/pl-pbs) writes the creature data: types and the type chart, moves, abilities, items, species, trainers, encounters, shops and the identifier tables. It reads one data model (pl_pbs::model) that two front ends fill: the PBS parser of Essentials v21.1’s dialect (raw, parse, load), and tools/pl-showdown, which reads Showdown’s published data without executing it. The model, the compiler and the coverage report know nothing of either source.
  • The build (tools/pl-assets/src/build.rs) runs the converters over the rest of the project and writes the game’s files (FILE, named by FNAM) and its tables of areas (AREA, ANAM, GAME and their kin).

Why PBS and the other text formats exist only on the host: the DS never sees text it would have to parse. A custom source format would compile to the same records, so it would not run any better (plan 07).

The build is split into units: one a map, one for the character sprites, one for the interface files, one for the text, one for the scripts. A unit records the hash of every input it read and the outputs it wrote in <project>/.ploom/build.state. A unit whose inputs hash the same, and whose outputs are still in the pack, does not run again: its files are taken from the pack as they are. A file is written only when its bytes change. Maps, scenes and scripts convert in parallel, and ploom build --maps builds only the maps named.

This is also what makes live reload cheap: the desktop watches the project (crates/pl-desktop/src/reload.rs), rebuilds what changed into the pack and the running game takes it.

Names become numbers at build time. A script says giveitem POTION and its module holds a number: the build compiles scripts against the names it knows from the game’s data and the project’s symbol files. game.sym, written beside the pack, lists the numbers the data compiler gave (species, moves, items and the rest) by name, for tools and tests that want one. The pack itself carries no names it does not need.

The compiler reports every move effect, ability and held item the data names, and whether the battle engine has it. A move with no FunctionCode behaves as its own identifier; an ability or item may name the engine behaviour it borrows with Effect = identifier. With --release an unimplemented effect is an error unless --allow names it. A game crate’s battle tables join here: the compiler gives the game’s identifiers the game’s numbers and lists what it changes and adds (plan 34, see custom mechanics).

crates/pl-data/src/lib.rs documents the layout. One little-endian file, every section on a 4-byte boundary:

Bytes Content
0 PFPK
4 format version (16 bits), section count (16 bits)
8 build hash: FNV-1a of everything from byte 16 to the end
12 length of the file
16.. the section table, 20 bytes a section: tag, kind, record size, count, offset, length

A section is one of three kinds:

  • a table of fixed-size records, addressed by number;
  • a blob directory: offsets and lengths, then the blobs (files, strings);
  • raw bytes that records point into (learnsets, the type chart).

Sections are named by four-letter tags (pl_data::tag): STRS the strings, TYPE, TCHT, MOVE, ABIL, ITEM, SPEC and their pools, TRNR, ENCT, SHOP, the identifier tables (MVID, the engine identifiers MVEI and so on), FILE and FNAM the game’s files found by name hash, AREA and its companions, GAME, WRLD the world table, and ENGV the battle engine’s content version.

crates/pl-data/src/records.rs:

  • A record is a whole number of 32-bit words, fields in order, no padding.
  • No pointers and nothing of variable length. Text is a string number into STRS (string 0 is empty). A list is an offset and a count into a raw pool of 16-bit values.
  • A field named after another table holds that table’s row; 0 means “none” for moves, abilities, items and species.

Why: a record can be read in place from any offset with one read_at, with no parsing and no allocation, on a machine that reads its cartridge one seek at a time.

Pack::open reads the header and the whole section table in one read and keeps only that table in memory (20 bytes a section). Everything else is read through Source::read_at when asked for: the game implements Source over the platform’s read_at, and a byte slice implements it for tests. A pack is never loaded whole.

The battle is the clearest case. pl_data::battle::BattleData loads, before a battle starts, only what it can need: the type chart and the records of the creatures taking part, at most 12 species and 48 moves, about 1.5 KB with the chart. The battle engine’s Dex trait cannot fail or read, so everything is loaded up front (see the battle engine).

Each file inside the pack has its own format, named by a magic whose last character is its version: PFA5 an area, PFA6 a world, PFM2 a mesh, PFS1 a script module, PFK2 a skin, PFG1 a tile-map map, and others (docs/registry.md, “File formats”, and the file formats reference).

What Where the version lives Now
The pack pl_data::VERSION 2
Each file format its magic as above
The battle content ENGV, compared with the engine’s refused if different
The save pl_systems::save::VERSION 6

Rules:

  • A pack is always built with its runtime. Pack::open refuses another version (PackError::BadVersion), and pl_data::battle::check_engine refuses battle data compiled for another engine (PackError::WrongEngine). The answer is always “build the pack again”, never a reader for the old format.
  • No legacy readers. A format change replaces the old reader outright (decision 306). That holds for saves too: pl_systems::save refuses a save of an older version (DecodeError::OlderVersion).
  • New versions are given out before use. A change to what an existing section or record means takes the next version from docs/registry.md first; a new optional section needs none. A test (xtask/tests/docs.rs) checks the registry’s versions against the code.
  • Tags are names. A new section adds a tag; nothing stores a tag’s position.

sample/pack/ and demo/pack/ are not in git (decision 158). The project files they are made from are. Reasons:

  • A built file in git goes stale. Every change to a converter, a record or a map would change a binary that someone must remember to rebuild and commit, and two changes at once would conflict in a file nobody can merge by hand.
  • Real data never enters the repository (hard rule 1). A creator’s pack holds data fetched from Showdown or imported from their own ROM; the build runs on their machine and writes only to git-ignored folders.
  • The build is deterministic. The same sources give byte-identical packs, so building on demand loses nothing.

Every test and tool that needs a pack asks tools/pl-packs for it first. It builds the sample’s or the demo’s pack the way ploom build does into the project’s pack/ folder, quickly after the first time because the build is incremental. A lock file in the project’s .ploom/ keeps two processes from building it at once.

To add Touch
A field to a record pl_data::records, the compiler or converter that writes it, the next pack version from the registry
A new section a tag in pl_data::tag and the writer; no version change if optional
A new file format a magic from the registry, a builder on the host and a reader in the crate that loads it
A new data source a front end that fills pl_pbs::model; the compiler is unchanged