Pack format
A project folder holds what a creator edits: game.toml, maps made in Tiled, OBJ
models, PNG pictures, text files, .pfs scripts, tracker modules, PBS data (see
project layout). The engine never reads any of that. It
reads one binary file, pack/game.pak, which ploom build makes.
project folder ──► converters (pl-assets) ─────────┐ ├──► pack/game.pak + pack/game.symPBS / Showdown ──► data model ──► compiler (pl-pbs) ┘Two tools write the one pack, and each keeps the other’s sections as it finds them,
its own after them, so the same sources give the same pack whichever ran last
(tools/pl-assets/src/pack.rs):
- The data compiler (
tools/pl-pbs) writes the creature data: types and the type chart, moves, abilities, items, species, trainers, encounters, shops and the identifier tables. It reads one data model (pl_pbs::model) that two front ends fill: the PBS parser of Essentials v21.1’s dialect (raw,parse,load), andtools/pl-showdown, which reads Showdown’s published data without executing it. The model, the compiler and the coverage report know nothing of either source. - The build (
tools/pl-assets/src/build.rs) runs the converters over the rest of the project and writes the game’s files (FILE, named byFNAM) and its tables of areas (AREA,ANAM,GAMEand their kin).
Why PBS and the other text formats exist only on the host: the DS never sees text it would have to parse. A custom source format would compile to the same records, so it would not run any better (plan 07).
The build is incremental
Section titled “The build is incremental”The build is split into units: one a map, one for the character sprites, one for
the interface files, one for the text, one for the scripts. A unit records the hash
of every input it read and the outputs it wrote in <project>/.ploom/build.state. A
unit whose inputs hash the same, and whose outputs are still in the pack, does not
run again: its files are taken from the pack as they are. A file is written only
when its bytes change. Maps, scenes and scripts convert in parallel, and
ploom build --maps builds only the maps named.
This is also what makes live reload cheap: the desktop watches the project
(crates/pl-desktop/src/reload.rs), rebuilds what changed into the pack and the
running game takes it.
The symbol file
Section titled “The symbol file”Names become numbers at build time. A script says giveitem POTION and its module
holds a number: the build compiles scripts against the names it knows from the game’s
data and the project’s symbol files. game.sym, written beside the pack, lists the
numbers the data compiler gave (species, moves, items and the rest) by name, for
tools and tests that want one. The pack itself carries no names it does not need.
The coverage report
Section titled “The coverage report”The compiler reports every move effect, ability and held item the data names, and
whether the battle engine has it. A move with no FunctionCode behaves as its own
identifier; an ability or item may name the engine behaviour it borrows with
Effect = identifier. With --release an unimplemented effect is an error unless
--allow names it. A game crate’s battle tables join here: the compiler gives the
game’s identifiers the game’s numbers and lists what it changes and adds (plan 34,
see custom mechanics).
Contents
Section titled “Contents”crates/pl-data/src/lib.rs documents the layout. One little-endian file, every
section on a 4-byte boundary:
| Bytes | Content |
|---|---|
| 0 | PFPK |
| 4 | format version (16 bits), section count (16 bits) |
| 8 | build hash: FNV-1a of everything from byte 16 to the end |
| 12 | length of the file |
| 16.. | the section table, 20 bytes a section: tag, kind, record size, count, offset, length |
A section is one of three kinds:
- a table of fixed-size records, addressed by number;
- a blob directory: offsets and lengths, then the blobs (files, strings);
- raw bytes that records point into (learnsets, the type chart).
Sections are named by four-letter tags (pl_data::tag): STRS the strings,
TYPE, TCHT, MOVE, ABIL, ITEM, SPEC and their pools, TRNR, ENCT,
SHOP, the identifier tables (MVID, the engine identifiers MVEI and so on),
FILE and FNAM the game’s files found by name hash, AREA and its companions,
GAME, WRLD the world table, and ENGV the battle engine’s content version.
Rules every record follows
Section titled “Rules every record follows”crates/pl-data/src/records.rs:
- A record is a whole number of 32-bit words, fields in order, no padding.
- No pointers and nothing of variable length. Text is a string number into
STRS(string 0 is empty). A list is an offset and a count into a raw pool of 16-bit values. - A field named after another table holds that table’s row; 0 means “none” for moves, abilities, items and species.
Why: a record can be read in place from any offset with one read_at, with no
parsing and no allocation, on a machine that reads its cartridge one seek at a time.
How it is read
Section titled “How it is read”Pack::open reads the header and the whole section table in one read and keeps only
that table in memory (20 bytes a section). Everything else is read through
Source::read_at when asked for: the game implements Source over the platform’s
read_at, and a byte slice implements it for tests. A pack is never loaded whole.
The battle is the clearest case. pl_data::battle::BattleData loads, before a battle
starts, only what it can need: the type chart and the records of the creatures taking
part, at most 12 species and 48 moves, about 1.5 KB with the chart. The battle
engine’s Dex trait cannot fail or read, so everything is loaded up front
(see the battle engine).
Versions
Section titled “Versions”Each file inside the pack has its own format, named by a magic whose last character
is its version: PFA5 an area, PFA6 a world, PFM2 a mesh, PFS1 a script module,
PFK2 a skin, PFG1 a tile-map map, and others (docs/registry.md, “File formats”,
and the file formats reference).
| What | Where the version lives | Now |
|---|---|---|
| The pack | pl_data::VERSION |
2 |
| Each file format | its magic | as above |
| The battle content | ENGV, compared with the engine’s |
refused if different |
| The save | pl_systems::save::VERSION |
6 |
Rules:
- A pack is always built with its runtime.
Pack::openrefuses another version (PackError::BadVersion), andpl_data::battle::check_enginerefuses battle data compiled for another engine (PackError::WrongEngine). The answer is always “build the pack again”, never a reader for the old format. - No legacy readers. A format change replaces the old reader outright
(decision 306). That holds for saves too:
pl_systems::saverefuses a save of an older version (DecodeError::OlderVersion). - New versions are given out before use. A change to what an existing section or
record means takes the next version from
docs/registry.mdfirst; a new optional section needs none. A test (xtask/tests/docs.rs) checks the registry’s versions against the code. - Tags are names. A new section adds a tag; nothing stores a tag’s position.
Policy
Section titled “Policy”sample/pack/ and demo/pack/ are not in git (decision 158). The project files they
are made from are. Reasons:
- A built file in git goes stale. Every change to a converter, a record or a map would change a binary that someone must remember to rebuild and commit, and two changes at once would conflict in a file nobody can merge by hand.
- Real data never enters the repository (hard rule 1). A creator’s pack holds data fetched from Showdown or imported from their own ROM; the build runs on their machine and writes only to git-ignored folders.
- The build is deterministic. The same sources give byte-identical packs, so building on demand loses nothing.
Every test and tool that needs a pack asks tools/pl-packs for it first. It builds
the sample’s or the demo’s pack the way ploom build does into the project’s
pack/ folder, quickly after the first time because the build is incremental. A lock
file in the project’s .ploom/ keeps two processes from building it at once.
Changing the format
Section titled “Changing the format”| To add | Touch |
|---|---|
| A field to a record | pl_data::records, the compiler or converter that writes it, the next pack version from the registry |
| A new section | a tag in pl_data::tag and the writer; no version change if optional |
| A new file format | a magic from the registry, a builder on the host and a reader in the crate that loads it |
| A new data source | a front end that fills pl_pbs::model; the compiler is unchanged |