Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Changes

3.2.0

With dayzmake-core 0.3.1.

  • New command group navmesh for navigation meshes of terrain mods: navmesh probe shows the world parameters (and one tile) of the navmesh data server of DayZDiag_x64.exe, navmesh generate builds the .nm file from that server or from a capture (--capture, --record; --range, --base/--skip-existing and --checkpoint to resume; --threads, --connections, several --server; --doors, --sea-level, --no-links, --no-island-filter, --seed), and navmesh info summarises a .nm file (--tiles, --json). The output is the same for any number of threads. New warnings navmesh-tile-failed, navmesh-stray-part and navmesh-seed. See the guide Navigation meshes for a terrain mod. dayzmake-core gets the module navmesh (feature navmesh, on by default). The MCP server gets the tools navmesh_probe, navmesh_generate and navmesh_info. --threads and --connections take at most 256 and refuse negative numbers (0 is the default). A tile that needs more than 2^26 heightfield spans fails instead of exhausting memory, --base must match the world’s grid and tile size, navmesh info shows the file version read (7 or 9), and the MCP sea_level takes a number or a string.
  • navmesh generate --capture does not load the whole capture: it indexes the file and reads only the tiles of the range (memory in proportion to the range, not to the file; a 3 x 3 range of a 50 GB capture ran in 6 s and 300 MB). dayzmake-core gets capture::CaptureFile and Source::CaptureFile; the capture format is unchanged.
  • navmesh generate: a data-server connection that is refused or reset is retried five times with 10 seconds of waiting (1, 2, 3 and 4 s between the tries), and when it still fails its tile goes back to the queue for the other connections instead of failing; a tile that fails on two working connections, or gets a wrong answer, is reported as not fetched without taking the connections down. --record may not name the --capture file. Only the tiles of --range are indexed from a capture (library: CaptureFileOptions; a run for a range wider than the indexed one is refused).

3.1.0

With dayzmake-core 0.3.0.

  • Parsing a config nested to the depth limit (128 levels) fits a 1 MiB stack again on Windows debug builds: the recursive grammar rules for classes and arrays have smaller frames (a debug build needs about 410 KiB for the deepest class nesting, it needed 920 KiB). The behaviour and the limits are unchanged (#108).
  • api config: the warning api-config-limit of a merge is raised once and names the PBO where the merge stopped; api-config-order names only the PBOs of the cycle, each as [<source>] <pbo>, and, like api-config-base-changed across PBOs, is raised on every query.
  • api config reads the includable text entries of a PBO only when it has a text config.cpp, skips a truncated or undecodable one with the new warning api-config-include-unreadable (the configs of the PBO still load, and an include of it names the read error), reads at most 64 MiB of them per PBO (new warning api-config-includes-capped), and resolves .. inside the PBO: #include "..\macros.hpp" from sub\config.cpp finds macros.hpp; a path that leaves the PBO’s root is still refused (#99).
  • PBO::insert_entry over an entry whose stored name is not UTF-8 (windows-1252) stores the raw bytes of the new name; it kept the old entry’s bytes (part of #98).
  • api config sorts the config PBOs of the game and of all selected mods together by requiredAddons, as the engine does (a game addon that requires a mod addon loads after it); ties go to the source (game, then the mods in -m order) and then to the path. -m all includes mods that have only a config. The api-config-base-changed warning shows the class path as written (CfgVehicles/Car_Base), the game is recognised by a flag and no longer by the name vanilla, and an infinite float is shown as 1e999 / -1e999 as derapify writes it (a string in --json) (part of #98).
  • extract refuses a new target folder inside the source also without -f, before it creates anything (it used to write into the mod folder, and the -f re-run was then refused). A file/folder clash across PBOs (p/a in one, p/a/b in another) no longer fails the extract: the first entry in the sorted PBO order is written, the later one is skipped with the new warning extract-path-clash (part of #98).
  • The extract-path-clash warning names the skipped entry, says whether it is a file or a folder and what the other PBO has at that path, with / separators on every platform; an extract source that cannot be resolved is an error of its own (part of #104).
  • api grep prints at most the first 512 bytes of a matching line (counted before escaping), cut at a character boundary and followed by […]; --json cuts text the same way and adds "truncated": true to cut hits. The MCP api_source refuses target together with context or lines instead of ignoring it, and also target without cat (error target is only used with cat; it did nothing there) (part of #98, #104).
  • inspect of an ODOL v56 model leaves out the Class: and Damage: lines when they are empty and a section (Textures, Materials, Proxies, Properties, Selections, Animations) that has no items; an empty material name was already printed as (none) (#96).
  • A config value entry (not an array or a class) may end at the line end instead of ;, as in CfgConvert; it keeps its value type, as in CfgConvert: x = 1.5 is a float, no longer the string "1.5", and x = 1e999 is an infinity (see below). At the very end of a file without a line end such an entry is a syntax error, as in CfgConvert (#95). An integer literal that overflows at a line end is handled as before a ; (see the next bullets).
  • The preprocessor strips a // comment at the very end of the input, with no line end after it, as CfgConvert does; before, x = 1; followed by // c without a final newline was a syntax error. An unterminated /* runs to the end of the file, also as in CfgConvert (silently there), and raises the new warning unterminated-comment with file and line; an included file’s comment ends with that file. When the last line of a config is picked out to decide whether its entry may end at the end of the file, a lone CR now counts as a line break; CfgConvert rejects x = 1\r#define Y 1 and x = 1\r// c, dayzmake accepts them (#103).
  • A config float literal that overflows f32 (1e999, -1e39) is rapified as an infinity, no longer an error, as CfgConvert does; a literal that underflows is 0. derapify writes an infinity as 1e999 / -1e999, so derapify then rapify keeps the bits (#102).
  • A decimal integer literal that does not fit 64 bits (99999999999999999999) is a float, as in CfgConvert (x = 1e20), at a line end and before a ;; in an array every decimal literal that does not fit 32 bits is a float ({3000000000} is 3000000000.0). It was an error. A decimal literal that fits 64 bits but not 32 is a 64-bit integer entry (x = 2147483648, entry subtype 6), as CfgConvert writes it; derapify and api config read it (they rejected subtype 6, Unrecognized variable entry subtype: 6). Followed by text it stays an unquoted string. Two other inputs were checked against CfgConvert and need no change: x = 1 // c and x = 2 \ + a comment line, both without a final newline, are syntax errors in both; an included file whose last line has no line end is accepted by both (CfgConvert glues its own #line marker to the value, dayzmake does not) (part of #104).
  • Config enum values are computed like CfgConvert, in f32 (#107): a literal is rounded to f32 and a value outside i32 is -2147483648 (A = 2147483647, 2147483648, 99999999999999999999 and -2147483649 all give it), hex keeps its low 32 bits at any length (0x1FFFFFFFFF is -1) and 0X works, and an implicit value is the last explicit value plus its distance, added in f32 (A = 16777217, B gives 16777216 twice; A = 16777216, B, C gives 16777216, 16777216, 16777218). Such values were an error. __EVAL of large numbers keeps its exact floats; CfgConvert’s imprecise low bits (__EVAL(2^31) is 0x4effffe4 there) are not copied.
  • A hex config literal does what CfgConvert does: without a sign it keeps its low 32 bits at any length (0x1FFFFFFFFF is -1, 0x100000000 is 0; it was an error), 0X works like 0x, and with a sign it is a float (-0x10 is -16.0, it was the integer -16) (#106).
  • A config syntax error no longer brings the unquoted-string warning that the failed parse raised on the error’s own line (x = 1 // c was tried as an unquoted string first); warnings from earlier lines are kept (part of #104).
  • Config errors raised while parsing a value (a failing __EVAL/__EXEC, nesting too deep) carry the In line <file>:<line>: context of a parse error (#94).
  • api config <path> looks up a config class such as CfgVehicles/Hatchback_02: its inheritance chain, its own and inherited properties each tagged with the class they come from, the classes inside it, its subclasses and the PBOs that define it. It indexes every config.bin and config.cpp of the dta and addons PBOs of the game (also the ones in folders of a PBO) and of the -m mods, loads the PBOs of all sources by CfgPatches requiredAddons (ties: vanilla, the mods in order, sorted path), and merges like the engine: a class defined again extends the earlier one, properties overwrite, the base class becomes what the last definition names (none included; the lookup prints Base changed: A -> (none) by [<source>] <pbo> and a mod’s change raises api-config-base-changed), delete removes, += appends, class X; creates nothing. --max cuts the output, --json prints an object, --refresh rebuilds the cached index of each source. An inheritance cycle is an error. An #include in a mod config.cpp is resolved from the entries of the same PBO only, never from the file system (#97). The MCP tool is api_config; new warnings api-config-order, api-config-base-changed and api-config-limit (#89).
  • extract [-f] [--derapify] [--only <pattern>]... [-x <pattern>]... <source> <targetfolder> unpacks every PBO in the dta and addons folders of a DayZ install or mod, or a single PBO, into <targetfolder>/<prefix>/, optionally derapifying the rapified .bin files to .cpp, and writes pbo-map.tsv (prefix with / separators, TAB, PBO path; \, TAB, CR and LF are escaped as \\, \t, \r, \n). The tree is written to a staging folder and swapped in, so a re-run drops stale files and a failure keeps the old tree; -f only replaces an empty folder or an earlier result (one with a pbo-map.tsv) and never one that contains or lies inside the source. Entries are streamed, not read whole, and an I/O or decompression error fails the extract. The MCP tool is extract (#87).
  • api grep <regex> searches the lines of every .c file of vanilla and of the selected mods, straight from the PBOs, and prints [<source>] <prefix>\<path>:<line>: <text> sorted by source, path and line. --ignore-case matches case-insensitively, --max cuts the output and bounds the memory used (all further matches are still counted for the ... N more lines note) and --json prints an array of {source, path, line, text}. The MCP tool is api_grep (#88).
  • api src --context <n> prints n lines around each declaration and --lines <a-b> lines a to b of each file, as <line>| <text> (with --json: "lines"); the options exclude each other and --cat. The MCP tool api_source takes context and lines (#90).
  • unpack takes -x <pattern> (leave entries out) and --only <pattern> (write only matching entries); -x wins. The MCP tool pbo_unpack has the arguments exclude and only. UnpackOptions gains exclude and only, and dayzmake_core::pbo::EntryFilter is the shared filter (#86).
  • inspect --json <file.pbo> prints the PBO header as one JSON object: the header extensions, the prefix and the entries, each name next to the lowercase hex of its raw bytes, so a prefix with a control character or a byte that is not UTF-8 is unambiguous. The MCP tool pbo_inspect takes the argument json. A model gives an error (#91).
  • Writing and signing a PBO keep the stored bytes of header extensions and entry names that a UTF-8 and a windows-1252 spelling would otherwise swap (#92).
  • Config values in exponent notation (1e-5, 1.5E+3, 3.162277e-5) are rapified as floats (#85), not as strings with an unquoted-string warning, so derapify then rapify keeps a float a float. Values such as 1e5x or 1e stay strings. A float that does not fit an f32 (1e999, -1e39) becomes an f32 infinity, as in CfgConvert (#102); followed by text (1e999x) it stays an unquoted string, like an integer that does not fit (99999999999x, #93), and a float entry may end at a closing brace without a semicolon.
  • inspect and pbo_inspect list ODOL v56 models (#84): skeleton, bounding box, LODs with vertex and face counts, textures, materials, proxies, properties, selections and animations. Other ODOL versions still show the format and version, with a note that only v56 is decoded. A model whose LOD ranges overlap is refused.
  • PAA reads ARGB4444, ARGB1555 and AI88 (#83): paa2img and paa_to_image decode the 19 such textures the game ships (LZSS mipmaps with a checksum); a wrong checksum or a cut-short stream is an error. A PNG file named .paa is reported as a PNG file.
  • Config enum blocks (#82): rapify accepts enum { A, B = 5, C }; at any class level (values count up from the last explicit one, as in C) and writes the names, lowercase, into the enum table of the config.bin; derapify prints the table as an enum { name = value, ... }; block at the top. A table that is missing or damaged gives no block and no error.

3.0.2

With dayzmake-core 0.2.2.

  • PAA fixes in dayzmake-core: Paa remembers per mipmap whether it is LZO-compressed, so Paa::write no longer flags a short raw mipmap as compressed (#76) and Paa::to_dynamic_image decodes a Paa built by Paa::from_dynamic_image or Paa::lzo_compress without a write/read round trip (#78). Paa::read drops the GGATSFFO offset tagg, so a re-written PAA has it once (#75); Paa::tagg("GGATSFFO") now returns None after read.
  • Paa::write and img2paa reject a mipmap of 16 MiB or more (its length field has 3 bytes), for example RGBA 2048×2048 and larger or a DXT5 4096×4096 texture that LZO cannot shrink, instead of writing a wrong length, and Paa::read keeps an LZO-compressed mipmap compressed, so a read and re-written PAA stays readable (#79).
  • dayzmake cat accepts -f to overwrite its target; the “Use –force” error was already printed for it (#71).
  • img2paa -c and the MCP tool image_to_paa (compress) LZO-compress DXT mipmaps wider than 128 pixels (#70). dayzmake already did that for every conversion, so the output does not change. New in dayzmake-core: Paa::lzo_compress.
  • derapify -d and the MCP tool config_derapify (indent) set the indentation: spaces and tabs only, empty allowed (#69). New in dayzmake-core: config::WriteOptions and Config::write_with.

3.0.1

  • Releases carry SHA256SUMS and SHA512SUMS of all archives with GPG signatures, and the public key (dayzmake-release-key.asc); see Verifying downloads.
  • The documentation is published as a web site with guides and a reference of every command, option, MCP tool and warning: https://bzed.github.io/dayzmake/.
  • pack no longer warns that models were not binarized; only build does (#72, fixed in dayzmake-core 0.2.1).

3.0.0

The changes since 2.0.0. Breaking: a missing #include no longer searches the working directory (it is only checked directly; pass -i for folders to search), stricter key and PBO checks (see below), and new errors on hostile config input. dayzmake 3.0.0 needs dayzmake-core 0.2.0.

  • Backslash line continuations count as source lines: in a #define and in disabled lines they used to be lost, and a macro use no longer shifts the line numbers after it, so warnings and errors point at the right line (#65).
  • Behaviour change: a missing absolute include no longer walks the whole working directory (#66): the implicit working directory is only checked directly (./DZ/data/x.hpp), so a file buried below it is no longer found without -i. Folders given with -i are still searched recursively, but the search does not follow directory symlinks that leave the folder, and enters no directory twice, and it fails with an error naming the include after 1,000,000 visited entries (the author’s P: drive copy has 109,000).
  • Line numbers in warnings and errors stay right after a disabled #ifdef/#ifndef branch (plain lines, #define, #undef, #include, nested branches); they used to drift up by one per disabled line (#63).
  • Macro expansion also charges its work: every call of a defined macro costs, so an exponential chain whose expansion is empty (#define E(x) with E(A)E(A) levels) fails like the others; at most 100,000 macro calls per line, real configs need at most 34 (#59).
  • Macro work is capped per file too: at most 100,000 macro calls over a file and its includes (a 3000-line file of A13 lines, each about 16,000 calls with an empty expansion, took 37 s); real configs under the author’s trees need at most 174 (#64).
  • At most 10,000 #includes are processed per preprocess call (nested ones and repeated inclusions count), so an exponential include fan-out fails at once; real configs under the author’s trees use at most 1 (#60).
  • Lines in a disabled #ifdef/#ifndef branch are skipped before any macro expansion, so they cannot raise macro errors or cost time (#61).
  • All Windows device names are recognised in PBO entry names and output targets: COM0-COM9 and LPT0-LPT9 also with the superscript digits (COM¹), CONIN$ and CONOUT$, in any case, with or without an extension (#62).
  • pack and build recognise their own output by file identity (device and inode, or volume and file index), so M\OUT.PBO over an existing m\out.pbo on a case-insensitive file system is never packed into itself (#37).
  • An existing regular file with a device-like name (NUL, COM1, reached through a verbatim path) is not overwritten in place and needs --force like any file (#40); a new target with such a name can still be created (#45).
  • unpack warns about a leading separator only for entries it extracts, and build’s duplicate-entry warning names the stored entry (config.bin, m.p3d) it kept (#41).
  • A closed stderr no longer panics: usage and other errors still end with their normal exit status (1, or 0 with only warnings) instead of 101 (#43).
  • Macro expansion is capped: a source line that makes expansion copy more than 500,000 tokens, or a file (with its includes) more than 16,000,000, is an error naming the macro, so an exponential definition chain (billion laughs) fails at once instead of exhausting memory (#57). The counters measure copying work, re-counted at each expansion level, not the size of the output. The most that the author’s DayZ trees (1596 configs) need is 888 tokens per line and 749,577 per file.
  • The same cap also counts bytes (64 MiB per line, 512 MiB per file), so that #x stringification, which collapses a large expansion into one token, cannot bypass it (#58). Real configs need at most 12 KB per line and 7.3 MB per file.
  • Nesting deeper than 128 levels (macro argument parentheses, config classes, config arrays, __EVAL parentheses and operator chains) is a parse error, not a stack overflow (#54). Parentheses inside macro arguments count even in strings, as the preprocessor reads them.
  • An include cycle (#include of a file already being included) or include nesting deeper than 32 is an error naming the chain, not a stack overflow (#53).
  • A closed stdout pipe (dayzmake inspect x.pbo | head -3) ends the command quietly with exit status 0 and no error line (it used to print Broken pipe and exit 1, or panic in api). A closed pipe on stderr, a target file or an external tool is still an error (#39).
  • OpenSSL is gone from the dependency tree, tests included (they use num-bigint as the independent check): building and testing need no system libraries, and the CI and the Dockerfile no longer install libssl-dev or pkg-config. dayzmake keygen generates the RSA key pair with the pure-Rust rsa crate and the operating system’s random number generator, and signing and verification run on rsa too (blinded, constant-time private-key operation; the result is checked against the public key before it is written). Key files and .bisign output are byte-identical to before. A signature value that is not smaller than the key’s modulus is a verification error. Our copies of key components read from a key file are wiped from memory when dropped (best effort: the rsa crate makes internal copies it does not wipe).
  • Two misleading errors are fixed: a .bisign with a bad header says “Invalid signature file” (not “public key”), and verifying with a key other than the one that made the signature says “The signature was made with a different key” (names and lengths only) instead of a complaint about the modulus (#56).
  • sign and key generation refuse keys below 384 bits, the CryptoAPI minimum BI’s tools and the game need (smaller keys could only sign with broken padding). Reading such keys and verifying their signatures still works (#55).
  • A .bisign file is read with the same checks as a public key: its embedded key must carry the RSA1 blob header and a modulus exactly as wide as the header length says; anything else is a read error (#52).
  • Macro expansion has a depth limit on every path (64 levels); a config that nests deeper, or expands a macro into itself through its arguments, is an error naming the macro instead of a stack overflow (#51).
  • Key files must carry the CryptoAPI blob header (type, key algorithm, RSA1/RSA2 magic) that dayzmake and BI’s tools write; any other header is a read error (#50).
  • Keys and .bisign files are limited to 16384 bits (288 at the low end); a larger header length is a read error (#49).
  • A key file whose modulus is not exactly as wide as its header length says, and a private key whose length is not a multiple of 16, is rejected when read (#48).
  • verify and sign recompute the SHA-1 of the PBO instead of trusting its stored trailer: a PBO whose bytes changed (config.bin, files a signature version leaves out, the header) no longer verifies, and a PBO with a wrong trailer is not signed. Both fail with a “checksum mismatch” error (#46); repack the PBO (dayzmake pack/build) before signing. BI’s DSCheckSignatures trusts the stored trailer, dayzmake verify does not. inspect and unpack still read such files.
  • dayzmake cat on an entry that is not in the PBO now exits 1 with an error (it used to exit 0 and print not found).
  • x = ; (or a value macro that expands to nothing) now raises the empty-value warning; the value is still stored as "" (#19).
  • build . (and a symlinked addon folder) without $PBOPREFIX$ uses the folder’s name as the prefix instead of failing with “Invalid addon directory” (#11).
  • Preprocessor parse errors report 1-based line numbers (#12).
  • Unpacking a PBO skips entry names with any control character (C1 included), .. or drive-letter components or an empty component (unsafe-path warning); a leading separator is stripped instead (\scripts\x.c is written as scripts/x.c, leading-separator warning). Modifying a PBO rejects such names. The API index (api) is more lenient: an entry with a leading or trailing separator is indexed under its normalised name instead of being skipped (#13, #21).
  • Rewriting a read PBO keeps the packing method of entries it does not know (#14).
  • Names, paths and values from untrusted files are shown with control characters escaped in errors, warnings and inspect listings. derapify refuses a class or entry name that cannot be written as config text (#15, #22, #26, #28).
  • Writing a P3D rejects an inconsistent model (bad selection size, vertex index out of range, a face without 3 or 4 vertices, a NUL in a name) before the first byte is written; the error names the LOD index (#16).
  • A string with an embedded NUL fails rapifying with an error naming the config entry, instead of writing a corrupt file (#23).
  • Proton binarizing rejects a $PBOPREFIX$ that would leave its sandbox (#17).
  • Output files (build, pack, rapify, binarize, keygen, …) are replaced atomically: a failed command keeps the old target, an existing target keeps its permissions (with --force a read-only target is replaced on unix and stays read-only; Windows refuses it and leaves it intact), a symlink is written through, and a device, FIFO or pipe (/dev/null, /dev/stdout, and on Windows NUL, CON, …) is written in place. A failure to create the temporary file names the directory (#18, #25).
  • build never packs its own output: the target and its temporary file are left out when they lie inside the source folder (#24). The same holds for pack and build with stdout redirected into the source folder (dayzmake pack m > m/out.pbo) (#38).
  • The api warning for a mod with an unsafe prefix shows the prefix escaped once, not twice (#31).
  • keygen creates the private key readable by its owner only (0600) (#20).
  • build and pack skip FIFOs, devices and sockets in the source folder (unsupported-file warning, no more hanging on a FIFO) and files whose names are unsafe as PBO entry names (unsafe-path warning); -x excludes apply first. Names that collide (case-insensitively, for example config.cpp and config.bin) give a duplicate-entry warning; the first in sorted order wins, except that an entry generated from source (the rapified config.cpp, a .p3do packed as .p3d) wins over a plain config.bin or .p3d of the same name (#35). An unreadable source file, such as a dangling symlink, is named in the error (#27, #30, #32, #33, #35).
  • build matches the model extensions .p3d, .rtm and .p3do case-insensitively (N.P3D is binarized like n.p3d; xp3d is not a model) (#34).