Skip to content

WC3_RUNTIME.md

Latest commit

 

History

History
69 lines (49 loc) · 20.7 KB

WC3_RUNTIME.md

File metadata and controls

69 lines (49 loc) · 20.7 KB
Raw

Warcraft III runtime facts for compiler work

What the game does that the compiler, the optimiser and the interpreter mocks have to respect. The game is the specification. The places a reader would look instead (Blizzard.j, the IM interpreter, the Lua test runtime) each differ from it, so every fact here says how it is known:

  • measured: observed in the real client, version given. Do not overturn one by reading code; measure again.
  • code: follows from what this repository emits or does; the pointer is the evidence.
  • reported: community documentation, not measured here. Likely, not proven.

The rules that follow are in AGENTS.md §7 and §11.

Three places code runs, one of them is the game

IM interpreter (compile time, executeProg()) Test Lua (testLua(true).executeProg()) The game
Reals Java float, round to nearest (code: ILconstReal) Lua doubles (a stock Lua: 5.3 on Windows and the Linux CI, unchecked elsewhere) Jass: 32-bit, literals not correctly rounded. Lua: 24-bit mantissa, arithmetic truncates (measured, below)
Lua libraries none every standard library of that interpreter; os.exit() is how testSuccess ends a run (code: LuaNatives) Lua 5.3 without some of them (reported, below)
Natives mocks in wurstio/jassinterpreter/providers wc3shim.lua plus the Reforged common.j/blizzard.j Lua dumps (src/test/resources/luaruntime/README.md) the engine

A test that passes in the first two columns says nothing about the third where they differ. The natives LuaNatives defines for a Jass native name (such as GetRandomInt) are fallbacks: the script defines one only if the game has not, so the real implementation wins (LuaTranslator.translateFunc). The __wurst_-prefixed ones are Wurst's own, never pre-defined by the game, and emitted without that guard.

Objects and memory

  • code Wurst has no garbage collector. A class instance is an integer id on both targets. new takes an id, destroy runs ondestroy and gives the id back, and a second destroy is an error (Double free: object of type X on Jass, Double free or invalid Wurst object. on Lua). A closure is a class instance too and keeps its id until destroyed. (RecycleCodeGeneratorQueue, LuaTranslator.createObjectManagement.)

  • code Both targets reuse the most recently freed id first (a stack, despite the name RecycleCodeGeneratorQueue), so destroy followed by new from the same allocator returns the same id and a stale reference then reads the new object. On Jass each group of classes joined by inheritance (a class with its superClasses, ImTranslator.buildClassPartitions) has one allocator: a subclass, its siblings and the base class share ids and the cap, so a stale id may come back as an object of a related class. The group runs out at JASS_MAX_ARRAY_SIZE live instances (default Constants.MAX_ARRAY_SIZE, 32768) with Out of memory: Could not create X.. On Lua there is one id space shared by every class and no cap, so a stale id may come back as an object of any class.

  • code Lua keeps one static table per field, indexed by id, and a class descriptor per live id. destroy removes the descriptor and recycles the id; field storage is not cleared, as Jass arrays are not. Until the id is reused a stale reference fails virtual dispatch and instanceof is false. Lua's collector therefore frees no Wurst object, and a table stored in a field stays reachable after destroy.

  • code An object that gets a reused id starts differently on each target, by design. On Jass a field without an initialiser keeps what the destroyed object left there, an array field's elements included: nothing resets the field's array at new, and doing it would add a store per field to every allocation. On Lua new writes the default of every field and gives each array field a fresh table (LuaFieldDefaults, LuaTranslator.createClassInitFunction), and in compile-time code the interpreter makes a fresh object. A field with an initialiser starts from it on every target. Resetting on Jass would change what existing maps read, so the difference stays; code which needs a known value writes it in the constructor or gives the field an initialiser. LuaTranslationTests.reusedObject* pins the three.

  • code The natives of the standard library's Reflection read this bookkeeping. A type id numbers a class from 1 (TypeId: by package, name and source position, each class followed by its subclasses); when compile-time code read a type id, the classes keep the ids they had then and Jass numbers the classes the generics add after them (TypeId.keeping), so an id stored at compile time stays its class's; maxTypeId is the largest and typeIdToTypeName the class's qualified name. instanceCount is the number of live objects of the class's allocator group, not of the class alone, and maxInstanceCount the most the group had alive at once, which on Jass is the group's highest id (EliminateClasses). Lua has no group allocator, so it keeps a live count and a peak per group, updated by the allocations and destroys of the group's classes, only in a program which calls these natives and only for the groups it can ask about (LuaReflection). Lua numbers the classes the emitted program keeps, except the copies a targeted specialisation makes, which have the type id and the group of the class they copy (LuaTranslator.getTypeId); a program which can see the numbering (maxTypeId, or a type id which is not C.typeId or o.typeId) keeps every class, so its numbers are Jass's. A number which is no type id answers "unknown" and 0 on both targets and in the interpreter. Two differences remain, both pinned in LuaTranslationTests: the Jass replay of compile-time objects sets the group's highest id to the highest one used at compile time, so the ids of the objects destroyed or dropped then count as live there, while Lua counts only the objects it makes again (compile-time code numbers a new-generic class's objects as the erased class's, so each instantiation's group continues after the highest id of them all: CompiletimeFunctionRunner.emitJassObjectAllocs); and a new-generic class is a class per instantiation on Jass but one erased class on Lua (AGENTS.md §9), so with Box<T:> and Sub<T:> extends Box<T>, Box<int>, Sub<int> and Box<string> are type ids 1, 2 and 3 in two groups on Jass, while Lua has the type ids 1 and 2 in one group which counts all three objects. Matching that would take a class per instantiation on Lua, or the type arguments at every allocation, which erased generic code does not have. The interpreter counts each allocator group's live objects and their peak as the targets do.

  • measured, 3.0.0 Hashtables: a handle load is typed (a saved unit loads as itself through LoadUnitHandle and LoadWidgetHandle, as null through LoadItemHandle or LoadTimerHandle); saving a null handle stores nothing and keeps what was there; FlushParentHashtable destroys the hashtable, so nothing saved to it afterwards stays and every load answers its default; a null hashtable answers every load with its default and ignores every write. The Lua emulation (LuaHashtable) and the interpreter (HashtableProvider) answer the same. A handle saved through a supertype (SaveAgentHandle, SaveWidgetHandle) and loaded as a subtype is answered on Lua, which cannot tell its type.

  • measured, 3.0.0 A null enum handle (a type which a common.j ConvertX(integer) native makes, such as unitstate, unittype, playerstate, attacktype, damagetype) is the handle of id 0 to the engine: GetUnitState(u, null) is the life, IsUnitType(hero, null) is true (UNIT_TYPE_HERO), GetPlayerState(p, null) is the game result, SetPlayerState(p, null, v) writes it, and UnitDamageTarget with a null attack, damage and weapon type deals the damage. The Lua backend's nil guard of a handle native therefore passes ConvertX(0) for a nil enum argument and answers the native's default only for any other nil handle; a blizzard.j helper is called directly and its body decides how to handle the null, as Jass does until a native reads it (LuaNativeLowering.createNilSafeWrapper, ImTranslator.enumConverter). The interpreter's mocks of IsUnitType, GetUnitState, GetPlayerState and SetPlayerState read a null enum as id 0 too, and the game result is 3 (PLAYER_GAME_RESULT_NEUTRAL) while the game runs.

  • measured, 3.0.0 Null strings: Jass keeps null and "" apart (null == "" is false; an unset string array element, field or global is null; SubString past the end, any SubString of "" and StringCase("") answer null; StringLength, S2I, S2R and StringHash of null answer 0, as of ""). The game's Lua natives answer "" in each of those places, and every string native probed raises on a nil argument ("string expected, got nil": StringLength, SubString, StringCase, S2I, S2R, StringHash, GetLocalizedString, BlzSetUnitName, SetTextTagText, BlzFrameSetText, DisplayTextToPlayer). So on Lua the null string is "", as the game's natives treat it (EliminateLocalTypes), and nothing the compiler emits answers nil for a string: the emulated LoadStr answers "" when nothing is stored. null == "" is therefore true on Lua and false on Jass; matching Jass would mean wrapping every native which returns a string. In Jass, BlzSetUnitName(u, null) crashes the client.

Jass function parameters

  • measured, 3.0.0.24268 (2026-10-13) Raw Jass calls with 30 and 31 integer parameters, and with 30 and 31 string parameters, completed and verified every argument against a runtime-derived value. A 31-parameter signature consisting of an integer receiver, 29 strings and a string trace position also completed. Calling the 31-string function during package initialization and again after loading completed too. Separate maps with 32 integer or 32 string parameters initialized, then the client process exited without reporting a result. These measurements support the compiler's 31-parameter limit; they do not reproduce issue #1445's reported crash at 31, or establish another patch's limit.
  • code The bundled pjass accepts those 31-argument calls and rejects a 32-argument call with "A function call can have at most 31 arguments". Compiler checks must count emitted parameters, including receivers, flattened tuple components and injected stacktrace positions (ImHelper.JASS_MAX_PARAMETERS, VarargEliminator, CyclicFunctionRemover). New generics avoid old-generic conversion overhead but do not remove this Jass call limit.

The game's Lua

  • reported It is Lua 5.3, with debug disabled (Hive guide) and collectgarbage inaccessible since a patch, "it could cause desync" (forum post). The repository already treats debug.traceback as unavailable (LuaTranslationTests.luaErrorWrappersAvoidUnavailableNativeTraceback). Nothing in emitted code needs io, os, package or require; they are not known to exist there. LuaTranslationTests.emittedLuaUsesOnlyLibrariesTheGameProvides fails if a program's emitted helpers reach for any of these. noLuaBackendLiteralNamesALibraryTheGameWithholds scans the string literals under translation/ (bar two files that only list names), because natives and fallback helpers are emitted only on demand. A name counts anywhere in the text, inside a Lua string or comment too, since a string can be run with load. Only the test-only testSuccess is exempt: it ends a test run with os.exit.
  • code debug.traceback is not used (above), so -stacktraces keeps its own call stack in the script (StackTraceInjector2: wurst_stack, wurst_stack_depth). Each call of an instrumented function pays for it: an extra position argument at the call site, a push and a pop in the callee. main and config are skipped. A generated wurst_run.args turns on -stacktraces for runs and builds and -inline only for builds (WurstCommands.getCompileArgs).
  • code A function reference (a code value, such as one handed to a native) gets an xpcall adapter (LuaTranslator.callbackAdapterFor) and package init is xpcall-wrapped, so an error is reported through the Wurst error function when the program has one, else through BJDebugMsg.
  • measured, 3.0.0.24268 (2026-10-21) The game's Lua enumeration natives behave as on Jass, nested ones included. Within a ForGroup, ForForce, EnumItemsInRect or EnumDestructablesInRect callback, the getter (GetEnumUnit, GetEnumPlayer, GetEnumItem, GetEnumDestructable) answers the inner enumeration's object inside a nested one, and the outer one's again after it returns. This holds for an enumeration nested in itself, in one of another kind (GetEnumUnit inside a ForForce or an EnumItemsInRect inside a ForGroup still answers the outer unit) and in a trigger run by TriggerEvaluate or TriggerExecute from the callback; outside any enumeration each getter answers null. ForGroup visits the units the group held when it was called: a unit removed during the enumeration is still visited, one added is not, and clearing the group stops nothing. So Lua calls these natives and getters directly, as Jass does (LuaNativeLowering). They used to be wrapped: each call collected the objects into a new table through a new closure, then ran the callback for each with a global override that every getter call tested. A Jass build, a Lua build with those wrappers and a Lua build without them recorded the same traces. Only the order a group visits its units differed between the Jass and the Lua build, alike with and without the wrappers. In the same client, 20,000 ForGroup calls over 16 units with one GetEnumUnit each took 1475 ms (median of 3, range 1301-1541) without the wrappers against 1957 ms (1725-3008) with them. 160,000 ForForce calls over one player took 960 ms (676-1006) against 1638 ms (1185-1779).
  • code The reference Lua 5.3 neither inlines a local function call nor caches a global or library lookup: luac -l on a loop that calls a one-line local function and reads a global and math.floor shows a CALL for the function and a GETTABUP/SETTABUP or GETTABUP+GETTABLE for each access, every iteration. If the game's VM is the reference 5.3 (reported above), every helper call, global read, table index and allocation the compiler leaves in runs, and the compiler's inliner and local optimisations are the only optimiser.
  • code Jass has no throw. I2S(1 div 0) is the deliberate thread abort (ErrorHandling.error() ends with it); Lua lowers exactly that shape to error("__wurst_abort_thread", 0), which the callback handlers ignore (ExprTranslation.isIntentionalThreadAbortCall, LuaNativeLowering). Folding or reshaping it breaks the abort (LuaBackendAuditTests, OptimizerTests).

Lockstep

Every client simulates the same game from the same inputs; anything that makes the simulation differ between machines desyncs it.

  • reported The iteration order of pairs() is not the same for different players (SyncedTable). So emitted Lua and stdlib code that compiles to Lua never iterate a table whose order is observable. A table is safe as a membership set (t[k] = true, t[k] ~= nil, t[k] = nil); anything iterated needs a separately kept insertion-ordered array, which is what a sparse set's dense array buys.
  • code The keyed table offers no iteration (LuaNatives). __wurst_keyedMapDestroy does loop with pairs, but only to set every entry to nil, which does not depend on order.
  • code The compiler's own output must be byte-identical for identical input (AGENTS.md §3).

Numbers

  • measured, 3.0.0 Jass literals and S2R are not correctly rounded and the two disagree: 0.1 reads one float high, 1.1 one float low, S2R("100.0") one above 100. Short exact binary fractions (0.5, 0.75, 100.0, 123456.78125) read exactly; integer parts past 2^24 are truncated. This is why SimpleRewrites.foldRealExactly folds only exact literals and exact results.
  • measured, 3.0.0 Lua reals have a 24-bit mantissa, not 53: (2^24 + 1) - 2^24 is 0, the literal 16777217. reads as 16777216, 0.9999999999999999 reads as exactly 1, and 0.7 + 0.1 + 0.1 + 0.1 computes to 1 - 2^-23. Literals are rounded to nearest and each arithmetic result is truncated, which neither double nor 32-bit float arithmetic reproduces. One R2I result is not evidence of a precision: R2I(0.7 + 0.1 + 0.1 + 0.1) is 0 for that reason, not because of doubles.
  • measured, 3.0.0 R2I truncates, and saturates outside the 32-bit range, in Jass and in the game's Lua alike: a computed 5000000320 is 2147483647, its negative -2147483648, a computed 1e20 2147483647. Inside the range the arithmetic x // 1 | 0 agrees with it; outside, the Lua lowering answers the bound itself (LuaNativeLowering.lowerRealToInt, ExprTranslation.realToInt). Integer arithmetic wraps at 32 bits on both (2147483647 + 1 is -2147483648).
  • measured, 3.0.0 Jass converts an integer stored where a real is declared (set r = n, local real r = n, an argument for a real parameter); the game's Lua keeps an integer an integer. 65536 stored into a real global, squared and divided by 65536. is 65536 on Jass and was 0 on Lua, the product wrapping at 32 bits. On Lua the translation converts at each such store with n + 0., as it already wrote an integer literal there as a real (ExprTranslation.isIntStoredAsReal); only Jass code stores an integer there.
  • measured, 3.0.0 R2S and R2SW print the whole integer part of a computed value (3e9 is 3000000000.000, -3e9 -3000000000.000); a computed 5e9 prints as 5000000320.000, which the interpreter does not model. Jass reads a real literal whose integer part passes 2^31 wrapped to 32 bits, signed: the literal 3000000000. is negative, 10000000000. reads as 1410065408 and 1e20 written out as 1661992960, which is 10^20 modulo 2^32. Lua reads such a literal correctly. So the Jass backend prints the float a literal means wherever Jass would read it otherwise (imtojass.ExprTranslation.realLiteral): past 2^24 the float's own digits (16777219. becomes 16777220.), and from 2^31 on a product of exact factors, its odd part and powers of two up to 2^30 (3000000000. becomes 5859375. * 512.), which reads as the literal's float in the game. A literal past the largest float, such as the stdlib's REAL_MAX (2^128), is printed as the largest float: Jass has no product which gives infinity (2^150 compares below 3e38 and prints as 0.000), and the largest float compares above 3e38 (its R2S is inf). A compile-time real is written as a plain decimal, since Jass parses no exponent.
  • measured, 3.0.0 A real modulo is Blizzard.j's ModuloReal on Jass, a - I2R(R2I(a / b)) * b plus b when negative. The same formula in the game's Lua gives Jass's result for every dividend both read alike (54 of 54 probed); Lua's float % and math.fmod do not: 360. % 7. is 3.0000153 with either and 3 in Jass, 7.5 % 360. is 7.4999995 and 7.5, and -7.5 % 2.5 is 2.4999993 with fmod and its correction and 0 in Jass. The integer % agrees with Jass's ModuloInteger for a positive divisor on every value probed, the 32-bit edges included. So a real modulo is that formula with R2I as Lua arithmetic, and only an integer one by a positive literal is % (LuaNativeLowering).
  • measured, 3.0.0 A real division by zero stops the thread in Jass: 1. / 0., -1. / 0., 0. / 0., 5. % 0. and ModuloReal(5., 0.) all do. The game's Lua goes on: 1. / 0. is 1.7014e38, which is 2^127: neither the largest real (about 3.4e38, which R2S prints as inf) nor an infinity, -1. / 0. its negative, 0. / 0. is 1, and the Lua real modulo by 0 answers a number. The interpreters stop as Jass does (WurstOperator.divideReal); the Lua backend adds no check, which would cost every real division.

Establishing a fact

  • Measure in the real client with discriminating values, such as 2^24 + 1 or sums just below an integer, computed at run time from array operands so that nothing is folded, and read the emitted Lua to confirm. State the client version where the fact is recorded.
  • Never launch Warcraft III yourself: it needs the maintainer's Battle.net sign-in and an install that matches the pinned patch. Ask for a probe.
  • Benchmark Lua output like a shipping map: -inline -localOptimizations and no -stacktraces, which inflated call-heavy blocks 3-4x. Accumulate inside a timed loop in a local, not a global: a global's lookup cost follows the global table's hash layout, which shifts when identifiers are renamed, so a block whose Lua is byte-identical moved 15% between builds. Compare alternating runs, at least three per build, and treat shifts under 15% in global-touching blocks as noise.