Skip to content

fix(parser): report errors instead of hitting TOML_ASSERT_ASSUME on malformed input - #321

Merged
marzer merged 1 commit into
marzer:masterfrom
emincan2000:fix-parser-asserts
Nov 1, 2026
Merged

marzer merged 1 commit into
marzer:masterfrom
emincan2000:fix-parser-asserts

Conversation

@emincan2000

Copy link
Copy Markdown
Contributor

Some malformed documents reach a TOML_ASSERT_ASSUME in the parser. In a debug build that aborts:

toml::parse("a = [}]"sv);
// parser.inl: parse_value(): Assertion `!is_value_terminator(*cp)' failed.

and with NDEBUG it turns into TOML_ASSUME, so the compiler is told something that the input can make false. In my release builds these inputs happened to produce a parse error, but that isn't guaranteed.

I found three such places:

input where
a = [}], a = [1, }], a = [<U+2028>] (also U+2029, U+0085) parse_array() passes a value terminator to parse_value(). consume_line_break() only handles \n/\r\n, but is_value_terminator() also covers non-ASCII vertical whitespace.
a = 1987-03-16T: parse_date_time() calls parse_time(), which assumes a digit.
a = 1987-03-16 1#, a = 1987-03-16 1, the date-time rescan in parse_value() calls scan() right before a terminator.

Each of them now sets a parse error instead (expected value or closing ']', expected 2-digit hour, and in the last case scan() just returns, the value is then parsed as a date and fails on the trailing 1). I didn't touch consume_line_break() because that would also reject U+2028 inside comments, which is valid.

Tests: added the inputs above to parsing_arrays.cpp and parsing_dates_and_times.cpp. On a debug build the test binary aborts with the assertion without the change and passes with it (the C and en_US.utf8 runs; the other locales aren't installed on my machine). toml.hpp is regenerated with tools/generate_single_header.py.

How I found them: comparing toml++ with Python's tomllib on random documents (100k, a part of them mutated). After the change there were no assertions, and for every document where the two parsers disagree, toml++ gives the same result as before the change, so nothing that parsed before behaves differently now.

@emincan2000

Copy link
Copy Markdown
Contributor Author

The three failing linux (... lld ...) jobs stop in the setup step before building: Error: Unable to locate package lld-14 from apt install. Looks like the runner image no longer has it (#320 two days ago was still green), so it isn't related to this change.

@BurnsGG

BurnsGG commented Oct 31, 2026

Copy link
Copy Markdown

Another reproduction for this PR's parse_date_time change, found by fuzzing a settings reader built on toml++ v3.4.0 (TOML_EXCEPTIONS=0, header-only, clang 22, Linux x86-64).

Input: x = 2026-10-29T:. Any non-digit right after the T/t does it, as long as the value has a : later so parse_value classifies it as a date-time. Our fuzzer's original was "Odd" = 2026-10-29T[[12:30:00Z.

With asserts on (no NDEBUG), it aborts:
parser.inl:2355: time toml::impl::parser::parse_time(bool): Assertion 'is_decimal_digit(*cp)' failed.
parse_date_time calls parse_time(true) after the separator without checking for a digit.

With NDEBUG (-O2), clang 22 currently produces a parse error (Error while parsing time: expected 2-digit hour, saw ':'), because consume_digit_sequence re-checks the digit. AddressSanitizer reports no out-of-bounds read. But -fsanitize=undefined reports parser.inl:2355:4: runtime error: assumption is violated during execution, so it is undefined behaviour.

The same holds for the PR's other two inputs: a = 2026-10-13 1\n (issue #313; parser.inl:2646) and a = [}] (parser.inl:2559).

The patch: we applied this PR's include/toml++/impl/parser.inl hunks unchanged to v3.4.0. Each input then gives a clean parse error:

  • Error while parsing date-time: expected 2-digit hour, saw ':'
  • Error while parsing key-value pair: expected a comment or whitespace, saw '1'
  • Error while parsing array: expected value or closing ']', saw '}'

The test suite passes with the same counts as without the patch (53 cases with exceptions, 55 without), and with the PR's new test inputs added.

@emincan2000

Copy link
Copy Markdown
Contributor Author

Thanks for testing it! Your input is the same case as the 1987-03-16T: one in the tests, so it's covered. Good to know it also shows up in real use.

@marzer

marzer commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Thanks for the fix! Looks like I need to update the CI for this repo since github has recently done some updates to their runners. I'll get on that tomorrow then we can merge this.

…alformed input

Three places assumed something about the input that malformed documents
can break, so they hit an assertion in debug builds and became
TOML_ASSUME (undefined behaviour) with NDEBUG:

- parse_array() passed a value terminator such as '}' or a non-ASCII line
  break (U+2028, U+2029, U+0085) to parse_value(): "a = [}]"
- parse_date_time() called parse_time() without checking that the time
  starts with a digit: "a = 1987-03-16T:"
- the date-time rescan in parse_value() called scan() right before a
  terminator: "a = 1987-03-16 1#"

Each now reports a parse error. Valid documents parse as before.
@marzer
marzer force-pushed the fix-parser-asserts branch from c27156b to 435fb98 Compare November 1, 2026 11:24
@marzer
marzer merged commit 1bf29cf into marzer:master Nov 1, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants