Repository navigation
fix(parser): report errors instead of hitting TOML_ASSERT_ASSUME on malformed input - #321
Conversation
|
The three failing |
|
Another reproduction for this PR's Input: With asserts on (no With The same holds for the PR's other two inputs: The patch: we applied this PR's
The test suite passes with the same counts as without the patch (53 cases with exceptions, 55 without), and with the PR's new test inputs added. |
|
Thanks for testing it! Your input is the same case as the |
|
Thanks for the fix! Looks like I need to update the CI for this repo since github has recently done some updates to their runners. I'll get on that tomorrow then we can merge this. |
…alformed input Three places assumed something about the input that malformed documents can break, so they hit an assertion in debug builds and became TOML_ASSUME (undefined behaviour) with NDEBUG: - parse_array() passed a value terminator such as '}' or a non-ASCII line break (U+2028, U+2029, U+0085) to parse_value(): "a = [}]" - parse_date_time() called parse_time() without checking that the time starts with a digit: "a = 1987-03-16T:" - the date-time rescan in parse_value() called scan() right before a terminator: "a = 1987-03-16 1#" Each now reports a parse error. Valid documents parse as before.
c27156b to
435fb98
Compare
Some malformed documents reach a
TOML_ASSERT_ASSUMEin the parser. In a debug build that aborts:and with
NDEBUGit turns intoTOML_ASSUME, so the compiler is told something that the input can make false. In my release builds these inputs happened to produce a parse error, but that isn't guaranteed.I found three such places:
a = [}],a = [1, }],a = [<U+2028>](also U+2029, U+0085)parse_array()passes a value terminator toparse_value().consume_line_break()only handles\n/\r\n, butis_value_terminator()also covers non-ASCII vertical whitespace.a = 1987-03-16T:parse_date_time()callsparse_time(), which assumes a digit.a = 1987-03-16 1#,a = 1987-03-16 1,parse_value()callsscan()right before a terminator.Each of them now sets a parse error instead (
expected value or closing ']',expected 2-digit hour, and in the last casescan()just returns, the value is then parsed as a date and fails on the trailing1). I didn't touchconsume_line_break()because that would also reject U+2028 inside comments, which is valid.Tests: added the inputs above to
parsing_arrays.cppandparsing_dates_and_times.cpp. On a debug build the test binary aborts with the assertion without the change and passes with it (theCanden_US.utf8runs; the other locales aren't installed on my machine).toml.hppis regenerated withtools/generate_single_header.py.How I found them: comparing toml++ with Python's
tomllibon random documents (100k, a part of them mutated). After the change there were no assertions, and for every document where the two parsers disagree, toml++ gives the same result as before the change, so nothing that parsed before behaves differently now.