Also backported tests from our stand-alone Json lib to tolerate subtle
differences between simdjson and our backend. The differences are
really minor:
1. simdjson tolerates invalid unicode codepoint escapes (\udd6a) whereas our
lib is more restrictive.
2. simdjson has a maximum nesting level of 1024 and our lib has 512.
In either case these subtleties are not a problem in practice -- just that the
tests needed to be updated to tolerate these differences.
- Added a "fast path" to the JSON parser for strings (""). The optimistic case is that the string data is just pure ascii, with no escape chars. In that case, we simply take a shallow copy of the buffer we are currently parsing. This avoids using the (slow) `JSONUTF8StringFilter` to process unicode and unicode escapes (which does `push_back()` per character). The fast path offers a 10% or more speedup on JSON parsing (as well as fewer mallocs -> less heap fragmentation over time).
- JSON parser takes shallow copies of string data whenever it can do so (conceptually similar to `std::string_view`), to avoid on redundant copies and mallocs.
- Overall JSON parser is ~15% faster now.
- Fixed the correctness bug where unicode code point `\u0000` would end up being interpreted as end-of-string. We can now pass the original UniValue lib's `round1.json` test. This bug was due to the way `QString::fromUtf8` worked for `QByteArray`s. It turns out passing QString a raw C pointer with a size param is better. See: [This line of code](ba3b53cb50/src/corelib/text/qstring.h (L701)) versus [this line of code](ba3b53cb50/src/corelib/text/qstring.h (L709)).
- Added more corner-case JSON tests
- We also set the C++ locale and the Qt default `QLocale` to `"C"` as well on startup.. just in case.
- JSON Number parse was needlessly pushing 1 character at a time to the
output QByteArray. We instead just accept characters until the end
of token, then push the entire token as 1 call. This avoids extra
mallocs and extra function calls.
- Fixed a bug where the parser would accept bare "-" by itself as if it
were a number, but then fail later.
- Optimized some expressions slightly.
- Fixed some read-past-end-of-buffer bugs (in practice would not have
caused problems since they would technically be reading the NUL byte).
- Added more tests for corner cases (such as the '-' situation described
above).
- Made the Json_Parser.cpp "Container" struct store the parsed
QByteArray directly rather than it being wastefully wrapped by a
QVariant.
- Added a note-to-self about why we didn't use a union class (it bought us
no performance to do so and it just added boilerplate code). Maybe in
the future when we have std::variant on all compilers we target that
may be a decent alternative.
These tests were taken from the BCHN UniValue lib's tests. They are
pretty good. Can be run if compiling with -DENABLE_TESTS and then
specifying --test json on the CLI.
