summaryrefslogtreecommitdiff
path: root/ext/json/parser/parser.h
AgeCommit message (Collapse)Author
2024-11-05[ruby/json] ResyncJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/12003
2024-11-01[ruby/json] json_string_unescape: assume the string doesn't need escapingJean Boussier
If that assumption holds true, then we don't need to copy the string into a buffer to unescape it. For small string is just saves copying, but for large ones it also saves a malloc/free combo. Before: ``` == Parsing twitter.json (567916 bytes) ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json 52.000 i/100ms oj 61.000 i/100ms oj strict 70.000 i/100ms Oj::Parser 71.000 i/100ms rapidjson 55.000 i/100ms Calculating ------------------------------------- json 510.111 (± 2.9%) i/s (1.96 ms/i) - 2.548k in 5.000029s oj 610.232 (± 3.1%) i/s (1.64 ms/i) - 3.050k in 5.003725s oj strict 713.231 (± 3.2%) i/s (1.40 ms/i) - 3.570k in 5.010902s Oj::Parser 762.598 (± 3.0%) i/s (1.31 ms/i) - 3.834k in 5.033130s rapidjson 553.029 (± 7.4%) i/s (1.81 ms/i) - 2.750k in 5.022630s Comparison: json: 510.1 i/s Oj::Parser: 762.6 i/s - 1.49x faster oj strict: 713.2 i/s - 1.40x faster oj: 610.2 i/s - 1.20x faster rapidjson: 553.0 i/s - same-ish: difference falls within error == Parsing citm_catalog.json (1727030 bytes) ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json 28.000 i/100ms oj 33.000 i/100ms oj strict 37.000 i/100ms Oj::Parser 43.000 i/100ms rapidjson 38.000 i/100ms Calculating ------------------------------------- json 303.853 (± 3.6%) i/s (3.29 ms/i) - 1.540k in 5.076079s oj 348.009 (± 2.0%) i/s (2.87 ms/i) - 1.749k in 5.027738s oj strict 396.679 (± 3.3%) i/s (2.52 ms/i) - 1.998k in 5.042271s Oj::Parser 406.699 (± 2.2%) i/s (2.46 ms/i) - 2.064k in 5.077587s rapidjson 393.463 (± 3.3%) i/s (2.54 ms/i) - 1.976k in 5.028501s Comparison: json: 303.9 i/s Oj::Parser: 406.7 i/s - 1.34x faster oj strict: 396.7 i/s - 1.31x faster rapidjson: 393.5 i/s - 1.29x faster oj: 348.0 i/s - 1.15x faster ``` After: ``` == Parsing twitter.json (567916 bytes) ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json 56.000 i/100ms oj 62.000 i/100ms oj strict 72.000 i/100ms Oj::Parser 77.000 i/100ms rapidjson 55.000 i/100ms Calculating ------------------------------------- json 568.025 (± 2.1%) i/s (1.76 ms/i) - 2.856k in 5.030272s oj 630.936 (± 1.4%) i/s (1.58 ms/i) - 3.162k in 5.012630s oj strict 705.784 (±11.2%) i/s (1.42 ms/i) - 3.456k in 5.006706s Oj::Parser 783.989 (± 1.7%) i/s (1.28 ms/i) - 3.927k in 5.010343s rapidjson 557.630 (± 2.0%) i/s (1.79 ms/i) - 2.805k in 5.032388s Comparison: json: 568.0 i/s Oj::Parser: 784.0 i/s - 1.38x faster oj strict: 705.8 i/s - 1.24x faster oj: 630.9 i/s - 1.11x faster rapidjson: 557.6 i/s - same-ish: difference falls within error == Parsing citm_catalog.json (1727030 bytes) ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- json 29.000 i/100ms oj 33.000 i/100ms oj strict 38.000 i/100ms Oj::Parser 43.000 i/100ms rapidjson 37.000 i/100ms Calculating ------------------------------------- json 319.271 (± 3.1%) i/s (3.13 ms/i) - 1.595k in 5.001128s oj 347.946 (± 1.7%) i/s (2.87 ms/i) - 1.749k in 5.028395s oj strict 396.914 (± 3.0%) i/s (2.52 ms/i) - 2.014k in 5.079645s Oj::Parser 409.311 (± 2.7%) i/s (2.44 ms/i) - 2.064k in 5.046626s rapidjson 394.752 (± 1.5%) i/s (2.53 ms/i) - 1.998k in 5.062776s Comparison: json: 319.3 i/s Oj::Parser: 409.3 i/s - 1.28x faster oj strict: 396.9 i/s - 1.24x faster rapidjson: 394.8 i/s - 1.24x faster oj: 347.9 i/s - 1.09x faster ``` https://github.com/ruby/json/commit/7e0f66546a
2024-11-01Deprecate unsafe default options of `JSON.load`Jean Boussier
[Feature #19528] Ref: https://bugs.ruby-lang.org/issues/19528 `load` is understood as the default method for serializer kind of libraries, and the default options of `JSON.load` has caused many security vulnerabilities over the years. The plan is to do like YAML/Psych, deprecate these default options and direct users toward using `JSON.unsafe_load` so at least it's obvious it should be used against untrusted data.
2024-10-30[ruby/json] Allocate the FBuffer struct on the stackJean Boussier
Ref: https://github.com/ruby/json/issues/655 The actual buffer is still on the heap, but this saves a pair of malloc/free. This helps a lot on micro-benchmarks Before: ``` ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- Oj 531.598k i/100ms JSON reuse 417.666k i/100ms Calculating ------------------------------------- Oj 5.735M (± 1.3%) i/s (174.35 ns/i) - 28.706M in 5.005900s JSON reuse 4.604M (± 1.4%) i/s (217.18 ns/i) - 23.389M in 5.080779s Comparison: Oj: 5735475.6 i/s JSON reuse: 4604380.3 i/s - 1.25x slower ``` After: ``` ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23] Warming up -------------------------------------- Oj 518.700k i/100ms JSON reuse 483.370k i/100ms Calculating ------------------------------------- Oj 5.722M (± 1.8%) i/s (174.76 ns/i) - 29.047M in 5.077823s JSON reuse 5.278M (± 1.5%) i/s (189.46 ns/i) - 26.585M in 5.038172s Comparison: Oj: 5722283.8 i/s JSON reuse: 5278061.7 i/s - 1.08x slower ``` Bench: ```ruby require 'benchmark/ips' require 'oj' require 'json' json_encoder = JSON::State.new(JSON.dump_default_options) test_data = [1, "string", { a: 1, b: 2 }, [3, 4, 5]] Oj.default_options = Oj.default_options.merge(mode: :compat) Benchmark.ips do |x| x.config(time: 5, warmup: 2) x.report("Oj") do Oj.dump(test_data) end x.report("JSON reuse") do json_encoder.generate(test_data) end x.compare!(order: :baseline) end ``` https://github.com/ruby/json/commit/72110f7992
2024-10-26[ruby/json] Use smaller types for JSON_Parser boolean fieldsJean Boussier
https://github.com/ruby/json/commit/7f079b25be
2024-10-18[ruby/json] Speedup Parser initializationJean Boussier
Extracted from: https://github.com/ruby/json/pull/512 Use `rb_hash_lookup2` to check for hash key existence instead of going through `rb_funcall`. https://github.com/ruby/json/commit/43835a0d13 Co-Authored-By: lukeg <[email protected]>
2024-10-17[ruby/json] Get rid of compatibility code for older rubiesJean Boussier
All of these are for rubies older than 2.3. https://github.com/ruby/json/commit/811297f86a
2024-10-08[ruby/json] Adjust to the CVTUTF code being goneLuke T. Shumaker
I, Luke T. Shumaker, am the sole author of the added code. I did not reference CVTUTF when writing it. I did reference the Unicode standard (15.0.0), the Wikipedia article on UTF-8, and the Wikipedia article on UTF-16. When I saw some tests fail, I did reference the old deleted code (but a JSON-specific part, inherently not as based on CVTUTF) to determine that script_safe should also escape U+2028 and U+2029. I targeted simplicity and clarity when writing the code--it can likely be optimized. In my mind, the obvious next optimization is to have it combine contiguous non-escaped characters into just one call to fbuffer_append(), instead of calling fbuffer_append() for each character. Regarding the use of the "modern" types `uint32_t`, `uint16_t`, and `bool`: - ruby.h is guaranteed to give us uint32_t and uint16_t. - Since Ruby 3.0.0, ruby.h is guaranteed to give us bool... but we support down to Ruby 2.3. But, ruby.h is guaranteed to give us HAVE_STDBOOL_H for the C99 stdbool.h; so use that to include stdbool.h if we can, and if not then fall back to a copy of the same bool definition that Ruby 3.0.5 uses with C89. https://github.com/ruby/json/commit/c96351f874
2024-10-08[ruby/json] Delete code that is based on CVTUTFLuke T. Shumaker
I did this based on manual inspection, comparing the code to my re-created history of CVTUTF at https://git.lukeshu.com/2git/cvtutf/ (created by the scripts at https://git.lukeshu.com/2git/cvtutf-make/) https://github.com/ruby/json/commit/0819553144
2024-06-04[flori/json] Cleanup useless ifdefJean Boussier
The json gem now requires Ruby 2.3, so there is no point keeping compatibility code for older releases that don't have the TypedData API. https://github.com/flori/json/commit/45c86e153f
2021-05-19ext/json/parser/parser.h: Add fallback MAYBE_UNUSEDNobuyoshi Nakada
https://github.com/flori/json/commit/e2ad91fc2094d3cc2f76adc6c55d420cd06f34d8
2021-05-17[flori/json] Deduplicate strings inside json_string_unescapeJean Boussier
[ci 2] https://github.com/flori/json/commit/1982070cb8
2021-05-17[flori/json] Refactor json_string_unescapeJean Boussier
https://github.com/flori/json/commit/f398769332
2020-10-20Implement a freeze: parser optionJean Boussier
If set to true all parsed objects will be immediately frozen, and strings will be deduplicated if the Ruby implementation allows it.
2017-06-16Merge json-2.1.0 from https://github.com/flori/jsonhsbt
https://github.com/flori/json/blob/master/CHANGES.md#2017-04-18-210 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59099 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-05* ext/json/*, test/json/*: Update json-2.0.1.hsbt
Changes of 2.0.0: https://github.com/flori/json/blob/f679ebd0c69a94e3e70a897ac9a229f5779c2ee1/CHANGES.md#2015-09-11-200 Changes of 2.0.1: https://github.com/flori/json/blob/f679ebd0c69a94e3e70a897ac9a229f5779c2ee1/CHANGES.md#2016-07-01-201 [Feature #12542][ruby-dev:49706][fix GH-1395] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-05-28parser.rl: allocate structs with wrappernobu
* ext/json/parser/parser.rl (cJSON_parser_s_allocate): allocate structs with making new wrapper objects and get rid of potential memory leak. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50660 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-12* ext/json/*, test/json/*: Reverted r50231. Because it's not works withhsbt
cross-compile environment. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-11* ext/json/*, test/json/*, defs/default_gems: Gemify JSON library.hsbt
[fix GH-867][Feature #11057] * test/ruby/test_extlibs.rb: removed json gem from existence extentions. * gems/bundled_gems: added json gem into bundled gem. * lib/rdoc/rubygems_hook.rb: ignored no json environment. * lib/rubygems/test_case.rb, test/rubygems/*: ditto. * lib/rdoc/test_case.rb, test/rdoc/*: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50231 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-13* ext/json: merge upstream from flori/jsonhsbt
change usage of TypedData. [Feature #10739][ruby-core:67564] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-28ext/json: for ancient backward compatibilitiesnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49050 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-27json: backward compatibilitiesnobu
* ext/json/generator/generator.c (JSON_Generator_State_type): add #ifdef for backward compatibility. * ext/json/parser/parser.rl (JSON_Parser_type): ditto. * ext/json/generator/generator.h (ZALLOC): add fallback definition. * ext/json/parser/parser.h (ZALLOC): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-04json/parser: typed datanobu
* ext/json/parser/parser.rl (JSON_Parser_type): turn into typed data. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47798 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-04ext: protoize no-arguments functionsnobu
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47793 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2012-03-11* ext/json: Merge 164a75c8bd2007d32c4d7665d53140d8fc126dcd.naruse
[ruby-core:41917] [Bug #5846] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@34971 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-08-30* ext/json: Merge json gem 1.5.4+ (2149f4185c598fb97db1).naruse
[Bug #5173] [ruby-core:38866] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33122 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-07-10* ext/json: Merge json gem 1.5.4+ (f7f78896607b6f6226cd).naruse
[Bug #4700] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@32493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-02-05* ext/json/parser/parser.h (GET_PARSER): raise TypeError.nobu
* ext/json/parser/parser.rl (cParser_initialize): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@30792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2011-02-05* ext/json/parser/parser.h (GET_PARSER): check if initialized.nobu
[ruby-core:35079] * ext/json/parser/parser.rl (cParser_initialize): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@30791 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-26Recommit of JSON; fix mixed declarations.naruse
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27501 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-26* ext/jason: revert r27493. came again after canceling gcc-ism.usa
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27500 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2010-04-26* ext/json: Update to JSON 1.4.1.naruse
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@27493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-19 19:31:55 +0000