summaryrefslogtreecommitdiff
path: root/test/prism/parser_test.rb
AgeCommit message (Collapse)Author
2024-05-30[ruby/prism] Tests overhaulKevin Newton
https://github.com/ruby/prism/commit/6f886be0a4
2024-04-12[ruby/prism] Fix parser translation's heredoc whitespace calculationStan Lo
Given this example: ```rb <<~HEREDOC #{x} HEREDOC ``` Both the parser gem and Prism's translation layer would generate the following AST: ``` s(:dstr, s(:begin, s(:int, 1)), s(:str, " a\n")) ``` However, the parser gem inserts a empty string node into this node's location, like: ``` <Parser::Source::Map::Heredoc:0x0000000104ce73b8 @expression=#<Parser::Source::Range (string) 0...10>, @heredoc_body=#<Parser::Source::Range (string) 11...20>, @heredoc_end=#<Parser::Source::Range (string) 20...27>, @node=s(:dstr, s(:str, ""), s(:begin, s(:int, 1)), s(:str, " a\n"))> ``` This is required to calculate the correct whitespace for the heredoc body. We need to adjust the translation layer to account for this. With this fix, we also won't need to ignore the tilde heredoc fixture anymore. https://github.com/ruby/prism/commit/e7372e3ba5
2024-04-12Sync latest prismKevin Newton
2024-03-15[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`Koichi ITO
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for the heredocs_leading_whitespace.txt test. https://github.com/ruby/prism/commit/7d45fb1eed
2024-03-13[ruby/prism] Turn on heredocs nested test for `Prism::Translation::Parser`Koichi ITO
This PR enables the heredocs_nested.txt that was previously skipped testing. https://github.com/ruby/prism/commit/9c54f7f10b
2024-03-12[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`Koichi ITO
In practice, the `BACKTICK` is mapped either as `:tXSTRING_BEG` or `:tBACK_REF2`. The former is used in xstrings like `` `foo` ``, while the latter is utilized as a back reference in contexts like `` A::` ``. This PR will make corrections to differentiate the use of `BACKTICK`. This mistake was discovered through the investigation of xstring.txt file. The PR will run tests from xstring.txt file except for `` `f\oo` ``, which will still fail, hence it will be separated into xstring_with_backslash.txt file. This separation will facilitate addressing the correction at a different time. https://github.com/ruby/prism/commit/49ad8df40a
2024-03-12[ruby/prism] Follow ruby/prism#2581Koichi ITO
Due to ruby/prism#2581, the test case for non_alphanumeric_methods.txt has also been restored. It also solved issues with `def self.`' not limited to constants for `tBACK_REF2` token. ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "def self.`; end"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:kDEF, ["def", #<Parser::Source::Range example.rb 0...3>]], [:kSELF, ["self", #<Parser::Source::Range example.rb 4...8>]], [:tDOT, [".", #<Parser::Source::Range example.rb 8...9>]], [:tBACK_REF2, ["`", #<Parser::Source::Range example.rb 9...10>]], [:tSEMI, [";", #<Parser::Source::Range example.rb 10...11>]], [:kEND, ["end", #<Parser::Source::Range example.rb 12...15>]]] ``` https://github.com/ruby/prism/commit/f814db8ada
2024-03-12[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`Koichi ITO
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for `tBACK_REF2`: ## Parser gem (Expected) Returns `tBACK_REF2` token: ```console $ bundle exec ruby -Ilib -rparser/ruby33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "A::`"; p Parser::Ruby33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tCONSTANT, ["A", #<Parser::Source::Range example.rb 0...1>]], [:tCOLON2, ["::", #<Parser::Source::Range example.rb 1...3>]], [:tBACK_REF2, ["`", #<Parser::Source::Range example.rb 3...4>]]] ``` ## `Prism::Translation::Parser` (Actual) Previously, the parser returned `tXSTRING_BEG` token when parsing the following: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "A::`"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tCONSTANT, ["A", #<Parser::Source::Range example.rb 0...1>]], [:tCOLON2, ["::", #<Parser::Source::Range example.rb 1...3>]], [:tXSTRING_BEG, ["`", #<Parser::Source::Range example.rb 3...4>]]] ``` After the update, the parser now returns `tBACK_REF2` token for the same input: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "A::`"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tCONSTANT, ["A", #<Parser::Source::Range example.rb 0...1>]], [:tCOLON2, ["::", #<Parser::Source::Range example.rb 1...3>]], [:tBACK_REF2, ["`", #<Parser::Source::Range example.rb 3...4>]]] ``` This correction enables the restoration of `constants.txt` as a test case. https://github.com/ruby/prism/commit/7f63b28f98
2024-03-12[ruby/prism] Follow #2579Koichi ITO
Due to #2579, the test case for range_begin_open_inclusive.txt has also been restored. https://github.com/ruby/prism/commit/da173526a1
2024-03-12[ruby/prism] Restore some skipped heredoc tests for `Prism::Translation::Parser`Koichi ITO
This PR restores some heredoc tests that have already passed for `Prism::Translation::Parser`. https://github.com/ruby/prism/commit/297b9e2380
2024-03-12[ruby/prism] Fix a token incompatibility for `Prism::Translation::Parser::Lexer`Koichi ITO
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for beginless range: ## Parser gem (Expected) Returns `tBDOT2` token: ```console $ bundle exec ruby -Ilib -rparser/ruby33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Parser::Ruby33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` ## `Prism::Translation::Parser` (Actual) Previously, the parser returned `tDOT2` token when parsing the following: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` After the update, the parser now returns `tBDOT2` token for the same input: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22] [[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` This correction enables the restoration of `endless_range_in_conditional.txt` as a test case. https://github.com/ruby/prism/commit/f624b99ab0
2024-03-07[ruby/prism] Fix up test suiteKevin Newton
https://github.com/ruby/prism/commit/2929c35630
2024-03-06[ruby/prism] Use the diagnostic types in the parser translation layerKevin Newton
https://github.com/ruby/prism/commit/1a8a0063dc
2024-03-06[ruby/prism] Fix implicit local variables in hashesKevin Newton
https://github.com/ruby/prism/commit/05e0c6792c
2024-03-04[ruby/prism] Fix up some minor parser incompatibilitiesKevin Newton
https://github.com/ruby/prism/commit/c6c771d1fa
2024-02-29[ruby/prism] Fix an incorrect parsing for `Prism::Translation::Parser`Koichi ITO
This PR fixes an incorrect parsing for `Prism::Translation::Parser` when one-line pattern mathing with Ruby 2.7 runtime. ## Expected Parsing should be done based on the specified Ruby parsing version, independent of the Ruby runtime version. When parsing for Ruby 3.3, it should return `:match_pattern_p` node: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("foo in bar")' ruby 3.0.6p216 (2023-03-30 revision https://github.com/ruby/prism/commit/23a532679b) [x86_64-darwin19] s(:match_pattern_p, s(:send, nil, :foo), s(:match_var, :bar)) ``` ## Actual When parsing with Ruby 2.7 runtime, `match_pattern` node is returned, even though it is expected to parse for Ruby 3.3: ```console $ ruby -rprism -rprism/translation/parser33 -ve 'p Prism::Translation::Parser33.parse("foo in bar")' ruby 2.7.8p225 (2023-03-30 revision https://github.com/ruby/prism/commit/1f4d455848) [x86_64-darwin19] s(:match_pattern, s(:send, nil, :foo), s(:match_var, :bar)) ``` The cause was the use of `RUBY_VERSION` for condition logic, which made it dependent on runtime Ruby version. `Prism::Translation::Parser` supports parsing for Ruby 3.3+. Therefore, the condition for parsing Ruby 2.7, which is not supported, is being removed. ## Background Found due to incompatibility with RuboCop's `Layout/SpaceAroundKeyword` and `Style/TernaryParentheses` cops. https://github.com/ruby/prism/commit/e752e251d2
2024-02-21[ruby/prism] Translation::Parser should process warnings, not just errorsNoah Gibbs
https://github.com/ruby/prism/commit/ea7e400f85
2024-02-18[ruby/prism] Account for encoding in regexp named capturesKevin Newton
https://github.com/ruby/prism/commit/17dc6b6281
2024-02-16[ruby/prism] Ignore incorrect filesKevin Newton
https://github.com/ruby/prism/commit/d1094ac232
2024-02-16[ruby/prism] Fix block_pass for []=Kevin Newton
https://github.com/ruby/prism/commit/bf79206220
2024-02-12[ruby/prism] Error messages closer to CRubyKevin Newton
https://github.com/ruby/prism/commit/19ffa0b980
2024-01-27[ruby/prism] Add parser translationKevin Newton
https://github.com/ruby/prism/commit/8cdec8070c
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-19 21:44:33 +0000