| Age | Commit message (Collapse) | Author |
|---|
|
https://github.com/ruby/prism/commit/12af4e144e
|
|
There hasn't been much that would actually affect parsers usage of it.
But, when adding new node types, these usually appear in the `Parser::Meta::NODE_TYPES`.
`itblock` was added, gets emitted by prism, and then `rubocop-ast` blindly delegates to `on_itblock`.
These methods are dynamically created through `NODE_TYPES`, which means that it will error if it
doesn't contain `itblock`.
This is unfortunate because in `rubocop-ast` these methods are eagerly defined but
the prism translator is lazily loaded on demand.
The simplest solution is to add them on the `parser` side (even if they are not emitted directly), and require that a version that contains those be used.
In summary when adding a new node type:
* Add it to `Parser::Meta::PRISM_TRANSLATION_PARSER_NODE_TYPES` (gets included in `NODE_TYPES`)
* Bump the minimum `parser` version used by `prism` to a version that contains the above change
* Actually emit that node type in `prism`
https://github.com/ruby/prism/commit/d73783d065
|
|
https://github.com/ruby/prism/commit/d85c72a1b9
|
|
builder class
In https://github.com/ruby/prism/pull/3494 I added a bit of code
so that using the new builder doesn't break stuff.
This code can be dropped when it is enforced that builder
is _always_ the correct subclass (and makes future issues like that unlikely).
https://github.com/ruby/prism/commit/193d4b806d
|
|
Caused by https://github.com/ruby/prism/pull/3478 and https://github.com/ruby/prism/pull/3443
I also made the builder reference more explicit to clearly distinquish
between `::Parser` and `Prism::Translation::Parser`
https://github.com/ruby/prism/commit/d52aaa75b6
|
|
I want to add new node types to the parser translator, for example `itblock`. The bulk of the work is already done by prism itself. In the `parser`
builder, this would be a 5-line change at most but we don't control that here.
Instead, we can add our own builder and either overwrite the few methods we need,
or just inline the complete builder. I'm not sure yet which would be better.
`rubocop-ast` uses its own builder for `parser`. For this to correctly work, it must explicitly choose to extend the
prism builder and use it, same as it currently chooses to use a different parser when prism is used.
I'd like to enforce that the builder for prism extends its custom one since it will lead to
some pretty weird issues otherwise. But first, I'd like to change `rubocop-ast` to make use of this.
https://github.com/ruby/prism/commit/b080e608a8
|
|
This restores the missing method comments in https://github.com/ruby/prism/pull/3479.
https://github.com/ruby/prism/commit/78b8f67dee
|
|
Caused by https://github.com/ruby/prism/pull/3478 and https://github.com/ruby/prism/pull/3443
I also made the builder reference more explicit to clearly distinquish
between `::Parser` and `Prism::Translation::Parser`
https://github.com/ruby/prism/commit/d52aaa75b6
|
|
I want to add new node types to the parser translator, for example `itblock`. The bulk of the work is already done by prism itself. In the `parser`
builder, this would be a 5-line change at most but we don't control that here.
Instead, we can add our own builder and either overwrite the few methods we need,
or just inline the complete builder. I'm not sure yet which would be better.
`rubocop-ast` uses its own builder for `parser`. For this to correctly work, it must explicitly choose to extend the
prism builder and use it, same as it currently chooses to use a different parser when prism is used.
I'd like to enforce that the builder for prism extends its custom one since it will lead to
some pretty weird issues otherwise. But first, I'd like to change `rubocop-ast` to make use of this.
https://github.com/ruby/prism/commit/b080e608a8
|
|
Follow-up to https://github.com/Shopify/ruby-lsp/pull/1849.
This is an extension of `Prism::Translation::Parser` to implement https://github.com/Shopify/ruby-lsp/pull/1849.
It is based on the comments in https://github.com/Shopify/ruby-lsp/pull/1849#pullrequestreview-1966020868,
but also adds a default argument for delegation to `Parser::Base` super class.
Using this API, https://github.com/rubocop/rubocop-ast/pull/359 has been implemented in RuboCop AST.
As detailed in https://github.com/rubocop/rubocop-ast/pull/359, this change is expected to improve performance by 1.3x
for some source code.
Achieving a 1.3x speedup with such this simple modification is a significant improvement for Ruby LSP and its users.
https://github.com/ruby/prism/commit/925725291c
|
|
Skipping detecting the encoding is almost always right, just for binary it should actually happen.
A symbol containing escapes that are invalid
in utf-8 would fail to parse since symbols must be valid in the script encoding.
Additionally, the parser gem would raise an exception somewhere during string handling
https://github.com/ruby/prism/commit/fa0154d9e4
|
|
Follow up https://github.com/ruby/prism/pull/3336.
Development for Ruby 3.5 has begun on the master branch:
https://github.com/ruby/ruby/commit/2f064b3b4b71f9495bbc4229e7efdbfad494862f
https://github.com/ruby/prism/commit/aa49c1bd78
|
|
Followup to https://github.com/ruby/prism/pull/3079
https://github.com/ruby/prism/commit/68f434e356
|
|
https://github.com/ruby/prism/commit/f7faedfb3f
|
|
https://github.com/ruby/prism/commit/d4eb13e703
|
|
Resolves https://github.com/ruby/prism/pull/2803.
This PR adds error handling for missing `parser` gem in `Prism::Translation`.
The `parser` gem is a required runtime dependency when using `Prism::Translation::Parser`.
But it is not required for other uses of Prism. To avoid unnecessary dependencies,
it is not added as a `runtime_dependency` in the prism.gemspec. Instead, if the dependency is missing,
instructions are given to add it to Gemfile.
## Before
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
/Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require': cannot load such file -- parser (LoadError)
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:6:in `<module:Translation>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:4:in `<module:Prism>'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser33.rb:3:in `<top (required)>'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
from /Users/koic/.rbenv/versions/3.3.1/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
from -e:1:in `<main>'
```
## After
```console
$ bundle exec ruby -e 'require "prism"; require "prism/translation/parser33"'
Error: Unable to load parser. Add `gem "parser"` to your Gemfile.
```
https://github.com/ruby/prism/commit/4880aec22d
|
|
Follow up https://github.com/ruby/prism/pull/2760.
This PR updates the `Translation::Parser` to use version 3.3.1 when the version 3.3 is specified.
The Parser gem is structured to support the latest patch versions, hence this aligns with Parser-compatible versioning.
As noted in https://github.com/ruby/prism/pull/2760, the behavior remains unchanged with this switch from 3.3.0 to 3.3.1.
https://github.com/ruby/prism/commit/efde09d318
|
|
This is similar to https://github.com/davidwessman/syntax_tree-erb/issues/81 but for RuboCop
The parser gem doesn't support these types of checks,
see https://github.com/whitequark/parser?tab=readme-ov-file#syntax-check-of-block-exits
While this is technically a bug in the parser gem, it does increase compatibility
and allows prism to be used when linting erb or haml with a RuboCop extension.
https://github.com/ruby/prism/commit/6c59ae6a00
|
|
https://github.com/ruby/prism/commit/445a0f0d22
|
|
Fixes https://github.com/ruby/prism/pull/2617.
There was an issue with the lexer as follows.
The following are valid regexp options:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```
The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```
As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,
```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```
Thus, it should probably not be construed as `IDENTIFIER` token.
Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.
https://github.com/ruby/prism/commit/d2a6096fcf
|
|
This PR fixes a diagnostic incompatibility when using no anonymous keyword rest parameter:
```ruby
foo(**)
```
Note, although the actual update applies only to the `foo(**)` case, for reference,
`foo(*)` and `foo(&) are also mentioned below.
## Ruby (Expected)
```console
$ ruby -cve 'foo(*)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(**)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous keyword rest parameter
-e: compile error (SyntaxError)
$ ruby -cve 'foo(&)'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
-e:1: no anonymous block parameter
-e: compile error (SyntaxError)
```
## Prism (Actual)
Before:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:expect_expression_after_splat_hash @message="expected an expression after `**` in a hash"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
After:
```console
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(*)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star @message="unexpected `*` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(**)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_star_star @message="unexpected `**` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=2 start_line=1> @level=:fatal>]
$ bundle exec ruby -Ilib -rprism -wve 'p Prism.parse("foo(&)").errors'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[#<Prism::ParseError @type=:argument_no_forwarding_amp @message="unexpected `&` when the parent method is not forwarding"
@location=#<Prism::Location @start_offset=4 @length=1 start_line=1> @level=:fatal>]
```
https://github.com/ruby/prism/commit/633c9d9fd4
|
|
This PR fixes a diagnostic incompatibility for `Prism::Translation::Parser` when using constant argument:
```ruby
def foo(A)
end
```
## Parser gem (Expected)
Displays `formal argument cannot be a constant (Parser::SyntaxError)`:
```console
$ bundle exec ruby -Ilib -rparser/ruby33 -ve 'p Parser::Ruby33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:274:in `diagnostic'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/ruby33.rb:12177:in `_reduce_663'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `_racc_do_parse_c'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/racc-1.7.3/lib/racc/parser.rb:267:in `do_parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:190:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
## `Prism::Translation::Parser` (Actual)
Previously, the error messages displayed by the Parser gem were different.
Before:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error:
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': Parser::SyntaxError (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
After:
```console
$ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve 'Prism::Translation::Parser33.parse("def foo(A) end")'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
(string):1:9: error: formal argument cannot be a constant
(string):1: def foo(A) end
(string):1: ^
/Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/diagnostic/engine.rb:72:
in `process': formal argument cannot be a constant (Parser::SyntaxError)
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:218:in `block in unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `each'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:216:in `unwrap'
from /Users/koic/src/github.com/ruby/prism/lib/prism/translation/parser.rb:49:in `parse'
from /Users/koic/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/parser-3.3.0.5/lib/parser/base.rb:33:in `parse'
from -e:1:in `<main>'
```
https://github.com/ruby/prism/commit/4f2af88520
|
|
In the case of the `**` and `&` ambiguous prefixes, incompatibilities
remained among https://github.com/ruby/prism/issues/2513.
https://github.com/ruby/prism/commit/3b8b231aae
|
|
https://github.com/ruby/prism/commit/1a8a0063dc
|
|
https://github.com/ruby/prism/commit/c6c771d1fa
|
|
Parser::Diagnostic
https://github.com/ruby/prism/commit/6a84a3c9fb
|
|
https://github.com/ruby/prism/commit/c3cc282343
Co-authored-by: Kevin Newton <[email protected]>
|
|
https://github.com/ruby/prism/commit/ea7e400f85
|
|
## Summary
Fixes https://github.com/ruby/prism/pull/2356.
I'm working on integrating Prism into RuboCop.
This PR introduces `Prism::Translation::Parser33` and `Prism::Translation::Parser34`, named
in accordance with the following comments.
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1932707748
Currently, `Prism::Translation::Parser` always operates in Ruby 3.4 mode.
This means it will not parse as Ruby 3.3 even if `TargetRubyVersion: 80_82_73_83_77.33` is specified.
Therefore, the `it` introduced in Ruby 3.4 is parsed incompatibly with Ruby 3.3. In Ruby 3.3,
the expected name for an `lvar` is `:it`, not `:"0it"`.
### Expected AST
The following is an expected AST when parsing Ruby 3.3 code:
```console
$ bundle exec ruby -rprism -rprism/translation/parser33 -ve "p Prism::Translation::Parser33.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
s(:block,
s(:send,
s(:send, nil, :items), :map),
s(:args),
s(:send,
s(:send, nil, :it), :do_something))
```
### Actual AST
The following is an actual AST when parsing Ruby 3.3 code:
```console
$ ruby -rprism -ve "p Prism::Translation::Parser.parse('items.map { it.do_something }')"
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
s(:block,
s(:send,
s(:send, nil, :items), :map),
s(:args),
s(:send,
s(:lvar, :"0it"), :do_something))
```
`Prism::Translation::Parser33` and `Prism::Translation::Parser34` aim to correspond to Ruby 3.3 and Ruby 3.4, respectively.
And, The hack of specifying `TargetRubyVersion: 80_82_73_83_77.33` is expected to become unnecessary in the future,
but the behavior will be maintained until RuboCop's support is finalized:
https://github.com/rubocop/rubocop/issues/12600#issuecomment-1933657732
## Additional Information
A private method named `convert_for_prism` is prepared to convert the `version` from Parser to the `version` expected by Prism.
For example, a Parser-compatible value is `3.3`, whereas Prism expects `"3.3.0"`.
`Parser#version` is not used in RuboCop, but it's unclear how it is utilized in other libraries that rely on the Parser gem.
Therefore, logic to maintain compatibility between Parser and Prism is implemented.
https://github.com/ruby/prism/commit/62d3991e22
|
|
https://github.com/ruby/prism/commit/8cd92eef79
|
|
translation
https://github.com/ruby/prism/commit/63979de21d
|
|
https://github.com/ruby/prism/commit/4327051c86
|
|
https://github.com/ruby/prism/commit/102b4a16f5
|
|
https://github.com/ruby/prism/commit/8cdec8070c
|
