| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2024-05-04 | Change return value of `gets` function to be `rb_parser_string_t *` instead ↵ | yui-knk | |
| of `VALUE` This change reduces parser's dependency on ruby object. | |||
| 2024-05-03 | Rename `vast` to `ast_value` | yui-knk | |
| There is an English word "vast". This commit changes the name to be more clear name to avoid confusion. | |||
| 2024-04-27 | Add line_count field to rb_ast_body_t | HASUMI Hitoshi | |
| This patch adds `int line_count` field to `rb_ast_body_t` structure. Instead, we no longer cast `script_lines` to Fixnum. ## Background Ref https://github.com/ruby/ruby/pull/10618 In the PR above, we have decoupled IMEMO from `rb_ast_t`. This means we could lift the five-words-restriction of the structure that forced us to unionize `rb_ast_t *` and `FIXNUM` in one field. ## Relating refactor - Remove the second parameter of `rb_ruby_ast_new()` function ## Attention I will remove a code that assigns -1 to line_count, in `rb_binding_add_dynavars()` of vm.c, because I don't think it is necessary. But I will make another PR for this so that we can atomically revert in case I was wrong (See the comment on the code) | |||
| 2024-04-26 | Set `SCRIPT_LINES__` outside of parser | yui-knk | |
| Parser should not depend on functions defiend on "ruby_parser.c". | |||
| 2024-04-26 | [Universal parser] Decouple IMEMO from rb_ast_t | HASUMI Hitoshi | |
| This patch removes the `VALUE flags` member from the `rb_ast_t` structure making `rb_ast_t` no longer an IMEMO object. ## Background We are trying to make the Ruby parser generated from parse.y a universal parser that can be used by other implementations such as mruby. To achieve this, it is necessary to exclude VALUE and IMEMO from parse.y, AST, and NODE. ## Summary (file by file) - `rubyparser.h` - Remove the `VALUE flags` member from `rb_ast_t` - `ruby_parser.c` and `internal/ruby_parser.h` - Use TypedData_Make_Struct VALUE which wraps `rb_ast_t` `in ast_alloc()` so that GC can manage it - You can retrieve `rb_ast_t` from the VALUE by `rb_ruby_ast_data_get()` - Change the return type of `rb_parser_compile_XXXX()` functions from `rb_ast_t *` to `VALUE` - rb_ruby_ast_new() which internally `calls ast_alloc()` is to create VALUE vast outside ruby_parser.c - `iseq.c` and `vm_core.h` - Amend the first parameter of `rb_iseq_new_XXXX()` functions from `rb_ast_body_t *` to `VALUE` - This keeps the VALUE of AST on the machine stack to prevent being removed by GC - `ast.c` - Almost all change is replacement `rb_ast_t *ast` with `VALUE vast` (sorry for the big diff) - Fix `node_memsize()` - Now it includes `rb_ast_local_table_link`, `tokens` and script_lines - `compile.c`, `load.c`, `node.c`, `parse.y`, `proc.c`, `ruby.c`, `template/prelude.c.tmpl`, `vm.c` and `vm_eval.c` - Follow-up due to the above changes - `imemo.{c|h}` - If an object with `imemo_ast` appears, considers it a bug Co-authored-by: Nobuyoshi Nakada <[email protected]> | |||
| 2024-04-23 | Refactor parser compile functions | yui-knk | |
| Refactor parser compile functions to reduce the dependence on ruby functions. This commit includes these changes 1. Refactor `gets`, `input` and `gets_` of `parser_params` Parser needs two different data structure to get next line, function (`gets`) and input data (`input`). However `gets_` is used for both function (`call`) and input data (`ptr`). `call` is used for managing general callback function when `rb_ruby_parser_compile_generic` is used. `ptr` is used for managing the current pointer on String when `parser_compile_string` is used. This commit changes parser to used only `gets` and `input` then removes `gets_`. 2. Move parser_compile functions and `gets` functions from parse.y to ruby_parser.c This change reduces the dependence on ruby functions from parser. 3. Change ruby_parser and ripper to take care of `VALUE input` GC mark Move the responsibility of calling `rb_gc_mark` for `VALUE input` from parser to ruby_parser and ripper. `input` is arbitrary data pointer from the viewpoint of parser. 4. Introduce rb_parser_compile_array function Caller of `rb_parser_compile_generic` needs to take care about GC because ruby_parser doesn’t know about the detail of `lex_gets` and `input`. Introduce `rb_parser_compile_array` to reduce the complexity of ast.c. | |||
| 2024-04-20 | Remove unused function | yui-knk | |
| 2024-04-15 | [Universal parser] DeVALUE of p->debug_lines and ast->body.script_lines | HASUMI Hitoshi | |
| This patch is part of universal parser work. ## Summary - Decouple VALUE from members below: - `(struct parser_params *)->debug_lines` - `(rb_ast_t *)->body.script_lines` - Instead, they are now `rb_parser_ary_t *` - They can also be a `(VALUE)FIXNUM` as before to hold line count - `ISEQ_BODY(iseq)->variable.script_lines` remains VALUE - In order to do this, - Add `VALUE script_lines` param to `rb_iseq_new_with_opt()` - Introduce `rb_parser_build_script_lines_from()` to convert `rb_parser_ary_t *` into `VALUE` ## Other details - Extend `rb_parser_ary_t *`. It previously could only store `rb_parser_ast_token *`, now can store script_lines, too - Change tactics of building the top-level `SCRIPT_LINES__` in `yycompile0()` - Before: While parsing, each line of the script is added to `SCRIPT_LINES__[path]` - After: After `yyparse(p)`, `SCRIPT_LINES__[path]` will be built from `p->debug_lines` - Remove the second parameter of `rb_parser_set_script_lines()` to make it simple - Introduce `script_lines_free()` to be called from `rb_ast_free()` because the GC no longer takes care of the script_lines - Introduce `rb_parser_string_deep_copy()` in parse.y to maintain script_lines when `rb_ruby_parser_free()` called - With regard to this, please see *Future tasks* below ## Future tasks - Decouple IMEMO from `rb_ast_t *` - This lifts the five-members-restriction of Ruby object, - So we will be able to move the ownership of the `lex.string_buffer` from parser to AST - Then we remove `rb_parser_string_deep_copy()` to make the whole thing simple | |||
| 2024-04-11 | compile.c: use rb_enc_interned_str to reduce allocations | Jean Boussier | |
| The `rb_fstring(rb_enc_str_new())` pattern is inneficient because: - It passes a mutable string to `rb_fstring` so if it has to be interned it will first be duped. - It an equivalent interned string already exists, we allocated the string for nothing. With `rb_enc_interned_str` we either directly get the pre-existing string with 0 allocations, or efficiently directly intern the one we create without first duping it. | |||
| 2024-04-04 | Move shareable_constant_value logic from parse.y to compile.c | yui-knk | |
| 2024-04-02 | Remove `rb_imemo_tmpbuf_t` from parser | yui-knk | |
| No parser semantic value types are `VALUE` then no need to use imemo for managing semantic value stack anymore. | |||
| 2024-03-21 | Fix Ripper memory allocation size when enabled Universal Parser | S-H-GAMELINKS | |
| The size of `struct parser_params` is 8 bytes difference in `ripper_s_allocate` and `rb_ruby_parser_allocate` when the universal parser is enabled. This causes a situation where `*r->p` is not fully initialized in `ripper_s_allocate` as shown below. ```console (gdb) p *r->p $2 = {heap = 0x0, lval = 0x0, yylloc = 0x0, lex = {strterm = 0x0, gets = 0x0, input = 0, string_buffer = {head = 0x0, last = 0x0}, lastlin e = 0x0, nextline = 0x0, pbeg = 0x0, pcur = 0x0, pend = 0x0, ptok = 0x0, gets_ = {ptr = 0, call = 0x0}, state = EXPR_NONE, paren_nest = 0, lpar _seen = 0, debug = 0, has_shebang = 0, token_seen = 0, token_info_enabled = 0, error_p = 0, cr_seen = 0, value = 0, result = 0, parsing_thread = 0, s_value = 0, s_lvalue = 0, s_value_stack = 2097} ```` This seems to cause `double free or corruption (!prev)` and SEGV. So, fixing this by introduce `rb_ripper_parser_params_allocate` and `rb_ruby_parser_config` functions for Ripper, and `struct parser_params` same size is returned. | |||
| 2024-02-21 | Introduce NODE_REGX to manage regexp literal | yui-knk | |
| 2024-02-20 | [Feature #20257] Rearchitect Ripper | yui-knk | |
| Introduce another semantic value stack for Ripper so that Ripper can manage both Node and Ruby Object separately. This rearchitectutre of Ripper solves these issues. Therefore adding test cases for them. * [Bug 10436] https://bugs.ruby-lang.org/issues/10436 * [Bug 18988] https://bugs.ruby-lang.org/issues/18988 * [Bug 20055] https://bugs.ruby-lang.org/issues/20055 Checked the differences of `Ripper.sexp` for files under `/test/ruby` are only on test_pattern_matching.rb. The differences comes from the differences between `new_hash_pattern_tail` functions between parser and Ripper. Ripper `new_hash_pattern_tail` didn’t call `assignable` then `kw_rest_arg` wasn’t marked as local variable. This is also fixed by this commit. ``` --- a/./tmp/before/test_pattern_matching.rb +++ b/./tmp/after/test_pattern_matching.rb @@ -3607,7 +3607,7 @@ [:in, [:hshptn, nil, [], [:var_field, [:@ident, “a”, [984, 13]]]], [[:binary, - [:vcall, [:@ident, “a”, [985, 10]]], + [:var_ref, [:@ident, “a”, [985, 10]]], :==, [:hash, nil]]], nil]]], @@ -3662,7 +3662,7 @@ [:in, [:hshptn, nil, [], [:var_field, [:@ident, “a”, [993, 13]]]], [[:binary, - [:vcall, [:@ident, “a”, [994, 10]]], + [:var_ref, [:@ident, “a”, [994, 10]]], :==, [:hash, [:assoclist_from_args, @@ -3813,7 +3813,7 @@ [:command, [:@ident, “raise”, [1022, 10]], [:args_add_block, - [[:vcall, [:@ident, “b”, [1022, 16]]]], + [[:var_ref, [:@ident, “b”, [1022, 16]]]], false]]], [:else, [[:var_ref, [:@kw, “true”, [1024, 10]]]]]]]], nil, @@ -3876,7 +3876,7 @@ [:@int, “0”, [1033, 15]]], :“&&“, [:binary, - [:vcall, [:@ident, “b”, [1033, 20]]], + [:var_ref, [:@ident, “b”, [1033, 20]]], :==, [:hash, nil]]]], nil]]], @@ -3946,7 +3946,7 @@ [:@int, “0”, [1042, 15]]], :“&&“, [:binary, - [:vcall, [:@ident, “b”, [1042, 20]]], + [:var_ref, [:@ident, “b”, [1042, 20]]], :==, [:hash, [:assoclist_from_args, @@ -5206,7 +5206,7 @@ [[:assoc_new, [:@label, “c:“, [1352, 22]], [:@int, “0”, [1352, 25]]]]]], - [:vcall, [:@ident, “r”, [1352, 29]]]], + [:var_ref, [:@ident, “r”, [1352, 29]]]], false]]], [:binary, [:call, @@ -5299,7 +5299,7 @@ [:assoc_new, [:@label, “c:“, [1367, 34]], [:@int, “0”, [1367, 37]]]]]], - [:vcall, [:@ident, “r”, [1367, 41]]]], + [:var_ref, [:@ident, “r”, [1367, 41]]]], false]]], [:binary, [:call, @@ -5931,7 +5931,7 @@ [:in, [:hshptn, nil, [], [:var_field, [:@ident, “r”, [1533, 11]]]], [[:binary, - [:vcall, [:@ident, “r”, [1534, 8]]], + [:var_ref, [:@ident, “r”, [1534, 8]]], :==, [:hash, [:assoclist_from_args, ``` | |||
| 2024-02-09 | Remove ruby object from string nodes | yui-knk | |
| String nodes holds ruby string object on `VALUE nd_lit`. This commit changes it to `struct rb_parser_string *string` to reduce dependency on ruby object. Sometimes these strings are concatenated with other string therefore string concatenate functions are needed. | |||
| 2024-01-31 | Introduced `rb_node_const_decl_val` function | S.H | |
| Introduce `rb_node_const_decl_val` function to allow `rb_ary_join` and `rb_ary_reverse` functions to be removed from Universal Parser. | |||
| 2024-01-27 | Introduce `NODE_ENCODING` | S.H | |
| `__ENCODING__ `was managed by `NODE_LIT` with Encoding object. Introduce `NODE_ENCODING` for 1. `__ENCODING__` is detectable from AST Node. 2. Reduce dependency Ruby object for parse.y | |||
| 2024-01-23 | Make lastline and nextline to be rb_parser_string | yui-knk | |
| This commit changes `struct parser_params` lastline and nextline from `VALUE` (String object) to `rb_parser_string_t *` so that dependency on Ruby Object is reduced. `parser_string_buffer_t string_buffer` is added to `struct parser_params` to manage `rb_parser_string_t` pointers of each line. All allocated line strings are freed in `rb_ruby_parser_free`. | |||
| 2024-01-12 | Statically allocate parser config | yui-knk | |
| 2024-01-09 | Introduce NODE_SYM to manage symbol literal | yui-knk | |
| `:sym` was managed by `NODE_LIT` with `Symbol` object. This commit introduces `NODE_SYM` so that 1. Symbol literal is detectable from AST Node 2. Reduce dependency on ruby object | |||
| 2024-01-08 | Change numeric node value functions argument to `NODE *` | yui-knk | |
| Change the argument to align with other node value functions like `rb_node_line_lineno_val`. | |||
| 2024-01-07 | Introduce Numeric Node's | S-H-GAMELINKS | |
| 2024-01-02 | Introduce NODE_FILE | yui-knk | |
| `__FILE__` was managed by `NODE_STR` with `String` object. This commit introduces `NODE_FILE` and `struct rb_parser_string` so that 1. `__FILE__` is detectable from AST Node 2. Reduce dependency ruby object | |||
| 2023-12-29 | Introduce NODE_LINE | yui-knk | |
| `__LINE__` was managed by `NODE_LIT` with `Integer` object. This commit introduces `NODE_LINE` so that 1. `__LINE__` is detectable from AST Node 2. Reduce dependency ruby object | |||
| 2023-11-08 | Export functions used for builtins | Nobuyoshi Nakada | |
| 2023-08-25 | Move SCRIPT_LINES__ away from parse.y | Nobuyoshi Nakada | |
| Notes: Merged: https://github.com/ruby/ruby/pull/8289 | |||
| 2023-08-25 | do not redefine a typedef | 卜部昌平 | |
| duplicated typedef declaration was not allowed in C99. Notes: Merged: https://github.com/ruby/ruby/pull/8274 | |||
| 2023-06-12 | [Feature #19719] Universal Parser | yui-knk | |
| Introduce Universal Parser mode for the parser. This commit includes these changes: * Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions are passed via `struct rb_parser_config_struct` when this macro is enabled. * Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu. Notes: Merged: https://github.com/ruby/ruby/pull/7927 | |||
 |
index : ruby.git | |
| The Ruby Programming Language |
| summaryrefslogtreecommitdiff |