summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)Author
46 hoursRefactor generic fields to use `T_IMEMO/fields` objects.Jean Boussier
Followup: https://github.com/ruby/ruby/pull/13589 This simplify a lot of things, as we no longer need to manually manage the memory, we can use the Read-Copy-Update pattern and avoid numerous race conditions. Co-Authored-By: Étienne Barrié <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/13626
46 hoursRefactor `rb_imemo_fields_new` to not assume T_CLASSJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13626
46 hoursRename `imemo_class_fields` -> `imemo_fields`Jean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13626
2 daysOptimize `benchmark/vm_ivar_of_class`Jean Boussier
``` compare-ruby: ruby 3.5.0dev (2025-06-17T08:45:40Z master e9d35671d2) +PRISM [arm64-darwin24] last_commit=[ruby/json] Fix a typo built-ruby: ruby 3.5.0dev (2025-06-17T09:27:05Z opt-getivar-for-cl.. ed1d7cd778) +PRISM [arm64-darwin24] | |compare-ruby|built-ruby| |:---------------------|-----------:|---------:| |vm_ivar_of_class_set | 12.306M| 13.957M| | | -| 1.13x| |vm_ivar_of_class | 16.167M| 24.029M| | | -| 1.49x| ``` Notes: Merged: https://github.com/ruby/ruby/pull/13639
5 daysFix blocking operation cancellation. (#13614)Samuel Williams
Expose `rb_thread_resolve_unblock_function` internally. Notes: Merged-By: ioquatix <[email protected]>
6 daysAdd SHAPE_ID_HAS_IVAR_MASK for quick ivar checkJean Boussier
This allow checking if an object has ivars with just a shape_id mask. Notes: Merged: https://github.com/ruby/ruby/pull/13606
7 daysAdd a new_thread flag to rb_interrupt_execJohn Hawthorn
Previously rb_ractor_interrupt_exec would use an intermediate function to create a new thread with the actual target function, replacing the data being passed in with a piece of malloc memory holding the "next" function and the original data. Because of this, passing rb_interrupt_exec_flag_value_data to rb_ractor_interrupt_exec didn't have the intended effect of allowing data to be passed in and marked. This commit adds a rb_interrupt_exec_flag_new_thread flag, which both simplifies the implementation and allows the original data to be marked. Notes: Merged: https://github.com/ruby/ruby/pull/13531
7 daysMake setting and accessing class ivars lock-freeJean Boussier
Now that class fields have been deletated to a T_IMEMO/class_fields when we're in multi-ractor mode, we can read and write class instance variable in an atomic way using Read-Copy-Update (RCU). Note when in multi-ractor mode, we always use RCU. In theory we don't need to, instead if we ensured the field is written before the shape is updated it would be safe. Benchmark: ```ruby Warning[:experimental] = false class Foo @foo = 1 @bar = 2 @baz = 3 @egg = 4 @spam = 5 class << self attr_reader :foo, :bar, :baz, :egg, :spam end end ractors = 8.times.map do Ractor.new do 1_000_000.times do Foo.bar + Foo.baz * Foo.egg - Foo.spam end end end if Ractor.method_defined?(:value) ractors.each(&:value) else ractors.each(&:take) end ``` This branch vs Ruby 3.4: ```bash $ hyperfine -w 1 'ruby --disable-all ../test.rb' './miniruby ../test.rb' Benchmark 1: ruby --disable-all ../test.rb Time (mean ± σ): 3.162 s ± 0.071 s [User: 2.783 s, System: 10.809 s] Range (min … max): 3.093 s … 3.337 s 10 runs Benchmark 2: ./miniruby ../test.rb Time (mean ± σ): 208.7 ms ± 4.6 ms [User: 889.7 ms, System: 6.9 ms] Range (min … max): 202.8 ms … 222.0 ms 14 runs Summary ./miniruby ../test.rb ran 15.15 ± 0.47 times faster than ruby --disable-all ../test.rb ``` Notes: Merged: https://github.com/ruby/ruby/pull/13594
7 daysFix class instance variable inside namespacesJean Boussier
Now that classes fields are delegated to an object with its own shape_id, we no longer need to mark all classes as TOO_COMPLEX. Notes: Merged: https://github.com/ruby/ruby/pull/13595
7 daysTurn `rb_classext_t.fields` into a T_IMEMO/class_fieldsJean Boussier
This behave almost exactly as a T_OBJECT, the layout is entirely compatible. This aims to solve two problems. First, it solves the problem of namspaced classes having a single `shape_id`. Now each namespaced classext has an object that can hold the namespace specific shape. Second, it open the door to later make class instance variable writes atomics, hence be able to read class variables without locking the VM. In the future, in multi-ractor mode, we can do the write on a copy of the `fields_obj` and then atomically swap it. Considerations: - Right now the `RClass` shape_id is always synchronized, but with namespace we should likely mark classes that have multiple namespace with a specific shape flag. Notes: Merged: https://github.com/ruby/ruby/pull/13411
8 daysRefactor the last references to `rb_shape_t`Jean Boussier
The type isn't opaque because Ruby isn't often compiled with LTO, so for optimization purpose it's better to allow as much inlining as possible. However ideally only `shape.c` and `shape.h` should deal with the actual struct, and everything else should just deal with opaque `shape_id_t`. Notes: Merged: https://github.com/ruby/ruby/pull/13586
10 daysOptimize callcache invalidation for refinementsalpaca-tc
Fixes [Bug #21201] This change addresses a performance regression where defining methods inside `refine` blocks caused severe slowdowns. The issue was due to `rb_clear_all_refinement_method_cache()` triggering a full object space scan via `rb_objspace_each_objects` to find and invalidate affected callcaches, which is very inefficient. To fix this, I introduce `vm->cc_refinement_table` to track callcaches related to refinements. This allows us to invalidate only the necessary callcaches without scanning the entire heap, resulting in significant performance improvement. Notes: Merged: https://github.com/ruby/ruby/pull/13077
12 daysshape.c: ensure heap_index is consistent for complex shapesJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13556
13 days`rb_io_blocking_operation_exit` should not execute with pending interrupts.Samuel Williams
Notes: Merged: https://github.com/ruby/ruby/pull/13437
14 daysRefactor raw accesses to rb_shape_t.capacityJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13524
2025-06-04Get rid of frozen shapes.Jean Boussier
Instead `shape_id_t` higher bits contain flags, and the first one tells whether the shape is frozen. This has multiple benefits: - Can check if a shape is frozen with a single bit check instead of dereferencing a pointer. - Guarantees it is always possible to transition to frozen. - This allow reclaiming `FL_FREEZE` (not done yet). The downside is you have to be careful to preserve these flags when transitioning. Notes: Merged: https://github.com/ruby/ruby/pull/13289
2025-06-02Remove unused RBASIC_RESET_FLAGSPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/13476
2025-05-29Read {max_iv,variation}_count from prime classextJohn Hawthorn
MAX_IV_COUNT is a hint which determines the size of variable width allocation we should use for a given class. We don't need to scope this by namespace, if we end up with larger builtin objects on some namespaces that isn't a user-visible problem, just extra memory use. Similarly variation_count is used to track if a given object has had too many branches in shapes it has used, and to use too_complex when that happens. That's also just a hint, so we can use the same value across namespaces without it being visible to users. Previously variation_count was being incremented (written to) on the RCLASS_EXT_READABLE ext, which seems incorrect if we wanted it to be different across namespaces Notes: Merged: https://github.com/ruby/ruby/pull/13434
2025-05-28Use flag for RCLASS_IS_INITIALIZEDJohn Hawthorn
Previously we used a flag to set whether a module was uninitialized. When checked whether a class was initialized, we first had to check that it had a non-zero superclass, as well as that it wasn't BasicObject. With the advent of namespaces, RCLASS_SUPER is now an expensive operation, and though we could just check for the prime superclass, we might as well take this opportunity to use a flag so that we can perform the initialized check with as few instructions as possible. It's possible in the future that we could prevent uninitialized classes from being available to the user, but currently there are a few ways to do that. Notes: Merged: https://github.com/ruby/ruby/pull/13443
2025-05-26Add shape_id to RBasic under 32 bitJohn Hawthorn
This makes `RBobject` `4B` larger on 32 bit systems but simplifies the implementation a lot. [Feature #21353] Co-authored-by: Jean Boussier <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/13341
2025-05-25Use RB_VM_LOCKINGNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/13439
2025-05-23Stricter assert for RCLASS_ALLOCATORJohn Hawthorn
I'd like to make this only valid to T_CLASS also, but currently it is called in some places for T_ICLASS and expected to return 0. Notes: Merged: https://github.com/ruby/ruby/pull/13416
2025-05-23Only call RCLASS_SET_ALLOCATOR on T_CLASS objectsJohn Hawthorn
It's invalid to set an allocator on a T_ICLASS or T_MODULE, as those use the other fields from the union. Notes: Merged: https://github.com/ruby/ruby/pull/13416
2025-05-23Don't use namespaced classext for superclassesJohn Hawthorn
Superclasses can't be modified by user code, so do not need namespace indirection. For example Object.superclass is always BasicObject, no matter what modules are included onto it. Notes: Merged: https://github.com/ruby/ruby/pull/13420
2025-05-23Allow `IO#close` to interrupt IO operations on fibers using ↵Samuel Williams
`fiber_interrupt` hook. (#12839) Notes: Merged-By: ioquatix <[email protected]>
2025-05-15Ensure that forked process do not see invalid blocking operations. (#13343)Samuel Williams
Notes: Merged-By: ioquatix <[email protected]>
2025-05-14Fix `object_id` for classes and modules in namespace contextJean Boussier
Given classes and modules have a different set of fields in every namespace, we can't store the object_id in fields for them. Given that some space was freed in `RClass` we can store it there instead. Notes: Merged: https://github.com/ruby/ruby/pull/13315
2025-05-14Reclaim one `VALUE` from `rb_classext_t` by shrinking `super_classdepth`Jean Boussier
By making `super_classdepth` `uint16_t`, classes and modules can now fit in 160B slots again. The downside of course is that before `super_classdepth` was large enough we never had to care about overflow, as you couldn't realistically create enough classes to ever go over it. With this change, while it is stupid, you could realistically create an ancestor chain containing 65k classes and modules. Notes: Merged: https://github.com/ruby/ruby/pull/13319
2025-05-13Reclaim one `VALUE` from `rb_classext_t`Jean Boussier
The `includer` field is only used for `T_ICLASS`, so by moving it into the existing union we can save one `VALUE` per class and module. Notes: Merged: https://github.com/ruby/ruby/pull/13316
2025-05-13Make `waiting_fd` behaviour per-IO. (#13127)Samuel Williams
- `rb_thread_fd_close` is deprecated and now a no-op. - IO operations (including close) no longer take a vm-wide lock. Notes: Merged-By: ioquatix <[email protected]>
2025-05-13variable.c: Refactor rb_obj_field_* to take shape_id_tJean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13314
2025-05-11Describe the basic documents of NamespaceSatoshi Tagomori
2025-05-11Show experimental warning when namespace is enabledSatoshi Tagomori
2025-05-11Delete code for debugging namespaceSatoshi Tagomori
2025-05-11Rename RCLASS_EXT() macro to RCLASS_EXT_PRIME() to prevent using it wronglySatoshi Tagomori
The macro RCLASS_EXT() accesses the prime classext directly, but it can be valid only in a limited situation when namespace is enabled. So, to prevent using RCLASS_EXT() in the wrong way, rename the macro and let the developer check it is ok to access the prime classext or not.
2025-05-11Compact prime classext readable/writable flagsSatoshi Tagomori
To make RClass size smaller, move flags of prime classext readable/writable to: readable - use ns_classext_tbl is NULL or not (if NULL, it's readable) writable - use FL_USER2 of RBasic flags
2025-05-11initialize method tables before any GC chanceSatoshi Tagomori
2025-05-11avoid calling ZALLOC after NEWOBJ_OF for RClass: need to return RClass not ↵Satoshi Tagomori
promoted
2025-05-11Remove unnecessary prototype declarationsYusuke Endoh
``` internal/class.h:158:20: warning: ‘RCLASS_SET_CLASSEXT_TABLE’ declared ‘static’ but never defined [-Wunused-function] 158 | static inline void RCLASS_SET_CLASSEXT_TABLE(VALUE obj, st_table *tbl); | ^~~~~~~~~~~~~~~~~~~~~~~~~ internal/class.h:271:20: warning: ‘RCLASS_WRITE_SUBCLASSES’ declared ‘static’ but never defined [-Wunused-function] 271 | static inline void RCLASS_WRITE_SUBCLASSES(VALUE klass, rb_subclass_anchor_t *anchor); | ^~~~~~~~~~~~~~~~~~~~~~~ ```
2025-05-11namespace on readSatoshi Tagomori
2025-05-10Rename `rb_field_get` -> `rb_obj_field_get`Jean Boussier
To be consistent with `rb_obj_field_set`. Notes: Merged: https://github.com/ruby/ruby/pull/13297
2025-05-09Rename `rb_shape_obj_too_complex` -> `rb_shape_obj_too_complex_p`Jean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13283
2025-05-09Rename `rb_shape_get_shape_by_id` -> `RSHAPE`Jean Boussier
Notes: Merged: https://github.com/ruby/ruby/pull/13283
2025-05-08Move `object_id` in object fields.Jean Boussier
And get rid of the `obj_to_id_tbl` It's no longer needed, the `object_id` is now stored inline in the object alongside instance variables. We still need the inverse table in case `_id2ref` is invoked, but we lazily build it by walking the heap if that happens. The `object_id` concern is also no longer a GC implementation concern, but a generic implementation. Co-Authored-By: Matt Valentine-House <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/13159
2025-05-08Rename `ivptr` -> `fields`, `next_iv_index` -> `next_field_index`Jean Boussier
Ivars will longer be the only thing stored inline via shapes, so keeping the `iv_index` and `ivptr` names would be confusing. Instance variables won't be the only thing stored inline via shapes, so keeping the `ivptr` name would be confusing. `field` encompass anything that can be stored in a VALUE array. Similarly, `gen_ivtbl` becomes `gen_fields_tbl`. Notes: Merged: https://github.com/ruby/ruby/pull/13159
2025-04-28Support Marshal.{dump,load} for core SetJeremy Evans
This was missed when adding core Set, because it's handled implicitly for T_OBJECT. Keep marshal compatibility between core Set and stdlib Set, so you can unmarshal core Set with stdlib Set and vice versa. Co-authored-by: Nobuyoshi Nakada <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/13185 Merged-By: jeremyevans <[email protected]>
2025-04-26Use `set_table` to track const cachesJean Boussier
Now that we have a `set_table` implementation, we can use it to track const caches and save some memory. We could even save some more memory if `numtable` didn't store a copy of the `hash` and instead recomputed it every time, but this is a quick win. Notes: Merged: https://github.com/ruby/ruby/pull/13184
2025-04-26Implement Set as a core classJeremy Evans
Set has been an autoloaded standard library since Ruby 3.2. The standard library Set is less efficient than it could be, as it uses Hash for storage, which stores unnecessary values for each key. Implementation details: * Core Set uses a modified version of `st_table`, named `set_table`. than `s/st_/set_/`, the main difference is that the stored records do not have values, making them 1/3 smaller. `st_table_entry` stores `hash`, `key`, and `record` (value), while `set_table_entry` only stores `hash` and `key`. This results in large sets using ~33% less memory compared to stdlib Set. For small sets, core Set uses 12% more memory (160 byte object slot and 64 malloc bytes, while stdlib set uses 40 for Set and 160 for Hash). More memory is used because the set_table is embedded and 72 bytes in the object slot are currently wasted. Hopefully we can make this more efficient and have it stored in an 80 byte object slot in the future. * All methods are implemented as cfuncs, except the pretty_print methods, which were moved to `lib/pp.rb` (which is where the pretty_print methods for other core classes are defined). As is typical for core classes, internal calls call C functions and not Ruby methods. For example, to check if something is a Set, `rb_obj_is_kind_of` is used, instead of calling `is_a?(Set)` on the related object. * Almost all methods use the same algorithm that the pure-Ruby implementation used. The exception is when calling `Set#divide` with a block with 2-arity. The pure-Ruby method used tsort to implement this. I developed an algorithm that only allocates a single intermediate hash and does not need tsort. * The `flatten_merge` protected method is no longer necessary, so it is not implemented (it could be). * Similar to Hash/Array, subclasses of Set are no longer reflected in `inspect` output. * RDoc from stdlib Set was moved to core Set, with minor updates. This includes a comprehensive benchmark suite for all public Set methods. As you would expect, the native version is faster in the vast majority of cases, and multiple times faster in many cases. There are a few cases where it is significantly slower: * Set.new with no arguments (~1.6x) * Set#compare_by_identity for small sets (~1.3x) * Set#clone for small sets (~1.5x) * Set#dup for small sets (~1.7x) These are slower as Set does not currently use the AR table optimization that Hash does, so a new set_table is initialized for each call. I'm not sure it's worth the complexity to have an AR table-like optimization for small sets (for hashes it makes sense, as small hashes are used everywhere in Ruby). The rbs and repl_type_completor bundled gems will need updates to support core Set. The pull request marks them as allowed failures. This passes all set tests with no changes. The following specs needed modification: * Modifying frozen set error message (changed for the better) * `Set#divide` when passed a 2-arity block no longer yields the same object as both the first and second argument (this seems like an issue with the previous implementation). * Set-like objects that override `is_a?` such that `is_a?(Set)` return `true` are no longer treated as Set instances. * `Set.allocate.hash` is no longer the same as `nil.hash` * `Set#join` no longer calls `Set#to_a` (it calls the underlying C function). * `Set#flatten_merge` protected method is not implemented. Previously, `set.rb` added a `SortedSet` autoload, which loads `set/sorted_set.rb`. This replaces the `Set` autoload in `prelude.rb` with a `SortedSet` autoload, but I recommend removing it and `set/sorted_set.rb`. This moves `test/set/test_set.rb` to `test/ruby/test_set.rb`, reflecting that switch to a core class. This does not move the spec files, as I'm not sure how they should be handled. Internally, this uses the st_* types and functions as much as possible, and only adds set_* types and functions as needed. The underlying set_table implementation is stored in st.c, but there is no public C-API for it, nor is there one planned, in order to keep the ability to change the internals going forward. For internal uses of st_table with Qtrue values, those can probably be replaced with set_table. To do that, include internal/set_table.h. To handle symbol visibility (rb_ prefix), internal/set_table.h uses the same macro approach that include/ruby/st.h uses. The Set class (rb_cSet) and all methods are defined in set.c. There isn't currently a C-API for the Set class, though C-API functions can be added as needed going forward. Implements [Feature #21216] Co-authored-by: Jean Boussier <[email protected]> Co-authored-by: Oliver Nutter <[email protected]>
2025-04-19Tidy up `rb_io_fptr_finalize`. (#13136)Samuel Williams
Notes: Merged-By: ioquatix <[email protected]>
2025-04-19Ensure `struct rb_io` is passed through to `thread.c`. (#13134)Samuel Williams
Notes: Merged-By: ioquatix <[email protected]>