summaryrefslogtreecommitdiff
path: root/bootstraptest/test_ractor.rb
AgeCommit message (Collapse)Author
5 days`Ractor#take` and warnKoichi Sasada
`Ractor#take` was deprecated but some libraries can use it as an alias for `Ractor#value` (i.e., to wait for a Ractor's temrination and retrieve its result). Therefore `Ractor#take` is simply an alias for `Ractor#value`. This method will remain available until the end of August 2025, unless there is further discussion. Notes: Merged: https://github.com/ruby/ruby/pull/13512
10 days`Ractor::Port`Koichi Sasada
* Added `Ractor::Port` * `Ractor::Port#receive` (support multi-threads) * `Rcator::Port#close` * `Ractor::Port#closed?` * Added some methods * `Ractor#join` * `Ractor#value` * `Ractor#monitor` * `Ractor#unmonitor` * Removed some methods * `Ractor#take` * `Ractor.yield` * Change the spec * `Racotr.select` You can wait for multiple sequences of messages with `Ractor::Port`. ```ruby ports = 3.times.map{ Ractor::Port.new } ports.map.with_index do |port, ri| Ractor.new port,ri do |port, ri| 3.times{|i| port << "r#{ri}-#{i}"} end end p ports.each{|port| pp 3.times.map{port.receive}} ``` In this example, we use 3 ports, and 3 Ractors send messages to them respectively. We can receive a series of messages from each port. You can use `Ractor#value` to get the last value of a Ractor's block: ```ruby result = Ractor.new do heavy_task() end.value ``` You can wait for the termination of a Ractor with `Ractor#join` like this: ```ruby Ractor.new do some_task() end.join ``` `#value` and `#join` are similar to `Thread#value` and `Thread#join`. To implement `#join`, `Ractor#monitor` (and `Ractor#unmonitor`) is introduced. This commit changes `Ractor.select()` method. It now only accepts ports or Ractors, and returns when a port receives a message or a Ractor terminates. We removes `Ractor.yield` and `Ractor#take` because: * `Ractor::Port` supports most of similar use cases in a simpler manner. * Removing them significantly simplifies the code. We also change the internal thread scheduler code (thread_pthread.c): * During barrier synchronization, we keep the `ractor_sched` lock to avoid deadlocks. This lock is released by `rb_ractor_sched_barrier_end()` which is called at the end of operations that require the barrier. * fix potential deadlock issues by checking interrupts just before setting UBF. https://bugs.ruby-lang.org/issues/21262 Notes: Merged: https://github.com/ruby/ruby/pull/13445
2025-05-26Don't copy FL_PROMOTED to new object in Ractor movePeter Zhu
We should not copy the FL_PROMOTED flag when we move an object in Ractor#send(move: true) because the newly created object may not be old. Notes: Merged: https://github.com/ruby/ruby/pull/13442
2025-05-23Fix moving old objects between RactorsPeter Zhu
The FL_PROMOTED flag was not copied when moving objects, causing assertions to fail when an old object is moved: gc/default/default.c:834: Assertion Failed: RVALUE_AGE_SET:age <= RVALUE_OLD_AGE Co-Authored-By: Luke Gruber <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/13415
2025-05-23lock vm around `rb_free_generic_ivar`Luke Gruber
Currently, this can be reproduced by: r = Ractor.new do a = [1, 2, 3] a.object_id a.dup # this frees the generic ivar for `object_id` on the copied object :done end r.take In debug builds, this hits an assertion failure without this fix. Notes: Merged: https://github.com/ruby/ruby/pull/13401
2025-05-21Fix Symbol#to_proc (rb_sym_to_proc) to be ractor safeLuke Gruber
In non-main ractors, don't use `sym_proc_cache`. It is not thread-safe to add to this array without a lock and also it leaks procs from one ractor to another. Instead, we create a new proc each time. If this results in poor performance we can come up with a solution later. Fixes [Bug #21354] Notes: Merged: https://github.com/ruby/ruby/pull/13380
2025-05-13Get ractor message passing working with > 1 thread sending/receiving values ↵Luke Gruber
in same ractor Rework ractors so that any ractor action (Ractor.receive, Ractor#send, Ractor.yield, Ractor#take, Ractor.select) will operate on the thread that called the action. It will put that thread to sleep if it's a blocking function and it needs to put it to sleep, and the awakening action (Ractor.yield, Ractor#send) will wake up the blocked thread. Before this change every blocking ractor action was associated with the ractor struct and its fields. If a ractor called Ractor.receive, its wait status was wait_receiving, and when another ractor calls r.send on it, it will look for that status in the ractor struct fields and wake it up. The problem was that what if 2 threads call blocking ractor actions in the same ractor. Imagine if 1 thread has called Ractor.receive and another r.take. Then, when a different ractor calls r.send on it, it doesn't know which ruby thread is associated to which ractor action, so what ruby thread should it schedule? This change moves some fields onto the ruby thread itself so that ruby threads are the ones that have ractor blocking statuses, and threads are then specifically scheduled when unblocked. Fixes [#17624] Fixes [#21037] Notes: Merged: https://github.com/ruby/ruby/pull/12633
2025-05-13Throw RuntimeError if getting/setting ractor local storage for non-main ractorlukeg
[Bug #19367] Notes: Merged: https://github.com/ruby/ruby/pull/7174
2025-05-09Fix crash when instantiating classes in RactorsPeter Zhu
[Bug #18119] When we create classes, it pushes the class to the subclass list of the superclass. This access needs to be synchronized because multiple Ractors may be creating classes with the same superclass, which would cause race conditions and cause the linked list to be corrupted. For example, we can reproduce with this script crashing: workers = (0...8).map do Ractor.new do loop do 100.times.map { Class.new } Ractor.yield nil end end end 100.times { Ractor.select(*workers) } With ASAN enabled, we can see that there are use-after-free errors: ==176013==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030000974f0 at pc 0x62f9e56f892d bp 0x7a503f1ffd90 sp 0x7a503f1ffd88 WRITE of size 8 at 0x5030000974f0 thread T4 #0 0x62f9e56f892c in rb_class_remove_from_super_subclasses class.c:149:24 #1 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9 #2 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21 #3 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13 #4 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9 #5 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13 #6 0x62f9e58fac93 in gc_start gc/default/default.c:6402:13 #7 0x62f9e58e8b69 in heap_prepare gc/default/default.c:2032:13 #8 0x62f9e58e8b69 in heap_next_free_page gc/default/default.c:2255:9 #9 0x62f9e58e8b69 in newobj_cache_miss gc/default/default.c:2362:38 ... 0x5030000974f0 is located 16 bytes inside of 24-byte region [0x5030000974e0,0x5030000974f8) freed by thread T4 here: #0 0x62f9e562f28a in free (miniruby+0x1fd28a) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233) #1 0x62f9e58ca2ab in rb_gc_impl_free gc/default/default.c:8102:9 #2 0x62f9e58ca2ab in ruby_sized_xfree gc.c:5029:13 #3 0x62f9e58ca2ab in ruby_xfree gc.c:5040:5 #4 0x62f9e56f88e6 in rb_class_remove_from_super_subclasses class.c:152:9 #5 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9 #6 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21 #7 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13 #8 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9 #9 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13 ... previously allocated by thread T5 here: #0 0x62f9e562f70d in calloc (miniruby+0x1fd70d) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233) #1 0x62f9e58c8e1a in calloc1 gc/default/default.c:1472:12 #2 0x62f9e58c8e1a in rb_gc_impl_calloc gc/default/default.c:8138:5 #3 0x62f9e58c8e1a in ruby_xcalloc_body gc.c:4964:12 #4 0x62f9e58c8e1a in ruby_xcalloc gc.c:4958:34 #5 0x62f9e56f906e in push_subclass_entry_to_list class.c:88:13 #6 0x62f9e56f906e in rb_class_subclass_add class.c:111:38 #7 0x62f9e56f906e in RCLASS_SET_SUPER internal/class.h:257:9 #8 0x62f9e56fca7a in make_metaclass class.c:786:5 #9 0x62f9e59db982 in rb_class_initialize object.c:2101:5 Notes: Merged: https://github.com/ruby/ruby/pull/13284
2025-05-08Raise error on take/send for Ractors in child processesAaron Patterson
Ractor objects that are available in a child process should raise a `Ractor::ClosedError` exception when called with `send` or `take` Co-authored-by: John Hawthorn <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/12982
2025-05-08Clean up Ractor cache after forkAaron Patterson
Ractors created in a parent process should be properly shut down in the child process. They need their cache cleared and status set to "terminated" Co-authored-by: John Hawthorn <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/12982
2025-05-08Rename `ivptr` -> `fields`, `next_iv_index` -> `next_field_index`Jean Boussier
Ivars will longer be the only thing stored inline via shapes, so keeping the `iv_index` and `ivptr` names would be confusing. Instance variables won't be the only thing stored inline via shapes, so keeping the `ivptr` name would be confusing. `field` encompass anything that can be stored in a VALUE array. Similarly, `gen_ivtbl` becomes `gen_fields_tbl`. Notes: Merged: https://github.com/ruby/ruby/pull/13159
2025-04-18Lock-free hash set for fstrings [Feature #21268]John Hawthorn
This implements a hash set which is wait-free for lookup and lock-free for insert (unless resizing) to use for fstring de-duplication. As highlighted in https://bugs.ruby-lang.org/issues/19288, heavy use of fstrings (frozen interned strings) can significantly reduce the parallelism of Ractors. I tried a few other approaches first: using an RWLock, striping a series of RWlocks (partitioning the hash N-ways to reduce lock contention), and putting a cache in front of it. All of these improved the situation, but were unsatisfying as all still required locks for writes (and granular locks are awkward, since we run the risk of needing to reach a vm barrier) and this table is somewhat write-heavy. My main reference for this was Cliff Click's talk on a lock free hash-table for java https://www.youtube.com/watch?v=HJ-719EGIts. It turns out this lock-free hash set is made easier to implement by a few properties: * We only need a hash set rather than a hash table (we only need keys, not values), and so the full entry can be written as a single VALUE * As a set we only need lookup/insert/delete, no update * Delete is only run inside GC so does not need to be atomic (It could be made concurrent) * I use rb_vm_barrier for the (rare) table rebuilds (It could be made concurrent) We VM lock (but don't require other threads to stop) for table rebuilds, as those are rare * The conservative garbage collector makes deferred replication easy, using a T_DATA object Another benefits of having a table specific to fstrings is that we compare by value on lookup/insert, but by identity on delete, as we only want to remove the exact string which is being freed. This is faster and provides a second way to avoid the race condition in https://bugs.ruby-lang.org/issues/21172. This is a pretty standard open-addressing hash table with quadratic probing. Similar to our existing st_table or id_table. Deletes (which happen on GC) replace existing keys with a tombstone, which is the only type of update which can occur. Tombstones are only cleared out on resize. Unlike st_table, the VALUEs are stored in the hash table itself (st_table's bins) rather than as a compact index. This avoids an extra pointer dereference and is possible because we don't need to preserve insertion order. The table targets a load factor of 2 (it is enlarged once it is half full). Notes: Merged: https://github.com/ruby/ruby/pull/12921
2025-04-15Add a test for moving composite object partsLuke Gruber
2025-04-07Fixed wrong condition to avoid flaky ractor_test.rbHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/13078
2025-04-06Fix the if condition to skip test_ractor.rb correctly (#13067)Naoto Ono
Follow-up for https://github.com/ruby/ruby/commit/a2b03ba7cb721d698bebee74c535dea4583a9c28 Notes: Merged-By: ono-max <[email protected]>
2025-04-04Ractor: revert to moving object bytes, but size pool awareJean Boussier
Using `rb_obj_clone` introduce other problems, such as `initialize_*` callbacks invocation in the context of the parent ractor. So we can revert back to copy the content of the object slots, but in a way that is aware of size pools. Notes: Merged: https://github.com/ruby/ruby/pull/13070
2025-03-31Don't preserve `object_id` when moving object to another RactorJean Boussier
That seemed like the logical thing to do to me, but ko1 disagree. Notes: Merged: https://github.com/ruby/ruby/pull/13008
2025-03-31Ractor: Fix moving embedded objectsJean Boussier
[Bug #20271] [Bug #20267] [Bug #20255] `rb_obj_alloc(RBASIC_CLASS(obj))` will always allocate from the basic 40B pool, so if `obj` is larger than `40B`, we'll create a corrupted object when we later copy the shape_id. Instead we can use the same logic than ractor copy, which is to use `rb_obj_clone`, and later ask the GC to free the original object. We then must turn it into a `T_OBJECT`, because otherwise just changing its class to `RactorMoved` leaves a lot of ways to keep using the object, e.g.: ``` a = [1, 2, 3] Ractor.new{}.send(a, move: true) [].concat(a) # Should raise, but wasn't. ``` If it turns out that `rb_obj_clone` isn't performant enough for some uses, we can always have carefully crafted specialized paths for the types that would benefit from it. Notes: Merged: https://github.com/ruby/ruby/pull/13008
2025-03-31Skip test_ractor.rb with ModGC workflow because this test is flakyHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/13015
2025-03-26Ractor.make_shareable(proc_obj) makes inner structure shareablelukeg
Proc objects are now traversed like other objects when making them shareable. Fixes [Bug #19372] Fixes [Bug #19374] Notes: Merged: https://github.com/ruby/ruby/pull/12977
2025-03-20Use atomic for method reference count [Bug #20934]John Hawthorn
This changes reference_count on rb_method_definition_struct into an atomic. Ractors can create additional references as part of `bind_call` or (presumably) similar. Because this can be done inside Ractors, we should use a lock or atomics so that we don't race and avoid incrementing. Co-authored-by: wanabe <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/12951
2025-02-13[Feature #21116] Extract RJIT as a third-party gemNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/12740
2025-01-10Revert "[MMTk/CI] Skip Ractor btests with MMTk"Peter Zhu
This reverts commit 58b4e249ed8d33fc78528bc77516d541c04d65f2. The bug that it encountered was fixed in f76d40789d2c7185df26e925636827c06eda7157. Notes: Merged: https://github.com/ruby/ruby/pull/12543
2025-01-10Don't check for presence of `ENV['GITHUB_WORKFLOW']`Peter Zhu
We already check whether `ENV['GITHUB_WORKFLOW']` is equal to `Compilations`, so we don't need to check that it's not nil. Notes: Merged: https://github.com/ruby/ruby/pull/12543
2025-01-10Rewrite Benchmark to Tempfile on bootstraptest/test_ractor.rbHiroshi SHIBATA
Notes: Merged: https://github.com/ruby/ruby/pull/12537
2024-12-24Fix ractor move of unshareable frozen objectsLuke Gruber
These objects didn't retain their frozen status after the move Bug [#19408] Notes: Merged: https://github.com/ruby/ruby/pull/9996
2024-12-13`Ractor.set_if_absent(key)`Koichi Sasada
to initialize ractor local storage in thread-safety. [Feature #20875] Notes: Merged: https://github.com/ruby/ruby/pull/12321
2024-12-06[MMTk/CI] Skip Ractor btests with MMTkMatt Valentine-House
currently these are flaky, so until we can make them more robust, we'll skip them for MMTk CI Notes: Merged: https://github.com/ruby/ruby/pull/12212
2024-11-08skip `SystemStackError`Koichi Sasada
with -O0 build, prism parser consumes a lot of machine stack and it doesn't work with minimum machine stack for threads, which specified with `RUBY_THREAD_MACHINE_STACK_SIZE=1`. So simply ignore `SystemStackError` for btest. Notes: Merged: https://github.com/ruby/ruby/pull/11142
2024-11-08support `require` in non-main RactorsKoichi Sasada
Many libraries should be loaded on the main ractor because of setting constants with unshareable objects and so on. This patch allows to call `requore` on non-main Ractors by asking the main ractor to call `require` on it. The calling ractor waits for the result of `require` from the main ractor. If the `require` call failed with some reasons, an exception objects will be deliverred from the main ractor to the calling ractor if it is copy-able. Same on `require_relative` and `require` by `autoload`. Now `Ractor.new{pp obj}` works well (the first call of `pp` requires `pp` library implicitly). [Feature #20627] Notes: Merged: https://github.com/ruby/ruby/pull/11142
2024-10-03Update bootstraptest test for colon-style hash inspecttompng
Notes: Merged: https://github.com/ruby/ruby/pull/10924
2024-07-12fix `defined?(@ivar)` with RactorsKoichi Sasada
`defined?(@ivar)` on the non main Ractor has two issues: 1. raising an exception ```ruby class C @iv1 = [] def self.defined_iv1 = defined?(@iv1) end Ractor.new{ p C.defined_iv1 #=> can not get unshareable values from instance variables of classes/modules from non-main Ractors (Ractor::IsolationError) }.take ``` -> Do not raise an exception but return `"instance-variable"` because it is defined. 2. returning `"instance-variable"` if there is not defined. ``` class C # @iv2 is not defined def self.defined_iv2 = defined?(@iv2) end Ractor.new{ p C.defined_iv2 #=> "instance-variable" }.take ``` -> returns `nil`
2024-05-05Fix interrupts during Ractor.selectLuke Gruber
Fixes [Bug #20168]
2024-04-26Skip a flaky Ractor test for YJITTakashi Kokubun
https://github.com/ruby/ruby/actions/runs/8852277192/job/24310631888 https://github.com/ruby/ruby/actions/runs/8851325573/job/24307638329 This seems like an existing, separate issue from what we're currently investigating. The `iseq->body->jit_entry` setup is not Ractor-safe? Let me suppress this for now and revisit this after resolving the ongoing issue.
2024-03-19Implement chilled stringsÉtienne Barrié
[Feature #20205] As a path toward enabling frozen string literals by default in the future, this commit introduce "chilled strings". From a user perspective chilled strings pretend to be frozen, but on the first attempt to mutate them, they lose their frozen status and emit a warning rather than to raise a `FrozenError`. Implementation wise, `rb_compile_option_struct.frozen_string_literal` is no longer a boolean but a tri-state of `enabled/disabled/unset`. When code is compiled with frozen string literals neither explictly enabled or disabled, string literals are compiled with a new `putchilledstring` instruction. This instruction is identical to `putstring` except it marks the String with the `STR_CHILLED (FL_USER3)` and `FL_FREEZE` flags. Chilled strings have the `FL_FREEZE` flag as to minimize the need to check for chilled strings across the codebase, and to improve compatibility with C extensions. Notes: - `String#freeze`: clears the chilled flag. - `String#-@`: acts as if the string was mutable. - `String#+@`: acts as if the string was mutable. - `String#clone`: copies the chilled flag. Co-authored-by: Jean Boussier <[email protected]>
2024-03-15Refactor frozen_string_literal check during compilationJean Boussier
In preparation for https://bugs.ruby-lang.org/issues/20205. The `frozen_string_literal` compilation option will no longer be a boolean but a tri-state: `on/off/default`.
2024-03-14Ensure test suite is compatible with --frozen-string-literalJean Boussier
As preparation for https://bugs.ruby-lang.org/issues/20205 making sure the test suite is compatible with frozen string literals is making things easier.
2024-01-22Skip a flaky Ractor test for YJITTakashi Kokubun
`[BUG] pthread_mutex_lock: Invalid argument (EINVAL)` doesn't seem like a fault of YJIT? https://github.com/ruby/ruby/actions/runs/7614455181/job/20736754975 https://github.com/ruby/ruby/actions/runs/7615316673/job/20739572487
2024-01-02Set Ractor moved object's shape to original object's shapeLuke Gruber
Fixes [Bug #19409]
2023-12-20moved object should not have a shape IDKoichi Sasada
fix [Bug #19917]
2023-12-16remove `Ractor::Selector` from Ruby levelKoichi Sasada
`Ractor::Selector` is not approved by Matz so remove it from Ruby-level. The implementation is used by `Ractor.select` so most of implementation was remaind and calling `rb_init_ractor_selector()`, `Ractor::Selector` will be defined. I will provide `ractor-selector` gem to try it.
2023-07-17Fix a typo [ci skip]Nobuyoshi Nakada
2023-03-15Fix indirect counter incrementNobuyoshi Nakada
`*pcnt++` just dereferences `pcnt` then increments the local variable, but has no side effect. Notes: Merged: https://github.com/ruby/ruby/pull/7496
2023-03-10RJIT: Skip a flaky test_thread test for nowTakashi Kokubun
and unskip a ractor test that was actually running
2023-03-06s/mjit/rjit/Takashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7462
2023-03-05Add more GC guardsTakashi Kokubun
Notes: Merged: https://github.com/ruby/ruby/pull/7448
2023-03-06add a test for RactorKoichi Sasada
Ractor should take care method cache invalidation. Added test will miss method cache on each method call. Notes: Merged: https://github.com/ruby/ruby/pull/7445
2023-03-03Another attempt to skip test_ractor on ci.rvm.jpTakashi Kokubun
This reverts commit 8d31a60f47fb053bcfe0c744a89bd666dae48539.
2023-03-03Fix a YJIT enablement checkTakashi Kokubun
This should be enough for `make test` and `make btest-ruby` while it doesn't work for `make btest`.