[ruby-core:102787] [Ruby master Bug#17679] Ractor incoming channel can consume unlimited resources
From:
marcandre-ruby-core@...
Date:
2021-03-09 01:36:43 UTC
List:
ruby-core #102787
Issue #17679 has been updated by marcandre (Marc-Andre Lafortune).
It's not clear to me that this should be implemented at the Ractor level.
Both suggested approaches can be handled in Ruby, for example using an intermediary Ractor...
```ruby
DONE = Object.new.freeze
MIDDLEMAN = Ractor.new do
on_queue = 0
loop do
message = Ractor.receive
if message == DONE
on_queue -= 1
else
if (on_queue > 32_000)
puts "Too many requests, skipping #{message}"
next
end
on_queue += 1
DOER.send(message)
end
end
end
DOER = Ractor.new do
loop do
message = Ractor.receive
sleep 0.01
puts "Processed #{message}"
MIDDLEMAN.send(DONE)
end
end
counter = 0
while true
counter += 1
MIDDLEMAN.send(counter)
end
```
If/when a non-blocking `receive` will be available, the receiving ractor could also handle it's waiting queue internally
----------------------------------------
Bug #17679: Ractor incoming channel can consume unlimited resources
https://bugs.ruby-lang.org/issues/17679#change-90806
* Author: marcotc (Marco Costa)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
## Background
In the [ddtrace](https://github.com/DataDog/dd-trace-rb) gem, we want to move telemetry trace sending to a separate background Ractor. We’re concerned that if something goes wrong/gets delayed in this background Ractor, more and more data will accumulate in the send/receive channel until the Ruby VM crashes because it runs out of memory.
## How to reproduce (Ruby version & script)
```ruby
receiver_ractor = Ractor.new do
loop do
message = Ractor.receive
sleep 1
puts "Processed #{message}"
end
end
counter = 0
while true
counter += 1
receiver_ractor.send(counter)
end
```
## Expectation and result
The result is that the Ruby VM crashes due to out of memory.
We expect the Ruby VM to not crash.
## Suggested solutions
Some ideas on how this can be improved:
* Having a way for the sender of data to detect if the receiver Ractor is falling behind (approximate size of queue, timestamp of last processed item, or similar?).
* Having a way to limit the Ractor message receive buffer.
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>