[ruby-core:103035] [Ruby master Bug#17679] Ractor incoming channel can consume unlimited resources
From:
knuckles@...
Date:
2021-03-26 09:17:00 UTC
List:
ruby-core #103035
Issue #17679 has been updated by ivoanjo (Ivo Anjo).
That's a reasonable point, @marcandre. I also did tried something similar at https://ivoanjo.me/blog/2021/02/14/ractor-experiments-safe-async/ .
Our concern at Datadog (I'm a colleague of @marcotc) is that adding these middle layer threads/queues/ractors is error-prone, and this seems like something that every Ractor user may need, so it can probably be solved much cleaner by Ruby itself.
For instance, it really looks like during enqueing of messages in https://github.com/ruby/ruby/blob/9143d21b1bf2f16b1e847d569a588510726d8860/ractor.c#L408 the sender already checks the size of the queue anyway, so having the option to back out of the queue was at a given size seems to be a couple of lines way.
----------------------------------------
Bug #17679: Ractor incoming channel can consume unlimited resources
https://bugs.ruby-lang.org/issues/17679#change-91099
* Author: marcotc (Marco Costa)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
## Background
In the [ddtrace](https://github.com/DataDog/dd-trace-rb) gem, we want to move telemetry trace sending to a separate background Ractor. We’re concerned that if something goes wrong/gets delayed in this background Ractor, more and more data will accumulate in the send/receive channel until the Ruby VM crashes because it runs out of memory.
## How to reproduce (Ruby version & script)
```ruby
receiver_ractor = Ractor.new do
loop do
message = Ractor.receive
sleep 1
puts "Processed #{message}"
end
end
counter = 0
while true
counter += 1
receiver_ractor.send(counter)
end
```
## Expectation and result
The result is that the Ruby VM crashes due to out of memory.
We expect the Ruby VM to not crash.
## Suggested solutions
Some ideas on how this can be improved:
* Having a way for the sender of data to detect if the receiver Ractor is falling behind (approximate size of queue, timestamp of last processed item, or similar?).
* Having a way to limit the Ractor message receive buffer.
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>