summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-08-21Backport warning feature for bundled gems from master (#11420)Hiroshi SHIBATA
* Make sure to always use the right `warn` * lib/bundled_gems.rb: more reliable caller detection The `2` skipped frames went out of sync and now it should be `3`. Rather than just update the offset, we can implement a way that is adaptative as long as all require decorators are also called require. Also we should compute the corresponding `uplevel` otherwise the warning will still point decorators. Co-authored-by: "Hiroshi SHIBATA" <[email protected]> * Warn ostruct for Ruby 3.5 * Warn pstore for Ruby 3.5 * Mark rdoc as bundled gems at Ruby 3.5 * Warn to use win32ole without Gemfile for Ruby 3.5 * EXACT list is mostly same as SINCE list on bundled gems. * Mark to warn fiddle as bundled gems for Ruby 3.5 * Mark to warn logger as bundled gems for Ruby 3.5 * We should use uplevel:2 in another case. Like the following scenario with bootsnap, that frames are same or smaller than frame_to_skip(=3). --- "/Users/hsbt/.local/share/rbenv/versions/3.3-dev/lib/ruby/3.3.0/bundled_gems.rb:69:in `block (2 levels) in replace_require'" "/Users/hsbt/.local/share/gem/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'" "test_warn_bootsnap.rb:11:in `<main>'" --- * Delete unnecessary rubocop disable comment * Show correct script name with sub-feature case * Skip to show script name with using ruby -r option * Don't show script name when bundle exec and call ruby script directly. * Pick word fix from 34adc07372c10170b8ca36111d216cbd8e4699be --------- Co-authored-by: David Rodríguez <[email protected]> Co-authored-by: Jean Boussier <[email protected]> Co-authored-by: Kentaro Takeyama <[email protected]>
2024-08-14Re-initialize vm->ractor.sched.lock after fork (#11372)John Hawthorn
[Bug #20633] Re-initialize vm->ractor.sched.lock after fork Previously under certain conditions it was possible to encounter a deadlock in the forked child process if ractor.sched.lock was held. Co-authored-by: Nathan Froyd <[email protected]>
2024-08-08Added bootstrap job for release workflowHiroshi SHIBATA
2024-08-06parse.y: const_decl_path don't replace destination node by a literal (#11314)Jean byroot Boussier
[Bug #20668] The `dest` node is assumed to be a `CDECL`, so overwriting it with a `LIT` cause a crash on the next iteration. Co-authored-by: Jean Boussier <[email protected]>
2024-07-30Update RubyGems 3.5.16 and Bundler 2.5.16 for Ruby 3.3 (#11252)Hiroshi SHIBATA
* Merge RubyGems-3.5.12 and Bundler-2.5.12 * Merge RubyGems-3.5.13 and Bundler-2.5.13 * Merge RubyGems-3.5.14 and Bundler-2.5.14 * Merge RubyGems-3.5.15 and Bundler-2.5.15 * Merge RubyGems-3.5.16 and Bundler-2.5.16
2024-07-30[Bug #20654] Fix floor and ceil when ndigits is large (#11277)Peter Zhu
* Fix floor when ndigits is large [Bug #20654] This commit fixes Integer#floor and Float#floor when the number is negative and ndigits is large such that 10**ndigits is a bignum. Previously, it would return 0 in such cases. However, this would cause unexpected behaviour such as: puts -1.floor(-5) # => -100000 puts -1.floor(-10) # => -10000000000 puts -1.floor(-20) # => 0 This commit changes the last result so that it will return -100000000000000000000. * Fix ceil when ndigits is large [Bug #20654] This commit fixes Integer#ceil and Float#ceil when the number is negative and ndigits is large such that 10**ndigits is a bignum. Previously, it would return 0 in such cases. However, this would cause unexpected behaviour such as: puts 1.ceil(-5) # => 100000 puts 1.ceil(-10) # => 10000000000 puts 1.ceil(-20) # => 0 This commit changes the last result so that it will return 100000000000000000000.
2024-07-29[Bug #20653] Fix memory leak in String#start_with? when regexp times out ↵Peter Zhu
(#11255) Fix memory leak in String#start_with? when regexp times out [Bug #20653] This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error. This fixes a memory leak in String#start_with when the regexp times out. For example: regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do str.start_with?(regex) rescue end puts `ps -o rss= -p #{$$}` end Before: 33216 51936 71152 81728 97152 103248 120384 133392 133520 133616 After: 14912 15376 15824 15824 16128 16128 16144 16144 16160 16160
2024-07-28[Bug #20088] Fix ARCH_FLAG for cross compilingNobuyoshi Nakada
2024-07-25[Bug #20650] Fix memory leak in Regexp capture group when timeout (#11244)Peter Zhu
Fix memory leak in Regexp capture group when timeout [Bug #20650] The capture group allocates memory that is leaked when it times out. For example: re = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001) str = "a" * 1000000 + "x" 10.times do 100.times do re =~ str rescue Regexp::TimeoutError end puts `ps -o rss= -p #{$$}` end Before: 34688 56416 78288 100368 120784 140704 161904 183568 204320 224800 After: 16288 16288 16880 16896 16912 16928 16944 17184 17184 17200
2024-07-22bundled_gems.rb: Add a fast path (#11221)Jean byroot Boussier
bundled_gems.rb: Add a fast path [Bug #20641] `Gem::BUNDLED_GEMS.warning?` adds a lot of extra work on top of `require`. When the call end up atually loading code the overhead is somewhat marginal. However it's not uncommon for code to go some late `require` in some paths, so it's expected that calling `require` with something already required is somewhat fast, and `bundled_gems.rb` breaks this assumption. To avoid this, we can have a fast path that in most case allow to short-circuit all the heavy computations. If we extract the feature basename and it doesn't match any of the bundled gems we care about we can return very early. With this change `require 'date'` is now only 1.33x slower on Ruby 3.3.3, than it was on Ruby 3.2.2, whereas before this change it was at least 100x slower. Co-authored-by: Jean Boussier <[email protected]>
2024-07-15[Backport #20633] Fix the condition for `atomic_signal_fence` (#11166)Ivo Anjo
[Bug #20633] Fix the condition for `atomic_signal_fence` `AC_CHECK_DECLS` defines `HAVE_DECL_SYMBOL` to 1 if declared, 0 otherwise, not undefined. Co-authored-by: kimuraw (Wataru Kimura) <[email protected]>
2024-07-09retry on cancelling of `getaddrinfo` (#11131)Koichi Sasada
When the registerred unblock function is called, it should retry the cancelled blocking function if possible after checkints. For example, `SIGCHLD` can cancel this method, but it should not raise any exception if there is no trap handlers. The following is repro-code: ```ruby require 'socket' PN = 10_000 1000000.times{ p _1 PN.times{ fork{ sleep rand(0.3) } } i = 0 while i<PN cpid = Process.wait -1, Process::WNOHANG if cpid # p [i, cpid] i += 1 end begin TCPServer.new(nil, 0).close rescue p $! exit! end end } ```
2024-07-08v3.3.4v3_3_4Takashi Kokubun
2024-07-08merge revision(s) fc33559c: [Backport #20570]Takashi Kokubun
clear `kw_flag` if given hash is nil https://bugs.ruby-lang.org/issues/20570 is caused I missed to clear the `kw_flag` even if `keyword_hash` is nil.
2024-07-08merge revision(s) 75aaeb35b82da26359b9418d2963384d0c55839c: [Backport #20239]Takashi Kokubun
[Bug #20239] Fix overflow at down-casting
2024-07-08merge revision(s) fba8aff7, d8c6e91748871ab2287d7703347847fe18a292d2: ↵Takashi Kokubun
[Backport #20592] [Bug #20592] Fix segfault when sending NULL to freeaddrinfo On alpine freeaddrinfo does not accept NULL pointer Fix dangling `else`
2024-07-08merge revision(s) 2dd46bb82ffc4dff01d7ea70922f0e407acafb4e: [Backport #20468]Takashi Kokubun
[Bug #20468] Fix safe navigation in `for` variable
2024-07-08merge revision(s) 01b13886: [Backport #20562]Takashi Kokubun
[Bug #20562] Categorize `RUBY_FREE_AT_EXIT` warning as experimental
2024-07-08Fix malformed JSON in macOS CITakashi Kokubun
2024-07-05Refine macOS CI (#11107)Hiroshi SHIBATA
Update macos runners with latest environments. * Use macos-14 instead of macos-arm-oss * Removed macos-11 and added macos-13
2024-07-03[Backport #11036] Add explicit compiler fence when pushing frames to ensure ↵Ivo Anjo
safe profiling (#11090) **What does this PR do?** This PR tweaks the `vm_push_frame` function to add an explicit compiler fence (`atomic_signal_fence`) to ensure profilers that use signals to interrupt applications (stackprof, vernier, pf2, Datadog profiler) can safely sample from the signal handler. This is a backport of #11036 to Ruby 3.3 . **Motivation:** The `vm_push_frame` was specifically tweaked in https://github.com/ruby/ruby/pull/3296 to initialize the a frame before updating the `cfp` pointer. But since there's nothing stopping the compiler from reordering the initialization of a frame (`*cfp =`) with the update of the cfp pointer (`ec->cfp = cfp`) we've been hesitant to rely on this on the Datadog profiler. In practice, after some experimentation + talking to folks, this reordering does not seem to happen. But since modern compilers have a way for us to exactly tell them not to do the reordering (`atomic_signal_fence`), this seems even better. I've actually extracted `vm_push_frame` into the "Compiler Explorer" website, which you can use to see the assembly output of this function across many compilers and architectures: https://godbolt.org/z/3oxd1446K On that link you can observe two things across many compilers: 1. The compilers are not reordering the writes 2. The barrier does not change the generated assembly output (== has no cost in practice) **Additional Notes:** The checks added in `configure.ac` define two new macros: * `HAVE_STDATOMIC_H` * `HAVE_DECL_ATOMIC_SIGNAL_FENCE` Since Ruby generates an arch-specific `config.h` header with these macros upon installation, this can be used by profilers and other libraries to test if Ruby was compiled with the fence enabled. **How to test the change?** As I mentioned above, you can check https://godbolt.org/z/3oxd1446K to confirm the compiled output of `vm_push_frame` does not change in most compilers (at least all that I've checked on that site).
2024-06-28[Bug #20598] Fix corruption of internal encoding string (#11069)Peter Zhu
Fix corruption of internal encoding string [Bug #20598] Just like [Bug #20595], Encoding#name_list and Encoding#aliases can have their strings corrupted when Encoding.default_internal is set to nil. Co-authored-by: Matthew Valentine-House <[email protected]>
2024-06-27[Bug #20595] Fix corruption of encoding name string (#11063)Peter Zhu
Fix corruption of encoding name string [Bug #20595] enc_set_default_encoding will free the C string if the encoding is nil, but the C string can be used by the encoding name string. This will cause the encoding name string to be corrupted. Consider the following code: Encoding.default_internal = Encoding::ASCII_8BIT names = Encoding.default_internal.names p names Encoding.default_internal = nil p names It outputs: ["ASCII-8BIT", "BINARY", "internal"] ["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"] Co-authored-by: Matthew Valentine-House <[email protected]>
2024-06-20[Bug #20581][3.3] Fix unintentional truncation for dependencies of bundled ↵Hiroshi SHIBATA
gems (#11006) * Try to load original gemspec from `.bundle/gems/foo-x.y.z/foo.gemspec`. `.bundle/specification/foo-x.y.z.gemspec` may be changed our toolchain * Try to find gemspec from `.bundle/specifications * Adjust indent
2024-06-20Add k0kubun to ruby_3_3 CODEOWNERSTakashi Kokubun
2024-06-20String.new(capacity:) don't substract termlen (#11027)Jean byroot Boussier
[Bug #20585] This was changed in 36a06efdd9f0604093dccbaf96d4e2cb17874dc8 because `String.new(1024)` would end up allocating `1025` bytes, but the problem with this change is that the caller may be trying to right size a String. So instead, we should just better document the behavior of `capacity:`. Co-authored-by: Jean Boussier <[email protected]>
2024-06-13Don't call `Warning.warn` unless the category is enabled (#10981)Aaron Patterson
Don't call `Warning.warn` unless the category is enabled The warning category should be enabled if we want to call `Warning.warn`. This commit speeds up the following benchmark: ```ruby eval "def test; " + 1000.times.map { "' '.chomp!" }.join(";") + "; end" def run_benchmark count i = 0 while i < count start = Process.clock_gettime(Process::CLOCK_MONOTONIC) yield ms = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start puts "itr ##{i}: #{(ms * 1000).to_i}ms" i += 1 end end run_benchmark(25) do 250.times do test end end ``` On `master` this runs at about 92ms per iteration. With this patch, it is 7ms per iteration. [Bug #20573]
2024-06-13Bump shlex from 1.1.0 to 1.3.0 in /yjit/bindgen (#10985)Alan Wu
`yjit-bindgen` isn't run to build Ruby releases at all, but people might be running security scanners on the source tarball. Upgrade this dependency to calm the scanners. Backport of <https://github.com/ruby/ruby/pull/9652>. See: <https://github.com/ruby/ruby/pull/10980>
2024-06-11v3.3.3v3_3_3Takashi Kokubun
2024-06-11[Bug #20270] Fix --parser=prism (#10970)Peter Zhu
Co-authored-by: Takashi Kokubun <[email protected]>
2024-06-11redmine-backporter.rb: Prepend commit: to shorter revsTakashi Kokubun
Some of the places in Redmine (e.g. Associated revisions) print revisions using only 8 characters. Even when I copied a revision from there, I want to prepend commit: in the message.
2024-06-11merge revision(s) 27321290: [Backport #20521]Takashi Kokubun
[Bug #20521] ripper: Clean up strterm
2024-06-11merge revision(s) 1e08a9f0e9058186db18f29efc6458c00f10a856: [Backport #20499]Takashi Kokubun
[Bug #20499] Use Xcode owned tools for Xcode clang Xcode has its own version tools that may be incompatible with genuine LLVM tools, use the tools in the same directory.
2024-06-11[3.3 backport] compile.c: use putspecialobject for RubyVM::FrozenCore (#10962)Jean byroot Boussier
compile.c: use putspecialobject for RubyVM::FrozenCore [Bug #20569] `putobject RubyVM::FrozenCore`, is not serializable, we have to use `putspecialobject VM_SPECIAL_OBJECT_VMCORE`. Co-authored-by: Jean Boussier <[email protected]>
2024-06-11Raise SyntaxError on invalid encoding symbol (#10967)Peter Zhu
[Bug #20280] Backport of #10014.
2024-06-10Fix inconsistent evaluation of keyword splat (#10959)Peter Zhu
[Bug #20180] Backports #9624.
2024-06-05Don't add `+YJIT` to `RUBY_DESCRIPTION` until it's actually enabled (#10920)Jean byroot Boussier
If you start Ruby with `--yjit-disable`, the `+YJIT` shouldn't be added until `RubyVM::YJIT.enable` is actually called. Otherwise it's confusing in crash reports etc. Co-authored-by: Jean Boussier <[email protected]>
2024-06-05merge revision(s) f8abd24b1f28998157da1230b231419ef7b81722: [Backport #20522]Takashi Kokubun
Improve YJIT performance warning regression test [Bug #20522]
2024-06-04Merge RubyGems 3.5.11 and Bundler 2.5.11 for Ruby 3.3 (#10870)Hiroshi SHIBATA
Co-authored-by: Nobuyoshi Nakada <[email protected]>
2024-06-04merge revision(s) 05553cf22d43dd78b8f30cc4591230b5c000c538: [Backport #20517]Takashi Kokubun
[Bug #20517] Make a multibyte character one token at meta escape
2024-06-04[3.3 backport] Do not emit shape transition warnings when YJIT is compiling ↵Jean byroot Boussier
(#10911) Do not emit shape transition warnings when YJIT is compiling [Bug #20522] If `Warning.warn` is redefined in Ruby, emitting a warning would invoke Ruby code, which can't safely be done when YJIT is compiling. Co-authored-by: Jean Boussier <[email protected]> Co-authored-by: Takashi Kokubun <[email protected]>
2024-06-04YJIT: Fix out of bounds access when splatting empty array (#10905)Alan Wu
This is a backport of 6c8ae44a388e5c03b7db90376af3652007b574e8 with a test tailored to crash the 3.3.x branch (from GH-10904). Previously, we read the last element array even when the array was empty, doing an out-of-bounds access. This sometimes caused a SEGV. [Bug #20496]
2024-06-04merge revision(s) f54369830f83a65fb54916d762883fbe6eeb7d0b, ↵Takashi Kokubun
338eb0065bd81ba8ae8b9402abc94804a24594cc, ac636f5709feb1d9d7a0c46a86be153be765cf21: [Backport #20516] Revert "Rollback to released version numbers of stringio and strscan" This reverts commit 6a79e53823e328281b9e9eee53cd141af28f8548. [ruby/strscan] StringScanner#captures: Return nil not "" for unmached capture (https://github.com/ruby/strscan/pull/72) fix https://github.com/ruby/strscan/issues/70 If there is no substring matching the group (s[3]), the behavior is different. If there is no substring matching the group, the corresponding element (s[3]) should be nil. ``` s = StringScanner.new('foobarbaz') #=> #<StringScanner 0/9 @ "fooba..."> s.scan /(foo)(bar)(BAZ)?/ #=> "foobar" s[0] #=> "foobar" s[1] #=> "foo" s[2] #=> "bar" s[3] #=> nil s.captures #=> ["foo", "bar", ""] s.captures.compact #=> ["foo", "bar", ""] ``` ``` s = StringScanner.new('foobarbaz') #=> #<StringScanner 0/9 @ "fooba..."> s.scan /(foo)(bar)(BAZ)?/ #=> "foobar" s[0] #=> "foobar" s[1] #=> "foo" s[2] #=> "bar" s[3] #=> nil s.captures #=> ["foo", "bar", nil] s.captures.compact #=> ["foo", "bar"] ``` https://docs.ruby-lang.org/ja/latest/method/MatchData/i/captures.html ``` /(foo)(bar)(BAZ)?/ =~ "foobarbaz" #=> 0 $~.to_a #=> ["foobar", "foo", "bar", nil] $~.captures #=> ["foo", "bar", nil] $~.captures.compact #=> ["foo", "bar"] ``` * StringScanner#captures is not yet documented. https://docs.ruby-lang.org/ja/latest/class/StringScanner.html https://github.com/ruby/strscan/commit/1fbfdd3c6f [ruby/strscan] Bump version https://github.com/ruby/strscan/commit/d6f97ec102
2024-06-04merge revision(s) 9f708d48f6df37ee9600db9d51b57a156609a13b, ↵Takashi Kokubun
0301473fb523c71d8cdc4966971f31f502001185, 874e9fc34d728f8e2444d15aa6759befd217c464, 7f0e26b7f99bf76408569892ce20318501f74729: [Backport #20516] Clear runtime dependencies if default gems is specified. The current build system uses runtime dependencies from only `.bundle` directory. We shouldn't install runtime dependencies from rubygems.org when `make test-bundled-gems` is invoked. Fixed dependencies list format Don't need to remove ruby2_keywords dependency from drb Re-use strscan with ruby repo
2024-06-04merger.rb: Put spaces in between revisionsTakashi Kokubun
so that they are linked correctly on GitHub
2024-05-31merge revision(s) 70ad58cb62b195ba86a5ef07a565b22b02a040ea: [Backport #20516]Takashi Kokubun
Update bundled_gems
2024-05-30merge revision(s) be7c91db44d6b8dba8fa11ff782965b4bfa0b3c8: [Backport #20515]Takashi Kokubun
Do not pollute toplevel namespace
2024-05-30merge revision(s) fd549b229b0822198ddc847703194263a2186ed1: [Backport #20515]Takashi Kokubun
test_bignum: defined? returns String (#10880) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit didn't verify the test is working properly due to mistaken auto-merge… [Bug #20515] bug: https://bugs.ruby-lang.org/issues/20515 follow-up: 22e4eeda6561693367fc7a00b92b90f46b09cabd follow-up: https://github.com/ruby/ruby/pull/10875
2024-05-30merge revision(s) ↵Takashi Kokubun
22e4eeda6561693367fc7a00b92b90f46b09cabd,1ab7c412d2e3880a7ad233c32e93961888f8145c: [Backport #20515] ci: Test whether GMP is working in compilers.yml (#10875) Avoid reoccurence of [Bug #20515] Requires https://github.com/ruby/ruby/pull/10876 since 18eaf0be905e3e251423b42d6f4e56b7cae1bc3b bug: https://bugs.ruby-lang.org/issues/20515 RUBY_CHECK_HEADER didn't define HAVE_{header-file} (#10876) --with-gmp is not working at all because HAVE_GMP_H was missing since 18eaf0be90. [Bug #20515] bug: https://bugs.ruby-lang.org/issues/20515 follow-up: https://bugs.ruby-lang.org/issues/20494 follow-up: 18eaf0be905e3e251423b42d6f4e56b7cae1bc3b follow-up: https://github.com/ruby/ruby/pull/10805
2024-05-30merge revision(s) ↵Takashi Kokubun
055613fd868a8c94e43893f8c58a00cdd2a81f6d,127d7a35df10ee2bc99f44b888972b2c5361d84f,e2a9b87126d59e4766479a7aa12cf7a648f46506: [Backport #20447] Fix pointer incompatiblity Since the subsecond part is discarded, WIDEVAL to VALUE conversion is needed. Some functions are not used when `THREAD_MODEL=none` `rb_thread_sched_destroy` is not used now at all