summaryrefslogtreecommitdiff
path: root/doc/contributing/testing_ruby.md
diff options
context:
space:
mode:
Diffstat (limited to 'doc/contributing/testing_ruby.md')
-rw-r--r--doc/contributing/testing_ruby.md71
1 files changed, 35 insertions, 36 deletions
diff --git a/doc/contributing/testing_ruby.md b/doc/contributing/testing_ruby.md
index ca6a991917..4c7ce7f6a8 100644
--- a/doc/contributing/testing_ruby.md
+++ b/doc/contributing/testing_ruby.md
@@ -1,5 +1,9 @@
# Testing Ruby
+All the commands below assume that you're running them from the `build/` directory made during [Building Ruby](building_ruby.md).
+
+Most commands below should work with [GNU make](https://www.gnu.org/software/make/) (the default on Linux and macOS), [BSD make](https://man.freebsd.org/cgi/man.cgi?make(1)) and [NMAKE](https://learn.microsoft.com/en-us/cpp/build/reference/nmake-reference), except where indicated otherwise.
+
## Test suites
There are several test suites in the Ruby codebase:
@@ -8,23 +12,23 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+
1. [bootstraptest/](https://github.com/ruby/ruby/tree/master/bootstraptest)
- This is a small test suite that runs on Miniruby (see [building Ruby](building_ruby.md#label-Miniruby+vs+Ruby)). We can run it with:
+ This is a small test suite that runs on [Miniruby](building_ruby.md#label-Miniruby+vs+Ruby). We can run it with:
```sh
make btest
```
- To run it with logs, we can use:
+ To run individual bootstrap tests, we can either specify a list of filenames or use the `--sets` flag in the variable `BTESTS`:
```sh
- make btest OPTS=-v
+ make btest BTESTS="../bootstraptest/test_string.rb ../bootstraptest/test_class.rb"
+ make btest BTESTS="--sets=string,class"
```
- To run individual bootstrap tests, we can either specify a list of filenames or use the `--sets` flag in the variable `BTESTS`:
+ To run these tests with verbose logging, we can add `-v` to the `OPTS`:
```sh
- make btest BTESTS="../bootstraptest/test_fork.rb ../bootstraptest/test_gc.rb"
- make btest BTESTS="--sets=fork,gc"
+ make btest OPTS="--sets=string,class -v"
```
If we want to run the bootstrap test suite on Ruby (not Miniruby), we can use:
@@ -33,47 +37,47 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+
make test
```
- To run it with logs, we can use:
+ To run these tests with verbose logging, we can add `-v` to the `OPTS`:
```sh
make test OPTS=-v
```
- To run a file or directory with GNU make, we can use:
+ (GNU make only) To run a specific file, we can use:
```sh
- make test/ruby/test_foo.rb
- make test/ruby/test_foo.rb TESTOPTS="-n /test_bar/"
+ make ../test/ruby/test_string.rb
```
-2. [test/](https://github.com/ruby/ruby/tree/master/test)
-
- This is a more comprehensive test suite that runs on Ruby. We can run it with:
+ You can use the `-n` test option to run a specific test with a regex:
```sh
- make test-all
+ make ../test/ruby/test_string.rb TESTOPTS="-n /test_.*_to_s/"
```
- We can run a specific test directory in this suite using the `TESTS` option, for example:
+2. [test/](https://github.com/ruby/ruby/tree/master/test)
+
+ This is a more comprehensive test suite that runs on Ruby. We can run it with:
```sh
- make test-all TESTS=test/rubygems
+ make test-all
```
- We can run a specific test file in this suite by also using the `TESTS` option, for example:
+ We can run a specific test file or directory in this suite using the `TESTS` option, for example:
```sh
- make test-all TESTS=test/ruby/test_array.rb
+ make test-all TESTS="../test/ruby/"
+ make test-all TESTS="../test/ruby/test_string.rb"
```
We can run a specific test in this suite using the `TESTS` option, specifying
first the file name, and then the test name, prefixed with `--name`. For example:
```sh
- make test-all TESTS="../test/ruby/test_alias.rb --name=TestAlias#test_alias_with_zsuper_method"
+ make test-all TESTS="../test/ruby/test_string.rb --name=TestString#test_to_s"
```
- To run these specs with logs, we can use:
+ To run these tests with verbose logging, we can add `-v` to `TESTS`:
```sh
make test-all TESTS=-v
@@ -85,7 +89,7 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+
make test-all TESTS=--help
```
- If we would like to run the `test/`, `bootstraptest/` and `spec/` test suites (the `spec/` is explained in a later section), we can run
+ We can run all the tests in `test/`, `bootstraptest/` and `spec/` (the `spec/` is explained in a later section) all together with:
```sh
make check
@@ -93,45 +97,40 @@ We can run any of the make scripts [in parallel](building_ruby.md#label-Running+
3. [spec/ruby](https://github.com/ruby/ruby/tree/master/spec/ruby)
- This is a test suite that exists in [the Ruby spec repository](https://github.com/ruby/spec) and is mirrored into the `spec/ruby` directory in the Ruby repository. It tests the behavior of the Ruby programming language. We can run this using:
+ This is a test suite defined in [the Ruby spec repository](https://github.com/ruby/spec), and is periodically mirrored into the `spec/ruby` directory of this repository. It tests the behavior of the Ruby programming language. We can run this using:
```sh
make test-spec
```
- To run a specific directory, we can use `SPECOPTS` to specify the directory:
-
- ```sh
- make test-spec SPECOPTS=spec/ruby/core/array
- ```
-
- To run a specific file, we can also use `SPECOPTS` to specify the file:
+ We can run a specific test file or directory in this suite using the `SPECOPTS` option, for example:
```sh
- make test-spec SPECOPTS=spec/ruby/core/array/any_spec.rb
+ make test-spec SPECOPTS="../spec/ruby/core/string/"
+ make test-spec SPECOPTS="../spec/ruby/core/string/to_s_spec.rb"
```
To run a specific test, we can use the `--example` flag to match against the test name:
```sh
- make test-spec SPECOPTS="../spec/ruby/core/array/any_spec.rb --example='is false if the array is empty'"
+ make test-spec SPECOPTS="../spec/ruby/core/string/to_s_spec.rb --example='returns self when self.class == String'"
```
- To run these specs with logs, we can use:
+ To run these specs with verbose logging, we can add `-v` to the `SPECOPTS`:
```sh
- make test-spec SPECOPTS=-Vfs
+ make test-spec SPECOPTS="../spec/ruby/core/string/to_s_spec.rb -Vfs"
```
- To run a ruby-spec file or directory with GNU make, we can use
+ (GNU make only) To run a ruby-spec file or directory, we can use
```sh
- make spec/ruby/core/foo/bar_spec.rb
+ make ../spec/ruby/core/string/to_s_spec.rb
```
4. [spec/bundler](https://github.com/ruby/ruby/tree/master/spec/bundler)
- The bundler test suite exists in [the RubyGems repository](https://github.com/rubygems/rubygems/tree/master/bundler/spec) and is mirrored into the `spec/bundler` directory in the Ruby repository. We can run this using:
+ The bundler test suite is defined in [the RubyGems repository](https://github.com/rubygems/rubygems/tree/master/bundler/spec), and is periodically mirrored into the `spec/ruby` directory of this repository. We can run this using:
```sh
make test-bundler