| Kent Tamura | 59ffb02 | 2018-11-27 05:30:56 | [diff] [blame] | 1 | # Web Test Expectations and Baselines |
| pwnall | d8a25072 | 2016-11-09 18:24:03 | [diff] [blame] | 2 | |
| Kent Tamura | 59ffb02 | 2018-11-27 05:30:56 | [diff] [blame] | 3 | The primary function of the web tests is as a regression test suite; this |
| pwnall | d8a25072 | 2016-11-09 18:24:03 | [diff] [blame] | 4 | means that, while we care about whether a page is being rendered correctly, we |
| 5 | care more about whether the page is being rendered the way we expect it to. In |
| 6 | other words, we look more for changes in behavior than we do for correctness. |
| 7 | |
| 8 | [TOC] |
| 9 | |
| Kent Tamura | 59ffb02 | 2018-11-27 05:30:56 | [diff] [blame] | 10 | All web tests have "expected results", or "baselines", which may be one of |
| pwnall | d8a25072 | 2016-11-09 18:24:03 | [diff] [blame] | 11 | several forms. The test may produce one or more of: |
| 12 | |
| 13 | * A text file containing JavaScript log messages. |
| 14 | * A text rendering of the Render Tree. |
| 15 | * A screen capture of the rendered page as a PNG file. |
| 16 | * WAV files of the audio output, for WebAudio tests. |
| 17 | |
| Kent Tamura | 59ffb02 | 2018-11-27 05:30:56 | [diff] [blame] | 18 | For any of these types of tests, baselines are checked into the web_tests |
| Robert Ma | 06f7acc | 2017-11-14 17:55:47 | [diff] [blame] | 19 | directory. The filename of a baseline is the same as that of the corresponding |
| 20 | test, but the extension is replaced with `-expected.{txt,png,wav}` (depending on |
| 21 | the type of test output). Baselines usually live alongside tests, with the |
| 22 | exception when baselines vary by platforms; read |
| Kent Tamura | 59ffb02 | 2018-11-27 05:30:56 | [diff] [blame] | 23 | [Web Test Baseline Fallback](web_test_baseline_fallback.md) for more |
| Robert Ma | 06f7acc | 2017-11-14 17:55:47 | [diff] [blame] | 24 | details. |
| 25 | |
| 26 | Lastly, we also support the concept of "reference tests", which check that two |
|
|
