source: trunk/essentials/dev-lang/python/Lib/test/regrtest.py@ 3951

Last change on this file since 3951 was 3402, checked in by bird, 19 years ago

os2emx -> os2knix.
File size: 41.0 KB
Line 
1#! /usr/bin/env python
2
3"""Regression test.
4
5This will find all modules whose name is "test_*" in the test
6directory, and run them. Various command line options provide
7additional facilities.
8
9Command line options:
10
11-v: verbose -- run tests in verbose mode with output to stdout
12-w: verbose2 -- re-run failed tests in verbose mode
13-q: quiet -- don't print anything except if a test fails
14-g: generate -- write the output file for a test instead of comparing it
15-x: exclude -- arguments are tests to *exclude*
16-s: single -- run only a single test (see below)
17-r: random -- randomize test execution order
18-f: fromfile -- read names of tests to run from a file (see below)
19-l: findleaks -- if GC is available detect tests that leak memory
20-u: use -- specify which special resource intensive tests to run
21-h: help -- print this text and exit
22-t: threshold -- call gc.set_threshold(N)
23-T: coverage -- turn on code coverage using the trace module
24-D: coverdir -- Directory where coverage files are put
25-N: nocoverdir -- Put coverage files alongside modules
26-L: runleaks -- run the leaks(1) command just before exit
27-R: huntrleaks -- search for reference leaks (needs debug build, v. slow)
28-M: memlimit -- run very large memory-consuming tests
29
30If non-option arguments are present, they are names for tests to run,
31unless -x is given, in which case they are names for tests not to run.
32If no test names are given, all tests are run.
33
34-v is incompatible with -g and does not compare test output files.
35
36-T turns on code coverage tracing with the trace module.
37
38-D specifies the directory where coverage files are put.
39
40-N Put coverage files alongside modules.
41
42-s means to run only a single test and exit. This is useful when
43doing memory analysis on the Python interpreter (which tend to consume
44too many resources to run the full regression test non-stop). The
45file /tmp/pynexttest is read to find the next test to run. If this
46file is missing, the first test_*.py file in testdir or on the command
47line is used. (actually tempfile.gettempdir() is used instead of
48/tmp).
49
50-f reads the names of tests from the file given as f's argument, one
51or more test names per line. Whitespace is ignored. Blank lines and
52lines beginning with '#' are ignored. This is especially useful for
53whittling down failures involving interactions among tests.
54
55-L causes the leaks(1) command to be run just before exit if it exists.
56leaks(1) is available on Mac OS X and presumably on some other
57FreeBSD-derived systems.
58
59-R runs each test several times and examines sys.gettotalrefcount() to
60see if the test appears to be leaking references. The argument should
61be of the form stab:run:fname where 'stab' is the number of times the
62test is run to let gettotalrefcount settle down, 'run' is the number
63of times further it is run and 'fname' is the name of the file the
64reports are written to. These parameters all have defaults (5, 4 and
65"reflog.txt" respectively), so the minimal invocation is '-R ::'.
66
67-M runs tests that require an exorbitant amount of memory. These tests
68typically try to ascertain containers keep working when containing more than
692 billion objects, which only works on 64-bit systems. There are also some
70tests that try to exhaust the address space of the process, which only makes
71sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
72which is a string in the form of '2.5Gb', determines howmuch memory the
73tests will limit themselves to (but they may go slightly over.) The number
74shouldn't be more memory than the machine has (including swap memory). You
75should also keep in mind that swap memory is generally much, much slower
76than RAM, and setting memlimit to all available RAM or higher will heavily
77tax the machine. On the other hand, it is no use running these tests with a
78limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
79to use more than memlimit memory will be skipped. The big-memory tests
80generally run very, very long.
81
82-u is used to specify which special resource intensive tests to run,
83such as those requiring large file support or network connectivity.
84The argument is a comma-separated list of words indicating the
85resources to test. Currently only the following are defined:
86
87 all - Enable all special resources.
88
89 audio - Tests that use the audio device. (There are known
90 cases of broken audio drivers that can crash Python or
91 even the Linux kernel.)
92
93 curses - Tests that use curses and will modify the terminal's
94 state and output modes.
95
96 largefile - It is okay to run some test that may create huge
97 files. These tests can take a long time and may
98 consume >2GB of disk space temporarily.
99
100 network - It is okay to run tests that use external network
101 resource, e.g. testing SSL support for sockets.
102
103 bsddb - It is okay to run the bsddb testsuite, which takes
104 a long time to complete.
105
106 decimal - Test the decimal module against a large suite that
107 verifies compliance with standards.
108
109 compiler - Test the compiler package by compiling all the source
110 in the standard library and test suite. This takes
111 a long time. Enabling this resource also allows
112 test_tokenize to verify round-trip lexing on every
113 file in the test library.
114
115 subprocess Run all tests for the subprocess module.
116
117 urlfetch - It is okay to download files required on testing.
118
119To enable all resources except one, use '-uall,-<resource>'. For
120example, to run all the tests except for the bsddb tests, give the
121option '-uall,-bsddb'.
122"""
123
124import os
125import sys
126import getopt
127import random
128import warnings
129import re
130import cStringIO
131import traceback
132
133# I see no other way to suppress these warnings;
134# putting them in test_grammar.py has no effect:
135warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
136 ".*test.test_grammar$")
137if sys.maxint > 0x7fffffff:
138 # Also suppress them in <string>, because for 64-bit platforms,
139 # that's where test_grammar.py hides them.
140 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
141 "<string>")
142
143# Ignore ImportWarnings that only occur in the source tree,
144# (because of modules with the same name as source-directories in Modules/)
145for mod in ("ctypes", "gzip", "zipfile", "tarfile", "encodings.zlib_codec",
146 "test.test_zipimport", "test.test_zlib", "test.test_zipfile",
147 "test.test_codecs", "test.string_tests"):
148 warnings.filterwarnings(module=".*%s$" % (mod,),
149 action="ignore", category=ImportWarning)
150
151# MacOSX (a.k.a. Darwin) has a default stack size that is too small
152# for deeply recursive regular expressions. We see this as crashes in
153# the Python test suite when running test_re.py and test_sre.py. The
154# fix is to set the stack limit to 2048.
155# This approach may also be useful for other Unixy platforms that
156# suffer from small default stack limits.
157if sys.platform == 'darwin':
158 try:
159 import resource
160 except ImportError:
161 pass
162 else:
163 soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
164 newsoft = min(hard, max(soft, 1024*2048))
165 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
166
167from test import test_support
168
169RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
170 'decimal', 'compiler', 'subprocess', 'urlfetch')
171
172
173def usage(code, msg=''):
174 print __doc__
175 if msg: print msg
176 sys.exit(code)
177
178
179def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,
180 exclude=False, single=False, randomize=False, fromfile=None,
181 findleaks=False, use_resources=None, trace=False, coverdir='coverage',
182 runleaks=False, huntrleaks=False, verbose2=False):
183 """Execute a test suite.
184
185 This also parses command-line options and modifies its behavior
186 accordingly.
187
188 tests -- a list of strings containing test names (optional)
189 testdir -- the directory in which to look for tests (optional)
190
191 Users other than the Python test suite will certainly want to
192 specify testdir; if it's omitted, the directory containing the
193 Python test suite is searched for.
194
195 If the tests argument is omitted, the tests listed on the
196 command-line will be used. If that's empty, too, then all *.py
197 files beginning with test_ will be used.
198
199 The other default arguments (verbose, quiet, generate, exclude, single,
200 randomize, findleaks, use_resources, trace and coverdir) allow programmers
201 calling main() directly to set the values that would normally be set by
202 flags on the command line.
203 """
204
205 test_support.record_original_stdout(sys.stdout)
206 try:
207 opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TD:NLR:wM:',
208 ['help', 'verbose', 'quiet', 'generate',
209 'exclude', 'single', 'random', 'fromfile',
210 'findleaks', 'use=', 'threshold=', 'trace',
211 'coverdir=', 'nocoverdir', 'runleaks',
212 'huntrleaks=', 'verbose2', 'memlimit=',
213 ])
214 except getopt.error, msg:
215 usage(2, msg)
216
217 # Defaults
218 if use_resources is None:
219 use_resources = []
220 for o, a in opts:
221 if o in ('-h', '--help'):
222 usage(0)
223 elif o in ('-v', '--verbose'):
224 verbose += 1
225 elif o in ('-w', '--verbose2'):
226 verbose2 = True
227 elif o in ('-q', '--quiet'):
228 quiet = True;
229 verbose = 0
230 elif o in ('-g', '--generate'):
231 generate = True
232 elif o in ('-x', '--exclude'):
233 exclude = True
234 elif o in ('-s', '--single'):
235 single = True
236 elif o in ('-r', '--randomize'):
237 randomize = True
238 elif o in ('-f', '--fromfile'):
239 fromfile = a
240 elif o in ('-l', '--findleaks'):
241 findleaks = True
242 elif o in ('-L', '--runleaks'):
243 runleaks = True
244 elif o in ('-t', '--threshold'):
245 import gc
246 gc.set_threshold(int(a))
247 elif o in ('-T', '--coverage'):
248 trace = True
249 elif o in ('-D', '--coverdir'):
250 coverdir = os.path.join(os.getcwd(), a)
251 elif o in ('-N', '--nocoverdir'):
252 coverdir = None
253 elif o in ('-R', '--huntrleaks'):
254 huntrleaks = a.split(':')
255 if len(huntrleaks) != 3:
256 print a, huntrleaks
257 usage(2, '-R takes three colon-separated arguments')
258 if len(huntrleaks[0]) == 0:
259 huntrleaks[0] = 5
260 else:
261 huntrleaks[0] = int(huntrleaks[0])
262 if len(huntrleaks[1]) == 0:
263 huntrleaks[1] = 4
264 else:
265 huntrleaks[1] = int(huntrleaks[1])
266 if len(huntrleaks[2]) == 0:
267 huntrleaks[2] = "reflog.txt"
268 elif o in ('-M', '--memlimit'):
269 test_support.set_memlimit(a)
270 elif o in ('-u', '--use'):
271 u = [x.lower() for x in a.split(',')]
272 for r in u:
273 if r == 'all':
274 use_resources[:] = RESOURCE_NAMES
275 continue
276 remove = False
277 if r[0] == '-':
278 remove = True
279 r = r[1:]
280 if r not in RESOURCE_NAMES:
281 usage(1, 'Invalid -u/--use option: ' + a)
282 if remove:
283 if r in use_resources:
284 use_resources.remove(r)
285 elif r not in use_resources:
286 use_resources.append(r)
287 if generate and verbose:
288 usage(2, "-g and -v don't go together!")
289 if single and fromfile:
290 usage(2, "-s and -f don't go together!")
291
292 good = []
293 bad = []
294 skipped = []
295 resource_denieds = []
296
297 if findleaks:
298 try:
299 import gc
300 except ImportError:
301 print 'No GC available, disabling findleaks.'
302 findleaks = False
303 else:
304 # Uncomment the line below to report garbage that is not
305 # freeable by reference counting alone. By default only
306 # garbage that is not collectable by the GC is reported.
307 #gc.set_debug(gc.DEBUG_SAVEALL)
308 found_garbage = []
309
310 if single:
311 from tempfile import gettempdir
312 filename = os.path.join(gettempdir(), 'pynexttest')
313 try:
314 fp = open(filename, 'r')
315 next = fp.read().strip()
316 tests = [next]
317 fp.close()
318 except IOError:
319 pass
320
321 if fromfile:
322 tests = []
323 fp = open(fromfile)
324 for line in fp:
325 guts = line.split() # assuming no test has whitespace in its name
326 if guts and not guts[0].startswith('#'):
327 tests.extend(guts)
328 fp.close()
329
330 # Strip .py extensions.
331 if args:
332 args = map(removepy, args)
333 if tests:
334 tests = map(removepy, tests)
335
336 stdtests = STDTESTS[:]
337 nottests = NOTTESTS[:]
338 if exclude:
339 for arg in args:
340 if arg in stdtests:
341 stdtests.remove(arg)
342 nottests[:0] = args
343 args = []
344 tests = tests or args or findtests(testdir, stdtests, nottests)
345 if single:
346 tests = tests[:1]
347 if randomize:
348 random.shuffle(tests)
349 if trace:
350 import trace
351 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
352 trace=False, count=True)
353 test_support.verbose = verbose # Tell tests to be moderately quiet
354 test_support.use_resources = use_resources
355 save_modules = sys.modules.keys()
356 for test in tests:
357 if not quiet:
358 print test
359 sys.stdout.flush()
360 if trace:
361 # If we're tracing code coverage, then we don't exit with status
362 # if on a false return value from main.
363 tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
364 globals=globals(), locals=vars())
365 else:
366 try:
367 ok = runtest(test, generate, verbose, quiet, testdir,
368 huntrleaks)
369 except KeyboardInterrupt:
370 # print a newline separate from the ^C
371 print
372 break
373 except:
374 raise
375 if ok > 0:
376 good.append(test)
377 elif ok == 0:
378 bad.append(test)
379 else:
380 skipped.append(test)
381 if ok == -2:
382 resource_denieds.append(test)
383 if findleaks:
384 gc.collect()
385 if gc.garbage:
386 print "Warning: test created", len(gc.garbage),
387 print "uncollectable object(s)."
388 # move the uncollectable objects somewhere so we don't see
389 # them again
390 found_garbage.extend(gc.garbage)
391 del gc.garbage[:]
392 # Unload the newly imported modules (best effort finalization)
393 for module in sys.modules.keys():
394 if module not in save_modules and module.startswith("test."):
395 test_support.unload(module)
396
397 # The lists won't be sorted if running with -r
398 good.sort()
399 bad.sort()
400 skipped.sort()
401
402 if good and not quiet:
403 if not bad and not skipped and len(good) > 1:
404 print "All",
405 print count(len(good), "test"), "OK."
406 if verbose:
407 print "CAUTION: stdout isn't compared in verbose mode:"
408 print "a test that passes in verbose mode may fail without it."
409 if bad:
410 print count(len(bad), "test"), "failed:"
411 printlist(bad)
412 if skipped and not quiet:
413 print count(len(skipped), "test"), "skipped:"
414 printlist(skipped)
415
416 e = _ExpectedSkips()
417 plat = sys.platform
418 if e.isvalid():
419 surprise = set(skipped) - e.getexpected() - set(resource_denieds)
420 if surprise:
421 print count(len(surprise), "skip"), \
422 "unexpected on", plat + ":"
423 printlist(surprise)
424 else:
425 print "Those skips are all expected on", plat + "."
426 else:
427 print "Ask someone to teach regrtest.py about which tests are"
428 print "expected to get skipped on", plat + "."
429
430 if verbose2 and bad:
431 print "Re-running failed tests in verbose mode"
432 for test in bad:
433 print "Re-running test %r in verbose mode" % test
434 sys.stdout.flush()
435 try:
436 test_support.verbose = 1
437 ok = runtest(test, generate, 1, quiet, testdir,
438 huntrleaks)
439 except KeyboardInterrupt:
440 # print a newline separate from the ^C
441 print
442 break
443 except:
444 raise
445
446 if single:
447 alltests = findtests(testdir, stdtests, nottests)
448 for i in range(len(alltests)):
449 if tests[0] == alltests[i]:
450 if i == len(alltests) - 1:
451 os.unlink(filename)
452 else:
453 fp = open(filename, 'w')
454 fp.write(alltests[i+1] + '\n')
455 fp.close()
456 break
457 else:
458 os.unlink(filename)
459
460 if trace:
461 r = tracer.results()
462 r.write_results(show_missing=True, summary=True, coverdir=coverdir)
463
464 if runleaks:
465 os.system("leaks %d" % os.getpid())
466
467 sys.exit(len(bad) > 0)
468
469
470STDTESTS = [
471 'test_grammar',
472 'test_opcodes',
473 'test_operations',
474 'test_builtin',
475 'test_exceptions',
476 'test_types',
477 ]
478
479NOTTESTS = [
480 'test_support',
481 'test_future1',
482 'test_future2',
483 'test_future3',
484 ]
485
486def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
487 """Return a list of all applicable test modules."""
488 if not testdir: testdir = findtestdir()
489 names = os.listdir(testdir)
490 tests = []
491 for name in names:
492 if name[:5] == "test_" and name[-3:] == os.extsep+"py":
493 modname = name[:-3]
494 if modname not in stdtests and modname not in nottests:
495 tests.append(modname)
496 tests.sort()
497 return stdtests + tests
498
499def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False):
500 """Run a single test.
501
502 test -- the name of the test
503 generate -- if true, generate output, instead of running the test
504 and comparing it to a previously created output file
505 verbose -- if true, print more messages
506 quiet -- if true, don't print 'skipped' messages (probably redundant)
507 testdir -- test directory
508 huntrleaks -- run multiple times to test for leaks; requires a debug
509 build; a triple corresponding to -R's three arguments
510 Return:
511 -2 test skipped because resource denied
512 -1 test skipped for some other reason
513 0 test failed
514 1 test passed
515 """
516
517 try:
518 return runtest_inner(test, generate, verbose, quiet, testdir,
519 huntrleaks)
520 finally:
521 cleanup_test_droppings(test, verbose)
522
523def runtest_inner(test, generate, verbose, quiet,
524 testdir=None, huntrleaks=False):
525 test_support.unload(test)
526 if not testdir:
527 testdir = findtestdir()
528 outputdir = os.path.join(testdir, "output")
529 outputfile = os.path.join(outputdir, test)
530 if verbose:
531 cfp = None
532 else:
533 cfp = cStringIO.StringIO()
534
535 try:
536 save_stdout = sys.stdout
537 try:
538 if cfp:
539 sys.stdout = cfp
540 print test # Output file starts with test name
541 if test.startswith('test.'):
542 abstest = test
543 else:
544 # Always import it from the test package
545 abstest = 'test.' + test
546 the_package = __import__(abstest, globals(), locals(), [])
547 the_module = getattr(the_package, test)
548 # Most tests run to completion simply as a side-effect of
549 # being imported. For the benefit of tests that can't run
550 # that way (like test_threaded_import), explicitly invoke
551 # their test_main() function (if it exists).
552 indirect_test = getattr(the_module, "test_main", None)
553 if indirect_test is not None:
554 indirect_test()
555 if huntrleaks:
556 dash_R(the_module, test, indirect_test, huntrleaks)
557 finally:
558 sys.stdout = save_stdout
559 except test_support.ResourceDenied, msg:
560 if not quiet:
561 print test, "skipped --", msg
562 sys.stdout.flush()
563 return -2
564 except (ImportError, test_support.TestSkipped), msg:
565 if not quiet:
566 print test, "skipped --", msg
567 sys.stdout.flush()
568 return -1
569 except KeyboardInterrupt:
570 raise
571 except test_support.TestFailed, msg:
572 print "test", test, "failed --", msg
573 sys.stdout.flush()
574 return 0
575 except:
576 type, value = sys.exc_info()[:2]
577 print "test", test, "crashed --", str(type) + ":", value
578 sys.stdout.flush()
579 if verbose:
580 traceback.print_exc(file=sys.stdout)
581 sys.stdout.flush()
582 return 0
583 else:
584 if not cfp:
585 return 1
586 output = cfp.getvalue()
587 if generate:
588 if output == test + "\n":
589 if os.path.exists(outputfile):
590 # Write it since it already exists (and the contents
591 # may have changed), but let the user know it isn't
592 # needed:
593 print "output file", outputfile, \
594 "is no longer needed; consider removing it"
595 else:
596 # We don't need it, so don't create it.
597 return 1
598 fp = open(outputfile, "w")
599 fp.write(output)
600 fp.close()
601 return 1
602 if os.path.exists(outputfile):
603 fp = open(outputfile, "r")
604 expected = fp.read()
605 fp.close()
606 else:
607 expected = test + "\n"
608 if output == expected or huntrleaks:
609 return 1
610 print "test", test, "produced unexpected output:"
611 sys.stdout.flush()
612 reportdiff(expected, output)
613 sys.stdout.flush()
614 return 0
615
616def cleanup_test_droppings(testname, verbose):
617 import shutil
618
619 # Try to clean up junk commonly left behind. While tests shouldn't leave
620 # any files or directories behind, when a test fails that can be tedious
621 # for it to arrange. The consequences can be especially nasty on Windows,
622 # since if a test leaves a file open, it cannot be deleted by name (while
623 # there's nothing we can do about that here either, we can display the
624 # name of the offending test, which is a real help).
625 for name in (test_support.TESTFN,
626 "db_home",
627 ):
628 if not os.path.exists(name):
629 continue
630
631 if os.path.isdir(name):
632 kind, nuker = "directory", shutil.rmtree
633 elif os.path.isfile(name):
634 kind, nuker = "file", os.unlink
635 else:
636 raise SystemError("os.path says %r exists but is neither "
637 "directory nor file" % name)
638
639 if verbose:
640 print "%r left behind %s %r" % (testname, kind, name)
641 try:
642 nuker(name)
643 except Exception, msg:
644 print >> sys.stderr, ("%r left behind %s %r and it couldn't be "
645 "removed: %s" % (testname, kind, name, msg))
646
647def dash_R(the_module, test, indirect_test, huntrleaks):
648 # This code is hackish and inelegant, but it seems to do the job.
649 import copy_reg
650
651 if not hasattr(sys, 'gettotalrefcount'):
652 raise Exception("Tracking reference leaks requires a debug build "
653 "of Python")
654
655 # Save current values for dash_R_cleanup() to restore.
656 fs = warnings.filters[:]
657 ps = copy_reg.dispatch_table.copy()
658 pic = sys.path_importer_cache.copy()
659
660 if indirect_test:
661 def run_the_test():
662 indirect_test()
663 else:
664 def run_the_test():
665 reload(the_module)
666
667 deltas = []
668 nwarmup, ntracked, fname = huntrleaks
669 repcount = nwarmup + ntracked
670 print >> sys.stderr, "beginning", repcount, "repetitions"
671 print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]
672 dash_R_cleanup(fs, ps, pic)
673 for i in range(repcount):
674 rc = sys.gettotalrefcount()
675 run_the_test()
676 sys.stderr.write('.')
677 dash_R_cleanup(fs, ps, pic)
678 if i >= nwarmup:
679 deltas.append(sys.gettotalrefcount() - rc - 2)
680 print >> sys.stderr
681 if any(deltas):
682 print >> sys.stderr, test, 'leaked', deltas, 'references'
683 refrep = open(fname, "a")
684 print >> refrep, test, 'leaked', deltas, 'references'
685 refrep.close()
686
687def dash_R_cleanup(fs, ps, pic):
688 import gc, copy_reg
689 import _strptime, linecache, dircache
690 import urlparse, urllib, urllib2, mimetypes, doctest
691 import struct, filecmp
692 from distutils.dir_util import _path_created
693
694 # Restore some original values.
695 warnings.filters[:] = fs
696 copy_reg.dispatch_table.clear()
697 copy_reg.dispatch_table.update(ps)
698 sys.path_importer_cache.clear()
699 sys.path_importer_cache.update(pic)
700
701 # Clear assorted module caches.
702 _path_created.clear()
703 re.purge()
704 _strptime._regex_cache.clear()
705 urlparse.clear_cache()
706 urllib.urlcleanup()
707 urllib2.install_opener(None)
708 dircache.reset()
709 linecache.clearcache()
710 mimetypes._default_mime_types()
711 struct._cache.clear()
712 filecmp._cache.clear()
713 doctest.master = None
714
715 # Collect cyclic trash.
716 gc.collect()
717
718def reportdiff(expected, output):
719 import difflib
720 print "*" * 70
721 a = expected.splitlines(1)
722 b = output.splitlines(1)
723 sm = difflib.SequenceMatcher(a=a, b=b)
724 tuples = sm.get_opcodes()
725
726 def pair(x0, x1):
727 # x0:x1 are 0-based slice indices; convert to 1-based line indices.
728 x0 += 1
729 if x0 >= x1:
730 return "line " + str(x0)
731 else:
732 return "lines %d-%d" % (x0, x1)
733
734 for op, a0, a1, b0, b1 in tuples:
735 if op == 'equal':
736 pass
737
738 elif op == 'delete':
739 print "***", pair(a0, a1), "of expected output missing:"
740 for line in a[a0:a1]:
741 print "-", line,
742
743 elif op == 'replace':
744 print "*** mismatch between", pair(a0, a1), "of expected", \
745 "output and", pair(b0, b1), "of actual output:"
746 for line in difflib.ndiff(a[a0:a1], b[b0:b1]):
747 print line,
748
749 elif op == 'insert':
750 print "***", pair(b0, b1), "of actual output doesn't appear", \
751 "in expected output after line", str(a1)+":"
752 for line in b[b0:b1]:
753 print "+", line,
754
755 else:
756 print "get_opcodes() returned bad tuple?!?!", (op, a0, a1, b0, b1)
757
758 print "*" * 70
759
760def findtestdir():
761 if __name__ == '__main__':
762 file = sys.argv[0]
763 else:
764 file = __file__
765 testdir = os.path.dirname(file) or os.curdir
766 return testdir
767
768def removepy(name):
769 if name.endswith(os.extsep + "py"):
770 name = name[:-3]
771 return name
772
773def count(n, word):
774 if n == 1:
775 return "%d %s" % (n, word)
776 else:
777 return "%d %ss" % (n, word)
778
779def printlist(x, width=70, indent=4):
780 """Print the elements of iterable x to stdout.
781
782 Optional arg width (default 70) is the maximum line length.
783 Optional arg indent (default 4) is the number of blanks with which to
784 begin each line.
785 """
786
787 from textwrap import fill
788 blanks = ' ' * indent
789 print fill(' '.join(map(str, x)), width,
790 initial_indent=blanks, subsequent_indent=blanks)
791
792# Map sys.platform to a string containing the basenames of tests
793# expected to be skipped on that platform.
794#
795# Special cases:
796# test_pep277
797# The _ExpectedSkips constructor adds this to the set of expected
798# skips if not os.path.supports_unicode_filenames.
799# test_socket_ssl
800# Controlled by test_socket_ssl.skip_expected. Requires the network
801# resource, and a socket module with ssl support.
802# test_timeout
803# Controlled by test_timeout.skip_expected. Requires the network
804# resource and a socket module.
805
806_expectations = {
807 'win32':
808 """
809 test__locale
810 test_applesingle
811 test_al
812 test_bsddb185
813 test_bsddb3
814 test_cd
815 test_cl
816 test_commands
817 test_crypt
818 test_curses
819 test_dbm
820 test_dl
821 test_fcntl
822 test_fork1
823 test_gdbm
824 test_gl
825 test_grp
826 test_imgfile
827 test_ioctl
828 test_largefile
829 test_linuxaudiodev
830 test_mhlib
831 test_nis
832 test_openpty
833 test_ossaudiodev
834 test_poll
835 test_posix
836 test_pty
837 test_pwd
838 test_resource
839 test_signal
840 test_sunaudiodev
841 test_threadsignals
842 test_timing
843 test_wait3
844 test_wait4
845 """,
846 'linux2':
847 """
848 test_al
849 test_applesingle
850 test_bsddb185
851 test_cd
852 test_cl
853 test_curses
854 test_dl
855 test_gl
856 test_imgfile
857 test_largefile
858 test_linuxaudiodev
859 test_nis
860 test_ntpath
861 test_ossaudiodev
862 test_sqlite
863 test_startfile
864 test_sunaudiodev
865 """,
866 'mac':
867 """
868 test_al
869 test_atexit
870 test_bsddb
871 test_bsddb185
872 test_bsddb3
873 test_bz2
874 test_cd
875 test_cl
876 test_commands
877 test_crypt
878 test_curses
879 test_dbm
880 test_dl
881 test_fcntl
882 test_fork1
883 test_gl
884 test_grp
885 test_ioctl
886 test_imgfile
887 test_largefile
888 test_linuxaudiodev
889 test_locale
890 test_mmap
891 test_nis
892 test_ntpath
893 test_openpty
894 test_ossaudiodev
895 test_poll
896 test_popen
897 test_popen2
898 test_posix
899 test_pty
900 test_pwd
901 test_resource
902 test_signal
903 test_sqlite
904 test_startfile
905 test_sunaudiodev
906 test_sundry
907 test_tarfile
908 test_timing
909 """,
910 'unixware7':
911 """
912 test_al
913 test_applesingle
914 test_bsddb
915 test_bsddb185
916 test_cd
917 test_cl
918 test_dl
919 test_gl
920 test_imgfile
921 test_largefile
922 test_linuxaudiodev
923 test_minidom
924 test_nis
925 test_ntpath
926 test_openpty
927 test_pyexpat
928 test_sax
929 test_startfile
930 test_sqlite
931 test_sunaudiodev
932 test_sundry
933 """,
934 'openunix8':
935 """
936 test_al
937 test_applesingle
938 test_bsddb
939 test_bsddb185
940 test_cd
941 test_cl
942 test_dl
943 test_gl
944 test_imgfile
945 test_largefile
946 test_linuxaudiodev
947 test_minidom
948 test_nis
949 test_ntpath
950 test_openpty
951 test_pyexpat
952 test_sax
953 test_sqlite
954 test_startfile
955 test_sunaudiodev
956 test_sundry
957 """,
958 'sco_sv3':
959 """
960 test_al
961 test_applesingle
962 test_asynchat
963 test_bsddb
964 test_bsddb185
965 test_cd
966 test_cl
967 test_dl
968 test_fork1
969 test_gettext
970 test_gl
971 test_imgfile
972 test_largefile
973 test_linuxaudiodev
974 test_locale
975 test_minidom
976 test_nis
977 test_ntpath
978 test_openpty
979 test_pyexpat
980 test_queue
981 test_sax
982 test_sqlite
983 test_startfile
984 test_sunaudiodev
985 test_sundry
986 test_thread
987 test_threaded_import
988 test_threadedtempfile
989 test_threading
990 """,
991 'riscos':
992 """
993 test_al
994 test_applesingle
995 test_asynchat
996 test_atexit
997 test_bsddb
998 test_bsddb185
999 test_bsddb3
1000 test_cd
1001 test_cl
1002 test_commands
1003 test_crypt
1004 test_dbm
1005 test_dl
1006 test_fcntl
1007 test_fork1
1008 test_gdbm
1009 test_gl
1010 test_grp
1011 test_imgfile
1012 test_largefile
1013 test_linuxaudiodev
1014 test_locale
1015 test_mmap
1016 test_nis
1017 test_ntpath
1018 test_openpty
1019 test_poll
1020 test_popen2
1021 test_pty
1022 test_pwd
1023 test_strop
1024 test_sqlite
1025 test_startfile
1026 test_sunaudiodev
1027 test_sundry
1028 test_thread
1029 test_threaded_import
1030 test_threadedtempfile
1031 test_threading
1032 test_timing
1033 """,
1034 'darwin':
1035 """
1036 test__locale
1037 test_al
1038 test_bsddb
1039 test_bsddb3
1040 test_cd
1041 test_cl
1042 test_curses
1043 test_gdbm
1044 test_gl
1045 test_imgfile
1046 test_largefile
1047 test_linuxaudiodev
1048 test_locale
1049 test_minidom
1050 test_nis
1051 test_ntpath
1052 test_ossaudiodev
1053 test_poll
1054 test_sqlite
1055 test_startfile
1056 test_sunaudiodev
1057 """,
1058 'sunos5':
1059 """
1060 test_al
1061 test_applesingle
1062 test_bsddb
1063 test_bsddb185
1064 test_cd
1065 test_cl
1066 test_curses
1067 test_dbm
1068 test_gdbm
1069 test_gl
1070 test_gzip
1071 test_imgfile
1072 test_linuxaudiodev
1073 test_openpty
1074 test_sqlite
1075 test_startfile
1076 test_zipfile
1077 test_zlib
1078 """,
1079 'hp-ux11':
1080 """
1081 test_al
1082 test_applesingle
1083 test_bsddb
1084 test_bsddb185
1085 test_cd
1086 test_cl
1087 test_curses
1088 test_dl
1089 test_gdbm
1090 test_gl
1091 test_gzip
1092 test_imgfile
1093 test_largefile
1094 test_linuxaudiodev
1095 test_locale
1096 test_minidom
1097 test_nis
1098 test_ntpath
1099 test_openpty
1100 test_pyexpat
1101 test_sax
1102 test_sqlite
1103 test_startfile
1104 test_sunaudiodev
1105 test_zipfile
1106 test_zlib
1107 """,
1108 'atheos':
1109 """
1110 test_al
1111 test_applesingle
1112 test_bsddb185
1113 test_cd
1114 test_cl
1115 test_curses
1116 test_dl
1117 test_gdbm
1118 test_gl
1119 test_imgfile
1120 test_largefile
1121 test_linuxaudiodev
1122 test_locale
1123 test_mhlib
1124 test_mmap
1125 test_nis
1126 test_poll
1127 test_popen2
1128 test_resource
1129 test_sqlite
1130 test_startfile
1131 test_sunaudiodev
1132 """,
1133 'cygwin':
1134 """
1135 test_al
1136 test_applesingle
1137 test_bsddb185
1138 test_bsddb3
1139 test_cd
1140 test_cl
1141 test_curses
1142 test_dbm
1143 test_gl
1144 test_imgfile
1145 test_ioctl
1146 test_largefile
1147 test_linuxaudiodev
1148 test_locale
1149 test_nis
1150 test_ossaudiodev
1151 test_socketserver
1152 test_sqlite
1153 test_sunaudiodev
1154 """,
1155 'os2emx':
1156 """
1157 test_al
1158 test_applesingle
1159 test_audioop
1160 test_bsddb185
1161 test_bsddb3
1162 test_cd
1163 test_cl
1164 test_commands
1165 test_curses
1166 test_dl
1167 test_gl
1168 test_imgfile
1169 test_largefile
1170 test_linuxaudiodev
1171 test_mhlib
1172 test_mmap
1173 test_nis
1174 test_openpty
1175 test_ossaudiodev
1176 test_pty
1177 test_resource
1178 test_signal
1179 test_sqlite
1180 test_startfile
1181 test_sunaudiodev
1182 """,
1183 'os2knix':
1184 """
1185 test_al
1186 test_applesingle
1187 test_audioop
1188 test_bsddb185
1189 test_bsddb3
1190 test_cd
1191 test_cl
1192 test_commands
1193 test_dl
1194 test_gl
1195 test_imgfile
1196 test_linuxaudiodev
1197 test_mhlib
1198 test_mmap
1199 test_nis
1200 test_openpty
1201 test_ossaudiodev
1202 test_pty
1203 test_resource
1204 test_sqlite
1205 test_startfile
1206 test_sunaudiodev
1207 """,
1208 'freebsd4':
1209 """
1210 test_aepack
1211 test_al
1212 test_applesingle
1213 test_bsddb
1214 test_bsddb3
1215 test_cd
1216 test_cl
1217 test_gdbm
1218 test_gl
1219 test_imgfile
1220 test_linuxaudiodev
1221 test_locale
1222 test_macfs
1223 test_macostools
1224 test_nis
1225 test_ossaudiodev
1226 test_pep277
1227 test_plistlib
1228 test_pty
1229 test_scriptpackages
1230 test_socket_ssl
1231 test_socketserver
1232 test_sqlite
1233 test_startfile
1234 test_sunaudiodev
1235 test_tcl
1236 test_timeout
1237 test_unicode_file
1238 test_urllibnet
1239 test_winreg
1240 test_winsound
1241 """,
1242 'aix5':
1243 """
1244 test_aepack
1245 test_al
1246 test_applesingle
1247 test_bsddb
1248 test_bsddb185
1249 test_bsddb3
1250 test_bz2
1251 test_cd
1252 test_cl
1253 test_dl
1254 test_gdbm
1255 test_gl
1256 test_gzip
1257 test_imgfile
1258 test_linuxaudiodev
1259 test_macfs
1260 test_macostools
1261 test_nis
1262 test_ossaudiodev
1263 test_sqlite
1264 test_startfile
1265 test_sunaudiodev
1266 test_tcl
1267 test_winreg
1268 test_winsound
1269 test_zipimport
1270 test_zlib
1271 """,
1272 'openbsd3':
1273 """
1274 test_aepack
1275 test_al
1276 test_applesingle
1277 test_bsddb
1278 test_bsddb3
1279 test_cd
1280 test_cl
1281 test_ctypes
1282 test_dl
1283 test_gdbm
1284 test_gl
1285 test_imgfile
1286 test_linuxaudiodev
1287 test_locale
1288 test_macfs
1289 test_macostools
1290 test_nis
1291 test_normalization
1292 test_ossaudiodev
1293 test_pep277
1294 test_plistlib
1295 test_scriptpackages
1296 test_tcl
1297 test_sqlite
1298 test_startfile
1299 test_sunaudiodev
1300 test_unicode_file
1301 test_winreg
1302 test_winsound
1303 """,
1304 'netbsd3':
1305 """
1306 test_aepack
1307 test_al
1308 test_applesingle
1309 test_bsddb
1310 test_bsddb185
1311 test_bsddb3
1312 test_cd
1313 test_cl
1314 test_ctypes
1315 test_curses
1316 test_dl
1317 test_gdbm
1318 test_gl
1319 test_imgfile
1320 test_linuxaudiodev
1321 test_locale
1322 test_macfs
1323 test_macostools
1324 test_nis
1325 test_ossaudiodev
1326 test_pep277
1327 test_sqlite
1328 test_startfile
1329 test_sunaudiodev
1330 test_tcl
1331 test_unicode_file
1332 test_winreg
1333 test_winsound
1334 """,
1335}
1336_expectations['freebsd5'] = _expectations['freebsd4']
1337_expectations['freebsd6'] = _expectations['freebsd4']
1338_expectations['freebsd7'] = _expectations['freebsd4']
1339
1340class _ExpectedSkips:
1341 def __init__(self):
1342 import os.path
1343 from test import test_socket_ssl
1344 from test import test_timeout
1345
1346 self.valid = False
1347 if sys.platform in _expectations:
1348 s = _expectations[sys.platform]
1349 self.expected = set(s.split())
1350
1351 if not os.path.supports_unicode_filenames:
1352 self.expected.add('test_pep277')
1353
1354 if test_socket_ssl.skip_expected:
1355 self.expected.add('test_socket_ssl')
1356
1357 if test_timeout.skip_expected:
1358 self.expected.add('test_timeout')
1359
1360 if sys.maxint == 9223372036854775807L:
1361 self.expected.add('test_rgbimg')
1362 self.expected.add('test_imageop')
1363
1364 if not sys.platform in ("mac", "darwin"):
1365 MAC_ONLY = ["test_macostools", "test_macfs", "test_aepack",
1366 "test_plistlib", "test_scriptpackages"]
1367 for skip in MAC_ONLY:
1368 self.expected.add(skip)
1369
1370 if sys.platform != "win32":
1371 WIN_ONLY = ["test_unicode_file", "test_winreg",
1372 "test_winsound"]
1373 for skip in WIN_ONLY:
1374 self.expected.add(skip)
1375
1376 self.valid = True
1377
1378 def isvalid(self):
1379 "Return true iff _ExpectedSkips knows about the current platform."
1380 return self.valid
1381
1382 def getexpected(self):
1383 """Return set of test names we expect to skip on current platform.
1384
1385 self.isvalid() must be true.
1386 """
1387
1388 assert self.isvalid()
1389 return self.expected
1390
1391if __name__ == '__main__':
1392 # Remove regrtest.py's own directory from the module search path. This
1393 # prevents relative imports from working, and relative imports will screw
1394 # up the testing framework. E.g. if both test.test_support and
1395 # test_support are imported, they will not contain the same globals, and
1396 # much of the testing framework relies on the globals in the
1397 # test.test_support module.
1398 mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0])))
1399 i = pathlen = len(sys.path)
1400 while i >= 0:
1401 i -= 1
1402 if os.path.abspath(os.path.normpath(sys.path[i])) == mydir:
1403 del sys.path[i]
1404 if len(sys.path) == pathlen:
1405 print 'Could not find %r in sys.path to remove it' % mydir
1406 main()
Note: See TracBrowser for help on using the repository browser.