You can benchmark Python programs without making any modifications to the code by using the time command.

In this tutorial, you will discover how to benchmark Python programs using the time command.

Let’s get started.

Need to Benchmark Python Script

Benchmarking Python code refers to comparing the performance of one program to variations of the program.

Benchmarking is the practice of comparing business processes and performance metrics to industry bests and best practices from other companies. Dimensions typically measured are quality, time and cost.

— Benchmarking, Wikipedia.

Typically, we make changes to the programs, such as adding concurrency, in order to improve the performance of the program on a given system.

Improving performance typically means reducing the run time of the program.

Therefore, when we benchmark programs in Python after adding concurrency, we typically are interested in recording how long a program takes to run.

Benchmarking ad hoc Python code is relatively straightforward, for example, we can use the timeit module to benchmark snippets or the time module to manually benchmark code blocks.

1. Benchmark Python code snippets with the timeit module.
2. Benchmark Python blocks snippets with the time module.

Nevertheless, Python does not provide a straightforward way to benchmark an entire Python program or script, without modifying the program.

How can we benchmark the performance of entire programs in Python?

Benchmark With the time Command

The time command is a tool that can be used to measure the execution time of any program on the command line interface.

Let’s first review the command line interface and then take a closer look at the time command.

What is the Command Line Interface

The command line or command line interface is a way of interacting with the computer using text commands, as opposed to clicking around on a graphical interface with a mouse.

A command-line interface (CLI) is a means of interacting with a device or computer program with commands from a user or client, and responses from the device or program, in the form of lines of text.

— Command-line interface, Wikipedia.

It is known by different names on different platforms, typically after the name of the program that provides the interface.

For example:

• Command Prompt: Windows
• Terminal: macOS
• Shell: Unix and Linux

Python can be used on the command line directly via the “python” command.

This will open the Python interpreter.

We can execute a Python script directly using the Python interpreter.

This can be achieved by specifying the “python” command followed by the script name.

For example:

This will execute the Python program in the script and return once the script is completed.

We can also call the Python interpreter with flags. For example, we can execute a line of code directly using the Python interpreter via the -c flag:

This will start the Python interpreter, execute the line of code, report the result, and close the interpreter.

Now that we know about the command line, let’s look at the time command.

What is the time Command?

The time command, also called the time Unix command, is a command line program for reporting the execution time of programs.

In computing, time is a command in Unix and Unix-like operating systems. It is used to determine the duration of execution of a particular command.

— time (Unix), Wikipedia

It is referred to as the “time Unix command” because it was originally developed for the Unix operating system.

The command is not available on almost all Unix-like operating systems and may be implemented as an executable command (e.g. GNU time) or as a keyword in the shell (e.g. bash time).

The “time” command runs a program, such as a Python script, and reports the overall execution time.

This approach is recommended if you want to benchmark the entire program without modifying it anyway.

It returns 3 benchmark results, real-time, user time, and system time.

Real-time is the wall clock time, the duration of the program.

The term “real time” in this context refers to elapsed wall-clock time, like using a stop watch. The total CPU time (user time + sys time) may be more or less than that value. Because a program may spend some time waiting and not executing at all (whether in user mode or system mode) the real time may be greater than the total CPU time. 

— time (Unix), Wikipedia

The CPU time is calculated as the sum of user time and system time which specify how long the program spent in each mode.

The total CPU time is the combination of the amount of time the CPU or CPUs spent performing some action for a program and the amount of time they spent performing system calls for the kernel on the program’s behalf. 

— time (Unix), Wikipedia

If the program is blocked by other programs running at the same time or sleeps, the CPU time may be shorter than the real-time. This is because clocks that record CPU time are paused when the program is blocked, and resume once the program resumes executing.

• Real-time: Wall-clock execution time of the program.
• CPU-time: Sum of user time and system time when executing the program.

Next, let’s consider the benefits of the time command.

Benefits of the time Command

Using the time Unix command to benchmark Python code provides several benefits, especially when you need a quick and simple way to measure the overall execution time of a script or program.

Here are some advantages of using the time command for benchmarking:

1. Quick and Convenient: The time command is readily available on Unix-like systems, making it a convenient option for quickly measuring the execution time of a Python script without the need for additional code modifications.
2. No Code Modification: You don’t need to modify your Python code to perform benchmarking. Simply prefix your Python command with the time command in the terminal.
3. Total Execution Time: The time command provides the total wall-clock time taken by the Python script to complete its execution, including any setup and teardown operations.
4. Simple Output: The output of the time command provides information about real (wall clock), user, and system time, which can give you insights into the distribution of time spent during execution.
5. Measures All Overheads: The time command measures all overheads associated with the execution of the Python script, including interpreter startup, module loading, and other setup operations.
6. External Factors: The time command captures external factors that might influence execution time, such as system load, I/O operations, and other concurrent activities.
7. Works with Any Script: The time command is not limited to Python scripts. You can use it to benchmark any executable, making it a versatile tool for performance measurement.
8. Built-in Statistics: The time command provides basic statistics about the execution time, including user time, system time, and real-time. These statistics can help you quickly assess the performance of your code.
9. Readily Available: Since the time command is a standard Unix utility, you don’t need to install any additional libraries or tools to use it for benchmarking.

Despite these benefits, it’s important to note that the time command has limitations.

Limitations of the time Command

The time command is not available on all platforms.

Generally, it is available on Unix and Unix-like systems, such as:

• Unix
• Linux
• macOS

It is not available on Windows platforms by default.

Nevertheless, it can be installed using a third-party software package, such as Cygwin or Microsoft PowerToys.

How to Benchmark with the time Command

The time command is used by pre-pending it to a normal command on the command line.

When using time to benchmark the execution time of Python programs, it is prepended to the python command.

For example:

This will run the script normally.

Once the script is completed, a summary of the execution time will be reported.

And that’s all there is to it.

Example of Benchmarking with the time Command

We can explore how to benchmark a Python script using the time command.

In this case, we can develop our program normally. We do not need to modify it to add any benchmarking code.

Our program will call a function that performs some CPU-intensive task. It will create a list of 100,000,000 squared integers.

The complete program is listed below.

We can then save our script to a file, such as “benchmark_time.py“.

Normally we would execute the program from the command line using the python command.

For example:

In order to benchmark the execution time of the Python program, we can prepend the “time” command to the execution of our script.

For example:

Running the example reports the duration both in terms of real-time (wall clock time) and CPU time (user + sys).

I recommend focusing on real-time.

In this case, we can see that the script took about 6.335 seconds to complete.

This highlights how we can benchmark a Python script using the time command.

Further Reading

This section provides additional resources that you may find helpful.

Books

• Python Benchmarking, Jason Brownlee (my book!)

Also, the following Python books have chapters on benchmarking that may be helpful:

• Python Cookbook, 2013. (sections 9.1, 9.10, 9.22, 13.13, and 14.13)
• High Performance Python, 2020. (chapter 2)

Guides

• 4 Ways to Benchmark Python Code
• 5 Ways to Measure Execution Time in Python
• Python Benchmark Comparison Metrics

Benchmarking APIs

• time — Time access and conversions
• timeit — Measure execution time of small code snippets
• The Python Profilers

References

• Benchmark (computing), Wikipedia.
• Profiling (computer programming), Wikipedia.

Takeaways

You now know how to benchmark Python programs using the time command.

Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Irena Carpaccio on Unsplash