python

To write something to a file or to read the content of a file in Python it is required to open() and close() the file properly.

The best practice in Python is to read/write to a file by using the with statement – the replacement for commonly used try...finally error-handling statements.

The with statement ensures that the file is closed when the block inside it is exited.

This note shows the best practice Python examples of how to write to a file or read the file’s content and save it to a variable. (more…)

Selenium is an open source tool for web browsers automation that is widely used in tandem with Python to test web applications.

As these tests are often automated and run on dedicated nodes, there is a sense to run Selenium in a headless mode, i.e. without opening a browser.

This note shows how to run headless Selenium in Python on the example of Chrome and Firefox browsers controlled by a ChromeDriver and GeckoDriver correspondingly. (more…)

Selenium is an open source tool that automates web browsers and is widely used for automated testing of web applications.

With Python and Selenium you can easily create a script that searches a web-page for a button containing a particular text and clicks on it.

This note shows an example of how to find a button by text and click on it using Selenium and Python. (more…)

It may happen that you execute a Python script and nothing happenes, i.e. it seems that it “hangs” without showing any output.

Even if you explicitly include some print() statements to debug the issue, it may still not print anything to a terminal.

One of the reasons of why your Python script does not show any output is because it buffers stdout and stderr steams instead of printing them.

This also can happen if you execute the Python script from a CI/CD pipeline (e.g. using Jenkins, Gitlab-CI, TeamCity, etc.) or if you run it using a Dockerfile.

This short note shows how to solve the issue when the Python script “hangs” and doesn’t show any output. (more…)

Selenium is an open source tool that automates web browsers and is widely used for automated testing of web applications.

With Python and Selenium you can easily create a script that automates login to a website, that is very useful, for example, for web scraping.

This note shows how to create a Python script to login to a website automatically using Selenium on the example of Instagram. (more…)

Some websites can block access to prevent web scraping, that can be easily detected if your Python script is sending multiple requests in a short period of time.

To not get banned you can try to add random delays between queries.

For this you can use the Python’s sleep() function that suspends (waits) execution of the current thread for a given number of seconds with the randint() function that returns a random integer. (more…)

A Python can be used to download a text or a binary data from a URL by reading the response of a urllib.request.urlopen.

The downloaded data can be stored as a variable and/or saved to a local drive as a file.

Below you will find the examples of the Python code snippets for downloading the different types of files from URLs and storing them as variables or saving locally. (more…)

A ‘User-Agent’ HTTP request header is a string that a web browser is sending to a web server along with each request to identify itself.

The ‘User-Agent’ string contains information about which browser is being used, what version and on which operating system.

Some websites block access from non-web browser ‘User-Agents’ to prevent web scraping, including from the default Python’s requests ‘User-Agent’.

In this note i will show how to set the ‘User-Agent’ HTTP request header while using the Python’s requests library. (more…)