Built-in Context Managers in Python

This article explores concept of context manager and some of the Python’s built-in context managers. So, let’s begin.

What is a Context Manager?

A context manager is a Python object that allows you to manage resources. Context managers in Python are a convenient and efficient way to manage resources such as file handles, sockets, and database connections. It abstracts the setup and cleanup logic, making your code more expressive and less prone to errors. A context manager is used with the aid of the with statement.

A context manager is an object that specifies the runtime context to be used when executing a with statement. The context manager is responsible for entering and exiting the desired runtime context for the block of code to be executed.

Technically, a context manager is an object which must implement the context management protocol. The context management protocol involves the two methods below:

1. __enter__(): Called when entering the with block. It sets up the resource. The setup code involves opening or preparing resources, like a file, socket or thread pool.

2. __exit__(exc_type, exc_value, traceback): Called when exiting the with block. It handles cleanup such as such as closing a prepared resource. It also handles exceptions.

The with Statement

Traditionally, Python developers have used the try ... finally (or try ... except ... finally)construct to ensure proper resource cleanup such as closing files, releasing locks etc. However, this approach can lead to verbose and repetitive code. The with statement simplifies this process.

The with statement introduces a more concise and readable way to manage resources. It ensures that the necessary cleanup actions (such as closing files or releasing locks) occur automatically, even if exceptions are raised within the block.

The with statement in Python is a very important tool for properly managing external resources in your programs. It enables you to use context managers to automatically handle the setup and teardown phases; whenever you interact with external resources or perform activities that need those phases.

The “with” statement ensures that the necessary cleanup actions occur automatically, even if exceptions are raised within the block.

The with statement calls the above special methods of a context manager at appropriate time i.e. at the entry and exit of with statement’s block.

Using the Python with Statement

The Python with statement creates a runtime context that allows you to run a group of statements under the control of a context manager.

The syntax

The with statement uses the following general syntax:

with context_manager_expression as target_var:
    do_process(target_var)

Look at the syntax, The context manager object results from evaluating the expression after the with statement.

In the syntax, the as specifier is optional. If you provide a target_var using as keyword, then the return value of calling .__enter__() on the context manager object is bound to that variable.

Built-in Context Managers

There are two types of context managers in Python — regular context managers and asynchronous context managers.

Regular context mangers can be used anywhere in the program. Regular context managers can be used via “with” statement.

Furthermore, the context management protocol enables you to design your own context managers, allowing you to tailor the way you interact with system resources.

Python has built-in regular context mangers to make your life easy when you are coding for resource management i.e. when working with files, performing high-precision calculations, writing a multithreaded program etc.

The next sections will walk you through some of the most-often used built-in context managers in Python as listed below:

1 — File handling.

2 — Traversing directories.

3 — High Precision Calculations.

4 — Threading and Locks

Files I/O

The open() function returns a file object which is a context manager when used in combination with the with statement. This context manager allows manipulation of files in a with statement. Using open()via the with statement is the recommended way of working with files because the with statement ensures that open file objects will be closed automatically. The file objects are closed when the execution control leaves the with code-block.

An example code is shown below.

Let the file content of “my_file.txt” be:

Enjoy Python Programming with DevTechie!
This example is about file (resource) management using with statement.

Code:

with open('my_file.txt', 'r') as input_file:
    file_contents = input_file.read()
    print (f"The file content is: \n  {file_contents} " )
    # you can perform other operations on the file

print(input_file.read()) # will raise an error

Output:

he file content is: 
  Enjoy Python Programming with DevTechie!
This example is about file (resource) management using with statement. 
Traceback (most recent call last):
  File "C:\Users\MCA\Desktop\Python\python code\demo.py", line 6, in <module>
    print(input_file.read()) # will raise an error
          ^^^^^^^^^^^^^^^^^
ValueError: I/O operation on closed file.

The error in the above output proves that the file is closed after the with block.

Traversing Directories

In Python, the scandir() method is a powerful tool for working with directories.

Calling scandir()with a path to a given directory as an argument returns an iterator of os.DirEntry objects corresponding to the entries in a specified directory. The returned iterator also supports the context management protocol i.e. it can work as a context manager.

The scandir() function is a part of the os module (library). Hence it is also referred to as os.scandir()

Unlike os.listdir(), which returns a list of filenames, os.scandir() yields detailed information about each entry. Further, os.scandir() is specially tuned for optimal performance when traversing a directory structure.

The special entries like '.' (current directory) and '..' (parent directory) are not included in the results of os.scandir().

These DirEntry objects hold the filename string and provide simple methods to access additional data returned by the operating system.

os.scandir() takes a single parameter “path” which is Optional. If the path is not specified, then the current working directory is used as a path.

The below example uses this context manger in a with statement.

import os
with os.scandir(".") as dir_list:
     for dir in dir_list:
         print (f"Name: {dir.name}, Directory or File: {dir.is_dir()}, Size: {dir.stat().st_size} bytes")

The above code scans the current directory where this code script is stored by the path using the special symbol “.” for path.

Each entry dir in the list dir_list returned by scandir() is either a file or a directory. Each entry contains some method or attributes for more info about the directory or file.

The stat() method returns various info like mode, user id, size etc. In the code, st_size is the size of the file in bytes, if it is a regular file or a symbolic link.

The is_dir() function returns True for a directory (folder) entry, False for a file. Another function is_file() returns False for a directory (folder), True for a file.

There are many other info returned by stat() method like st_atime, st_ino, st_uid, st_gid etc.

Output:

Name: .ipynb_checkpoints, Directory or File: True, Size: 0 bytes
Name: .vscode, Directory or File: True, Size: 0 bytes
Name: asyncwith.py, Directory or File: False, Size: 1639 bytes
Name: carswebsite, Directory or File: True, Size: 0 bytes
Name: classes.py, Directory or File: False, Size: 11519 bytes
Name: controlstsmt.py, Directory or File: False, Size: 2644 bytes
Name: decorators.py, Directory or File: False, Size: 12473 bytes
Name: demo.py, Directory or File: False, Size: 2774 bytes
Name: dictionary.py, Directory or File: False, Size: 3177 bytes
Name: enumerate.py, Directory or File: False, Size: 372 bytes
Name: ex1.html, Directory or File: False, Size: 186 bytes
Name: file.py, Directory or File: False, Size: 4367 bytes
Name: fileImport.py, Directory or File: False, Size: 894 bytes
Name: flask, Directory or File: True, Size: 0 bytes
Name: functions.py, Directory or File: False, Size: 5816 bytes
Name: global.py, Directory or File: False, Size: 2131 bytes
Name: hello.py, Directory or File: False, Size: 2535 bytes
Name: is_versus_==.py, Directory or File: False, Size: 325 bytes
Name: iterator.py, Directory or File: False, Size: 1247 bytes
Name: lambdaFuntcion.py, Directory or File: False, Size: 1101 bytes
Name: Lists.py, Directory or File: False, Size: 3362 bytes
Name: mapfilterreduce.py, Directory or File: False, Size: 5978 bytes
Name: mapreducefilter.txt, Directory or File: False, Size: 1118 bytes
Name: modules.py, Directory or File: False, Size: 2931 bytes
Name: multilevelinheritance.py, Directory or File: False, Size: 0 bytes
Name: multipleinheritance.py, Directory or File: False, Size: 2328 bytes
Name: myclasses.py, Directory or File: False, Size: 1211 bytes
Name: my_file.txt, Directory or File: False, Size: 111 bytes
Name: os.py, Directory or File: False, Size: 1550 bytes
Name: re.txt, Directory or File: False, Size: 1263 bytes
Name: sets.py, Directory or File: False, Size: 0 bytes
Name: singleinheritance.py, Directory or File: False, Size: 3067 bytes
Name: tryexcept.py, Directory or File: False, Size: 1494 bytes
Name: Untitled.ipynb, Directory or File: False, Size: 4961 bytes
Name: untitled.txt, Directory or File: False, Size: 0 bytes
Name: Untitled1.ipynb, Directory or File: False, Size: 5994 bytes
Name: __pycache__, Directory or File: True, Size: 0 bytes

The for loop inside the with block let’s you iterate over the entries in the selected directory (".") and print their name and size on the screen.

The __exit__() method of the context manager calls the scandir.close() to ‌close the iterator (to close the acquired resources).

Recapitulate

This article talks only about regular built-in context managers. The article introduced some of the fantastic built-in context managers of Python world. The other two regular built-in context managers in the domain of High Precision Calculations and Threading and Locks will be taken up in a different article since this article is already a lengthy one. Happy Coding.