"async with" expression in Python

The with statement is used with asynchronous context managers. But let us first understand how context managers works in general.

Context managers provides convenience when working with objects that require set up and tear down operations. Typically, these are the objects that can be used in a with statement such as files, streams, sockets, database, etc.

Traditionally, a context manager object must implement the context manager protocol which consists of two methods i.e __enter__() and __exit__().The __enter__() method performs the setup operation while the __exit__() method performs the tear down operation. For example in case with files, the __enter__() method will open the file either for read, write or both, while the __exit__() method closes the file and releases any associated resources.

Any object that implements the context manager protocol is eligible for use in a with statement.

When using the with statement, the two methods of a context manager object i.e __enter__() and __exit__() are called automatically. __enter__() is called automatically as the first method before entering the with block while the __exit__() method is called immediately before leaving the block.

To understand this better let us implement a very simple context manager object.

import asyncio

class Example:
   def __enter__(self, *args, **kwargs):
      print("__enter__() called")
   def __exit__(self, *args, **kwargs):
      print("__exit__() called")

with Example():
   pass

As you can see above the two method have been called automatically, when we used the object in a with statement.

The with statement ensures that the __exit__() method of an object is called no matter how the block exists, even if due to an exception being raised.

In our example, the __enter__() method doesn't return anything, it simply displays a massage showing that it was called, but in most practical cases, it usually returns an instance of the context manager object through self.

import asyncio

class Example:
   def __enter__(self, *args, **kwargs):
      print("__enter__() called")
      return self
   def __exit__(self, *args, **kwargs):
      print("__exit__() called")

with Example() as manager:
   print(manager)

As shown above, the object returned by __enter__() is accessible within the with statement through the alias name used in the as clause.

Asynchronous context managers

We have seen how regular context managers work, now let us look at asynchronous context managers.

Asynchronous context managers must also implement two methods i.e __aenter__() and __aexit__().

Unlike the regular context manager, an asynchronous context manager is capable of pausing the execution of its enter and exit methods. This is very useful as connection to some context manager objects like sockets and databases may take time or face delays.  In case of delays, the asynchronous context manager can pause the current task, move to another task and then return later on.

When working with asynchronous context managers, we need to use the async with statement rather than just with as in traditional context managers.

import asyncio

class AsyncExample:
   def do_something(self):
      print('Hello, World!')

   async def __aenter__(self, *args, **kwargs):
      print("__aenter__() called")
      await asyncio.sleep(0.05) #simulate a delay
      return self

   async def __aexit__(self, *args, **kwargs):
      print("__aexit__() called")
      await asyncio.sleep(0.005) #simulate a delay

async def main():
   print('Started ')
   async with AsyncExample() as manager:
      #within the manager
      manager.do_something()
   print("Finished")

asyncio.run(main())

It is very important to note that  the context manager methods i.e __aenter__() and __aexit__() are implemented as coroutines i.e with async def.

The __aenter__() method displays a message showing that it was called and then  returns an instance of the manager(through self) which then becomes accessible inside the with block through the alias name used in the as clause i.e manager in the above case.

The async with statement  cannot be used on its own, it has to be inside of a coroutine(an async function). Trying this will raise a SyntaxError exception as demonstrated below.

import asyncio

class AsyncExample:
   async def __aenter__(self, *args, **kwargs):
      print("__aenter__() called")

   async def __aexit__(self, *args, **kwargs):
      print("__aexit__() called")

async with AsyncExample() as manager:
   pass

asyncio.run(main())

Asynchronous context managers without "async with"

While it is generally considered a good practice to use asynchronous context managers in an "async with" statement and traditional context managers in a regular with statement, it is also possible to use them manually outside of the with statements. This involves explicitly calling the set up and tear down methods.

For example,  you can open a file in two ways. One is within a with statement as shown below:

with open('myfile.txt') as f:
   #do something

In the above case, the file will be closed automatically. The other approach  where we do not use the with statement requires you to manually perform the tear down operation by calling the close() method.

f = open('myfile.txt') #setup operation

f.read() #do something

f.close() #teardown operation.

The same can be replicated with an asynchronous object so that it has two interfaces, one which will be used automatically by the async with statement and the second which we can use manually.

Conclusion

• Context managers are normally used with objects that require set up annd tear down operations.
• A context manager object must implement the enter and exit methods.
• Traditional context manager implements ___enter__() and __exit__() methods.
• Asynchronous context manager implements __aenter__() and __aexit__() methods.
• Asynchronous context manager can pause the execution of either __aenter__() or __aexit__() in case of delays or other factors.
• The with statement is used to manage traditional context managers.
• The async with statement  is used to manage asynchronous context managers.