mmap module in Python

The mmap module in the standard library provides a way to access operating system's memory mapping functions.

Memory-mapping is a technique that allows files to be mapped directly into the virtual memory of a process/program.

Traditionally, file data is typically read into a buffer, and then copied into the application's memory space. Memory-mapping avoids this extra copying step as the file is directly mapped into memory.

This technique typically improves file operations as the file is accessed directly without its contents being loaded into the application's main memory.

The mmap() Function

The basic tool offered by the mmap module is the mmap() function which allows you to map a file or a file-like object into memory.

The basic syntax of the mmap() function is as follows:

mmap.mmap(fileno, access = mmap.ACCESS_READ, length = 0, offset = 0)

Reading from a file

The first argument of the mmap() function is a file descriptor, either from the fileno() method  of file objects, or an opened file object that has been opened using the builtin open() function.

The mapped file object, returned by the mmap() functions can be used either as file objects or as mutable byte strings.

As file objects

#import the mmap module
import mmap

with open("myfile.txt", 'r') as file:
    
    with mmap.mmap(file.fileno(), 0, access = mmap.ACCESS_READ) as mmaped_file:
    
        #read the file contents
        print(mapped_file.read())
    
 

As shown above, the memory-mapped file objects are very similar to regular file objects, the share some common methods such as read(), readline(), seek(), write(), tell(), etc.

as byte strings

The memory-mapped file objects also have a string API which allows them to be manipulated as if they are mutable strings.

#import the mmap module
import mmap

with open("myfile.txt", 'r') as file:
    
    with mmap.mmap(file.fileno(), 0, access = mmap.ACCESS_READ) as mapped_file:
    
        #read the first 20 bytes from the file
        print(mapped_file[:20])
        
        #find a sub-string from the file
        print(mapped_file.find(b"Hello"))

        #find a sub-string from the file starting from right
        print(mapped_file.rfind(b"Hello"))
 
 

Writing to a file

To make a memory-mapped file to accept write operations, we need to open the target file object using the 'r+' mode(not 'w') . We then use either of the two supported API's to add or remove data to or from the file.

#import the mmap function
import mmap

with open("myfile.txt", 'r+') as file:
    
    #open the file with 40 bytes length
    with mmap.mmap(file.fileno(), 40, access = mmap.ACCESS_WRITE) as mapped_file:
    
        #write to the file
        mapped_file.write(b"Hello, world!") 

        #write to the file using string API
        mapped_file.seek(0)
        mapped_file[13:28] = b"Goodbye, world!"

        print(mapped_file.read(28))