Python functools module

STDL Algorithms

functools module

module usage

reduce()

lru_cache()

total_ordering()

update_wrapper()

wraps()

singledispatch()

cmp_to_key() function

cached_property()

itertools module

module usage

accumulate()

chain()

combinations()

permutations()

combinations_with_replacement()

tee()

product()

repeat()

filterfalse()

takewhile()

dropwhile()

zip_longest()

cycle()

count()

islice()

starmap()

groupby()

pairwise()

compress()

infinite iterator

Operator module

module usage

contextlib module

module usage

The functools module in the standard library provides functions and classes for manipulating functions. The primary tool defined in the module is the partial class, which as we will see in a while, allows the partial application of a function. This means that a callable object is created with some of the arguments already filled in, reducing the number of arguments that must be supplied to subsequent calls.

The module also provides a number of decorators which can be used to wrap functions and classes in order to extend them with additional capabilities.

Partial Objects

The partial class makes it possible to create a version of a callable object( e.g functions) with some of the arguments pre-filled.

Syntax:

partial(callable, *args, **kwargs)

ExampleEdit & Run

#import the functools module
import functools

myprint = functools.partial(print, sep = ', ')

#use the partial function
myprint(*range(10))
myprint('Python', 'C++', 'Java', 'Ruby', 'Javascript') Visualize

Output:

0, 1, 2, 3, 4, 5, 6, 7, 8, 9Python, C++, Java, Ruby, Javascript[Finished in 0.013403145130723715s]

In the above example, we created a new partial function, myprint(), from the builtin print() function. The original print() function would have separated each item with a space, but by using functools.partial function, we were able to set the sep keyword argument to a comma and a space( ', ').

In the following example we use a user-defined function to create a partial function.

ExampleEdit & Run

import functools

def evaluate(num1, num2, oper):
    """Evaluates the arithmetic result of applying operator, 'oper' on 'num1' and 'num2'"""

    num1, num2 = int(num1), int(num2)

    result = None
    match oper:
        case "+":
            result = num1 + num2
        case "-":
            result = num1 - num2
        case "*":
            result = num1 * num2
        case "/":
            result = num1 / num2
        case "%":
            result = num1 % num2
        case "**":
            result = num1 ** num2
        case _:
            return ("Invalid operator '%s'"%oper)

    return f"{num1} {oper} {num2} = {result}"

add = functools.partial(evaluate, oper = '+')

print(add(3, 5))
print(add(10, 20))
print(add(50, 60)) Visualize

Output:

3 + 5 = 810 + 20 = 3050 + 60 = 110[Finished in 0.013146159937605262s]

The following example uses a class rather than a function as the callable

ExampleEdit & Run

import functools

class Person:
   def __init__(self, name, country, nationality):
      self.name = name
      self.country = country
      self.nationality = nationality

   def info(self):
      return f"""name: {self.name}
country: {self.country}
nationality: {self.nationality}"""

#create a partial constructor for people of a specific country
indian = functools.partial(Person, country = 'India', nationality = "Indian")

p1 = indian('Rahul')
print(p1.info()) Visualize

Output:

name: Rahulcountry: Indianationality: Indian[Finished in 0.013028382789343596s]

Acquire properties of the original callable

By default, the returned partial object, does not inherit the __name__ and the __doc__ attributes from the original callable object.This attributes are very essential especially for debugging purposes.

ExampleEdit & Run

import functools

myprint = functools.partial(print, sep = ', ')

#get the __doc__ attribute
print(myprint.__doc__) Visualize

Output:

partial(func, *args, **keywords) - new function with partial application    of the given arguments and keywords.[Finished in 0.012763238977640867s]

As you can see above, myprint() function does not inherit the docstring of the print() function, from which it is a partial object.

The update_wrapper() function attaches the relevant information to the partial object from another object.

ExampleEdit & Run

import functools

myprint = functools.partial(print, sep = ', ')

functools.update_wrapper(myprint, print)


#get the __doc__ attribute
print(myprint.__doc__) Visualize

Output:

Prints the values to a stream, or to sys.stdout by default.  sep    string inserted between values, default a space.  end    string appended after the last value, default a newline.  file    a file-like object (stream); defaults to the current sys.stdout.  flush    whether to forcibly flush the stream.[Finished in 0.012765167048200965s]

partialmethod class

The partial class, as we have seen, works with bare functions. The partialmethod class similarly creates partial objects but from methods. The partial method should be defined inside the class as shown below.

ExampleEdit & Run

import functools

class Person:
   def __init__(self, name, country, nationality):
      self.name = name
      self.country = country
      self.nationality = nationality

   def change_nationality(self, new_country, new_nationality):
       self.country = new_country
       self.nationality = new_nationality

   #create a partial metthod for 'change_nationality'
   to_japanese = functools.partialmethod(change_nationality, new_country = 'Japan', new_nationality = 'Japanese')

p1 = Person('Rahul', 'India', 'Indian')
print(p1.name, p1.country, p1.nationality)

#call the partialmethod 'to_japanese' on object
p1.to_japanese()
print(p1.name, p1.country, p1.nationality) Visualize

Output:

Rahul India IndianRahul Japan Japanese[Finished in 0.012537727132439613s]

Functions defined in the functools module

Apart from the two classes that we have looked at i.e partial and partialmethod, the module also defines several utility functions that can be used to further manipulate functions through decoration.

`cmp_to_key()`

Some high order functions such builtin sorted(), filter(), max(), and min(), takes an optional parameter called key which is used to specify a particular function to be applied to each element of the iterable prior to making comparisons.

ExampleEdit & Run

L = [-10, -2, 5, 0, -1, 4]

#the key function
def absolute(x):
     if x < 0:
        return -x
     return x

#sort the elements by their absolute value
print(sorted(L, key = absolute)) Visualize

Output:

[0, -1, -2, 4, 5, -10][Finished in 0.010473222006112337s]

The cmp_to_key() function is used to create a key function from a traditional comparison function. The function given as an argument must return either 1, 0, or -1 depending on the arguments it receives.

ExampleEdit & Run

import functools

L = [('Python', 3), ('Java', 4),  ('C++', 2), ('Javascript', 1), ('PHP', 5)]


# function to Sort the tuples by their second items

@functools.cmp_to_key
def func(a, b):
    if a[-1] > b[-1]:
        return 1
    elif a[-1] < b[-1]:
        return -1
    else:
        return 0

#sort the elements by their last element
print(sorted(L, key = func)) Visualize

Output:

[('Javascript', 1), ('C++', 2), ('Python', 3), ('Java', 4), ('PHP', 5)][Finished in 0.012865976197645068s]

`reduce()`

The reduce() function applies a given function cumulatively to an iterable. It typically takes two arguments: a function and an iterable, it then applies the function cumulatively on the elements of the given iterable.

Syntax:

reduce(func, iterable)

The function given as func must accept two arguments.

ExampleEdit & Run

import functools

L = [1, 2, 3, 4, 5, 6, 7, 8, 9]

def add(a, b):
    return a + b

cumulative_sum = functools.reduce(add, L)

print(cumulative_sum) Visualize

Output:

45[Finished in 0.012869203928858042s]

ExampleEdit & Run

import functools

L = [1, 2, 3, 4, 5, 6, 7, 8, 9]

def prod(a, b):
    return a * b

cumulative_prod = functools.reduce(prod, L)

print(cumulative_prod) Visualize

Output:

362880[Finished in 0.012642775196582079s]

`cache()`

The cache() function is used to cache the result of an expensive computation for future use. This means that if the same function is called with the same parameters, the results are cached and the computation does not need to be redone.

ExampleEdit & Run

import functools 

@functools.cache
def fibonacci(num): 
    if num in [0, 1]: 
       return num 
    return fibonacci(num - 1) + fibonacci(num - 2)

# Let's run it and check the cache 
print(fibonacci(10))
print(fibonacci(10)) Visualize

Output:

5555[Finished in 0.012312764069065452s]

`lru_cache()`

The lru_cache() function implements a least recently used (LRU) cache for efficient memoization of a function. When the function is called, the lru_cache() will store any inputs and outputs of the function in an order of least recently used. When the cache is full, the least recently used items are discarded to make space for new data. This is beneficial as it allows the cache to store more relevant data rather than having to store all data from the function call.

ExampleEdit & Run

import functools 

@functools.lru_cache(maxsize=4) 
def fibonacci(num): 
    if num in [0, 1]: 
       return num 
    return fibonacci(num - 1) + fibonacci(num - 2)

# Let's run it and check the cache 
print(fibonacci(10))
print(fibonacci(10)) Visualize

Output:

5555[Finished in 0.01287722890265286s]

`singledispatch()`

The singledispatch() decorator function is used to create functions that can dispatch on the type of a single argument such that certain behaviors are dependent on the type. When decorated with singledispatch(), a function becomes a "generic function", meaning that it can have multiple different implementations, depending on the type of the argument passed to it.

The implementation for a particular type is registered using the register() method of the decorated function.

ExampleEdit & Run

import functools 

@functools.singledispatch 
def add(a, b):
    raise NotImplementedError

@add.register(str) 
def _(a, b): 
    return f"{a} {b}" 

@add.register(int) 
def _(a, b): 
    return a + b 

#with ints
print(add(1, 2)) 

#with strings
print(add("Hello", "World"))

#an error is raised for a non-implemented types
print(add([1, 2], [3, 4])) Visualize

Output:

3Hello WorldTraceback (most recent call last):  File "<string>", line 22, in <module>  File "/app/.heroku/python/lib/python3.11/functools.py", line 909, in wrapper    return dispatch(args[0].__class__)(*args, **kw)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "<string>", line 5, in addNotImplementedError[Finished in 0.02100956393405795s]

`@total_ordering`

The @functools.total_ordering() is used to automatically fill in comparison methods for a class by defining only two of the six rich comparison methods (__lt__, __le__, __eq__, __ne__, __gt__ and __ge__) . This is typically achieved by decorating the class with the @total_ordering decorator and defining the __eq__ method alongside any other of (__lt__, __le__,__gt__, __ge__).

ExampleEdit & Run

import functools

@functools.total_ordering 
class Student: 
    def __init__(self, name, grade):
        self.name = name
        self.grade = grade 

    def __repr__(self): 
        return self.name 

    def __eq__(self, other): 
        return self.grade == other.grade 

    def __lt__(self, other): 
        return self.grade < other.grade 

john = Student("John", 83) 
jane = Student("Jane", 87) 

print(john == jane) 
print(john != jane)
print(john < jane) 
print(john <= jane)
print(john > jane)
print(john >= jane)
 Visualize

Output:

FalseTrueTrueTrueFalseFalse[Finished in 0.013035796815529466s]

STDL Algorithms

functools module

itertools module

Operator module

contextlib module

Python functools module

Partial Objects

Acquire properties of the original callable

partialmethod class

Functions defined in the functools module

cmp_to_key()

reduce()

cache()

lru_cache()

singledispatch()

@total_ordering

Related articles

`cmp_to_key()`

`reduce()`

`cache()`

`lru_cache()`

`singledispatch()`

`@total_ordering`