Use cases • API • Motivations • Key Features • Installation • Download • License
The chain function enables the use of the chaining syntax:
chain(<init_val>)(func, *params)(...)()
. The chain function is a shorthand for the Just monad, monad(<init_val>, Just)(...)()
# With monad
val = np.array([[1,2,3],[1,2,3]])
chain(val)(np.transpose)(np.sum, axis=1)(np.mean)(lambda x: x/100)()
It is much more verbose without using monads:
# Without
data = np.array([[1,2,3],[1,2,3]])
transposed_data = np.transpose(data)
summed_data = np.sum(transposed_data, axis=1)
mean_data = np.mean(summed_data)
perc_data = mean_data / 100
Clearly, the syntax becomes more more succinct using notmonad. Moreover, there is no need to come up with many intermediate variable names, preventing the pollution of the namespace.
With a pipeline of transformations, error handling means that you end up with the try-except/catch tower of terror or a pyramid of doom. While TanStack Query style [value, error] destructuring works, the Maybe Monad solves this problem much more elegantly and more concisely.
Railroad error handling with Maybe monad:
val = np.array([[1,2,3],[1,2,3]])
monad(val, compose(maybe))(np.transpose)(np.sum, axis=1)(np.mean)(lambda x: x/0)(lambda x: x/100)() == None
Using try except results in the formation of tower of terror:
try:
data = np.array([[1,2,3],[1,2,3]])
except Exception as e:
data = None
try:
transposed_data = np.transpose(data)
except Exception as e:
transposed_data = None
try:
summed_data = np.sum(transposed_data, axis=1)
except Exception as e:
summed_data = None
try:
mean_data = np.mean(summed_data)
except Exception as e:
mean_data = None
try:
perc_data = mean_data / 100
except Exception as e:
perc_data = None
One downside of using such one-liner chains is that if one function within the chain fails, the error message given does not specify which part of the chain that failed. To resolve this, we can use the Debug monad.
monad(5, compose(debug, maybe))(add, 1)(lambda x: x / 0)(add, 3).keywords["_debug_trace"]
# OUTPUT:
# [
# {
# "func": "add",
# "args": (5, 1),
# "kwargs": {},
# "value": 6,
# "errors": "",
# "repr": "add(5, 1) -> 6 []",
# },
# {
# "func": "<lambda>",
# "args": (6,),
# "kwargs": {},
# "value": None,
# "errors": "division by zero",
# "repr": "<lambda>(6,) -> None [division by zero]",
# },
# ]
The debug trace shows exactly the functions that run at each step of the pipeline, the arguments that it was called with, the return values and the errors at each step.
Monads can be combined together using the compose
function. This allows you to use multiple monads in a single data pipeline. For example:
monad(5, compose(debug, log, shout maybe))(add, 1)(lambda x: x / 0)(add, 3)()
# OUTPUT:
# shout: I am shouting! add
# log: add
# debug: {
# "func": "add",
# "args": (5, 1),
# "kwargs": {},
# "value": 6,
# "errors": "",
# "repr": "add(5, 1) -> 6 []",
# },
# shout: I am shouting! lambda
# log: lambda
# debug: {
# "func": "<lambda>",
# "args": (6,),
# "kwargs": {},
# "value": None,
# "errors": "division by zero",
# "repr": "<lambda>(6,) -> None [division by zero]",
# }
With these composed monads, we run each of these monads at every step of the function pipeline, as a result, we will get a print of 'I am shouting' from the shout
monad at every step. All this computation is done with the compose
function. As the name suggests, compose
function allows you to compose multiple monads and use them together. It takes in monads as an arguments and returns a new monad which is a composition of all the monads that was passed in.
We can see monads as the functions that in the intermediary step between two chained functions. There are two main types of monads:
- Side-effect monads
- Caller monads
Caller monads execute the function passed with the arguments provided and hence performs data transformation on the initial value. This can be indicated by the @caller
decorator. In contrast, side-effect monads do not modify the underlying data, but instead log something, print something, do some intermediary computaton and pass that onto the next step in the pipeline.
Any composition of monad can only contains a single caller monad and any number of side-effect monads. This is because applying two caller monads will cause the data transformation function to be applied twice.
NotMonad also allows you to create your own custom monads to fit your use case. A monad can be used via the compose
function. To create a monad that is compatible with the compose
function, it must comply with the following format:
- Args:
value, func, *args, **kwargs
- Returns:
value, func, *args, **kwargs
If the function is called to modify the value returned, then it must have the @caller
decorator.
With the introduction of loops and memory storage, you can now write any python application entirely within NotMonad.
Loops can be written inside a lambda function or using the loop
function. If else statements can be written inside a lambda function.
Now, with the memory storage API, you can post your current calculations into the function's memory, and hop onto a new set of calculations by __mount
or __get
, and hence continue calculations from there.
After getting both to the stage where you want to combine both calculations, you can use the __call
method.
chain(<val>)(<func>, *params)()
- chain functions together, last parentheses returns the value
monad(<val>, <Just | Maybe>)(<func>, *params)()
- Just or Maybe (with cap first letter) are convenient monads that allows the use of monads without compose
monad(<val>, compose(<monads>, <monads>,..))(<func>, *params)()
- Compose monads together
In full admission, I do not study computer science or math, I do not understand monads or anything about category theory. Therefore, my use of the word 'monad' here is obviously very inaccurate and likely outright wrong.
This project spawned out of a need. I was performing some data analysis that requried lots of data augmentation and manipulation. It resembled a series of functions being applied to the same piece of data with that piece of data being passed from one function to the next.
In basic python, you can either do this by nesting one function into another, or you can create a bunch of intermediary variables names for each step. Nesting one function into another is a pain, because you start losing track deep within the parenthesis, and the function names read from right to left (which is inconvenient). Creating intermediary variables pollute the namespace, and when working with something like Jupyter notebooks, it often results in namespace pollution related bugs. Moreover, it is just a pain to think of variables names that you would not use again.
Chaining functions solves this problem, it prevents nesting, allows an easy to read left to right syntax, and pure use of parenthesis allows it to be very succinct, and also avoids namespace pollution.
Traditionally, monads are created by wrapping a value with a 'container'. Then that container has functions that allow another monad to act on it with a flatmap (to ensure only 1 layer of wrapping). However, the disadvantage is that the functions have to be responsible for the wrapping. That means that the monadic functions that you write cannot be used anywhere else. Moreover, you definitely cannot be lambda functions on the fly when you want to use chaining with traditional moands. Therefore, the approach was taken to separate the concerns of 'wrapping' and 'unwrapping' away, such that the only thing the functions need to do is to perform the data augmentation and nothing else.
In the context of this project, I like to think of monads as advanced decorators for every function in the pipeline that pass state from one function to the next.
Thinking of monads this way, allows you to realise that monads can literally be any function that does some intermediary computation. So here, we can do error handling, logging or even send an API request as part of the monad! Moreover, it then also makes sense to be able to do two intermediary computations between the functions. And this spawned monad composition, you can combine, mix and match any monads you have and it would work just as expected. Even more interestingly, we realise that this combination of monads can be done with a monad too. Therefore, there is a mmonad
function that allows you to chain together monads to create a new combined monad. The monad's monad! It is interesting to think about the implications:
The chain of functions can be thought of as a series of data transformations...
f(x): a -> b
g(x): b -> c
chain(x)(f)(g): a -> c
We can see that for function chaining to work, the type of the output of f must match up with the type of the output of g. And it is because of this reason, the order matters. The results of chain(x)(f)(g)
is different from chain(x)(g)(f)
it might even throw an error because the c
and a
are not matching types.
Now, we can see monads themselves as a series of intermediary data transformations, this is how the execution looks:
(function 1)(monad 1)(monad 2)(monad 3)(function 2)(monad 1)(monad 2)(...)...
The only difference is that monads carry the parameters that should be passed into the functions, and each monad conducts side effects based on the function parameters or even modify the function parameters themselves before it is being fed into the function.
As mentioned before, each monad takes in the params:value, func, *args, **kwargs
and also returns the same params value, func, *args, **kwargs
. The input and output types are the same. This means that:
m(x): a -> a
c(x): a -> a
chain(x)(m)(c): a -> a
Therefore, because the input and output types of the monads are the same (monoidic) they can be chained in any order and it doesn't matter. It is also because that chaining monads is a monadic thing, we can use a meta-monad to generate chained monads (to modify the behaviour of how the monads are chained).
- Just Monad
- Easy chaining of functions with parenthesis syntax
- Maybe Monad
- Railway style error handling
- Debug Monad
- Generate a debug trace to follow through pipeline execution
- Log Monad
- Log the function execution history throughout the pipelien
- Order Args Monad
- Enable ordering of arguments at each step of processing pipeline
- Composable Monads
- Combine multiple monads together using a monad-monad or via the
compose
function.
- Combine multiple monads together using a monad-monad or via the
- Custom Monads
- Create your own custom monad that fits your use case
To clone and run this application, you'll need Git installed on your computer. From your command line:
# Clone this repository
$ git clone https://github.com/yiheinchai/notmonad
# Start python shell
$ python
# Import the repo
$ from notmonad import *
NotMonad is an emailware. Meaning, if you liked using this app or it has helped you in any way, I'd like you send me an email at [email protected] about anything you'd want to say about this software. I'd really appreciate it!
Or
MIT
Personal Website yiheinchai.github.com ·
GitHub @yiheinchai ·
Twitter @chaiyihein