2.1 Tutorial ‐ Implementing Liquid Engine Methods

Creating a Python package using Liquid Engine

If you want to use the Liquid Engine in a Python package the best way to do is to use our Liquid Engine cookiecutter.

Following the instruction on the above repo yields a working Python package with an example Liquid Engine method ready to be edited.

A video tutorial is also available here.

Implementing Your Own Liquid Engine Class

Check out our video tutorial here if you want to follow along!

This tutorial teaches you how to implement your own methods using the Liquid Engine. For this tutorial you need to have a python environment with the LiquidEngine package installed. You can install it via pip:

pip install liquid_engine

Ensure that this environment has all the necessary packages for the methods you want to accelerate.

For the purpose of this tutorial, let's try to implement the two different non-local means denoising algorithms that are present in scikit-image.

pip install scikit-image

1. Create a new python file

Create a new python file for your custom Liquid Engine class.
Let's call it myliquidengineclass.py.

2. Create your new class inside the newly created file

Define a class that inherits from the LiquidEngine parent class, and define the necessary methods.

import numpy as np
from liquid_engine import LiquidEngine

from skimage.restoration import denoise_nl_means

class MyLiquidEngineClass(LiquidEngine):
    def __init__(self):
        self._designation = "MyLiquidEngineClass"
        super().__init__()

    def run(self, image: np.ndarray, patch_size: int, patch_distance: int, h:float, sigma:float, run_type=None):
        return self._run(image, patch_size=patch_size, patch_distance=patch_distance, h=h, sigma=sigma)

    def _run_ski_nlm_1(self, image, patch_size, patch_distance, h, sigma):
        return denoise_nl_means(image, patch_size=patch_size, patch_distance=patch_distance, h=h, sigma=sigma, fast_mode=True)

    def _run_ski_nlm_2(self, image, patch_size, patch_distance, h, sigma):
        return denoise_nl_means(image, patch_size=patch_size, patch_distance=patch_distance, h=h, sigma=sigma, fast_mode=False)

3. Benchmark it!

Import the your new class and test it! Follow along in this video.

import numpy as np
from myliquidengineclass import MyLiquidEngineClass

t_img = np.random.random((1,100,100)).astype(np.float32)
my_engine = MyLiquidEngineClass()
my_engine.benchmark(t_img,patch_size=5,patch_distance=5,h=0.1,sigma=1.0)

By default all LiquidEngine classes will store their benchmark files in the home folder of your user.

• In Windows, it should be similar to C:\Users\username\.liquid_engine
• In MacOS and Linux it should be similar to /Users/username/.liquid_engine

Optional features

• clear_benchmarks and testing are optional boolean arguments that you can pass to the parent LiquidEngine class __init__. They are both False by default, but you can define them in your child class so you can change them at will. If clear_benchmarks is True, previous benchmarks (if they exist) are deleted at class initialization time. If testingis True, everytime the benchmarkmethod is called all of the outputs are compared with each other and a warning is printed if any of them differ. Be careful. In order to compare, the outputs of every method have to be saved into memory.

(...)
class MyLiquidEngineClass(LiquidEngine):
    def __init__(self, clear_benchmarks=False, testing=False):
        self._designation = "MyLiquidEngineClass"
        super().__init__(clear_benchmarks=clear_benchmarks, testing=testing)
(...)

• The _compare_runs is the method used to compare the outputs of benchmarks method whenever testing is True. By default it assumes all methods defined in your LiquidEngine class return a 2D or 3D array and checks if they are different using Pearson's correlation. You can define custom comparison methods, as long as your method outputs are consistent. For example, if all methods in a LiquidEngine class return floats:

def _compare_runs(self, output_1, output_2):
    
    if abs(output_1-output_2) <= 1e-3:
        return True
    else:
        return False

• You can tag each function with custom identifiers by adding the tag prefixed by a @ in the docstring of the respective _run method. The tags serve as identifiers that the LiquidEngine agent can use to select the appropriate run type. For example, you can tag all methods that run on CPU with the tag "@cpu". If you run your LiquidEngine run class method with run_type='cpu', the agent will choose the fastest method with the tag 'cpu'. Take, for example the LiquidEngine class built in the tutorial. You can define the _run method as follows:

    def _run_ski_nlm_1(self, image, patch_size, patch_distance, h, sigma):
        """
        @one
        @skimagenlm
        """
        return denoise_nl_means(image, patch_size=patch_size, patch_distance=patch_distance, h=h, sigma=sigma, fast_mode=True)

    def _run_ski_nlm_2(self, image, patch_size, patch_distance, h, sigma):
        """
        @two
        @skimagenlm
        """
        return denoise_nl_means(image, patch_size=patch_size, patch_distance=patch_distance, h=h, sigma=sigma, fast_mode=False)

You can interface with these methods in several ways:

mle = MyLiquidEngineClass()

# These two are equivalent
mle.run(*args,**kwargs,run_type='ski_nlm_1')
mle.run(*args,**kwargs,run_type='one')

# These are also equivalent
mle.run(*args,**kwargs,run_type='ski_nlm_2')
mle.run(*args,**kwargs,run_type='two')

# This one will choose the fastest of the two
mle.run(*args,**kwargs,run_type='skimagenlm')