v0.96g "Terrible No Matter How Much You Struggle"

Note: this is still experimental. The prerelease is to gather comments on the API.

This release introduces two new filters (Select to replace some common use cases of std.FrameEval and PropExpr to replace some common use cases of std.ModifyFrame), along with some enhancements and bugfixes to existing filters.

The most notable enhancement is the removal of 26 input clips limitation across all Expr filters (Expr, Select and PropEdit). Use srcN to access the N-th clip (For example, src0 is x, src25 is w.) There is no hardcoded limitation on how many inputs you can pass, but using too many might have VS performance implications. I have tested up to 255 input clips and all seem fine.

New Feature: Select filter

This release introduces a Select filter that aims to replace common uses of std.FrameEval when you just need to select one clip from multiple using the frame properties of another set of clips (e.g. mvsfunc.FilterIf)

Unlike std.FrameEval, the Select filter doesn't rely on Python, so it's unaffected by GIL and is fully parallelized. It also supports most of the operators supported by Expr (except those that access pixel values).

akarin.Select(clip[] clip_src, clip[] prop_src, string[] expr)

For each frame n, expr computes an index (idx) using frame properties for frame n of all clips in prop_src clips and then the n-th frame of clip_src[idx] will be returned.

As all the clips in prop_src are always requested for each frame, you should aim to minimize the cost for those clips. For each frame, only one clip from clip_src will be requested, so those clips should be the heavyweight in terms of computation cost.

If you specify multiple expressions for expr, then they will be used to select each planes of the output (the last specified expression will be repeated as necessary). But note that each output plane always comes from the corresponding plane of one of the clip_src input. That is, you can select different input clips for different planes, but you can't use the input's U plane as the output's V plane.

A simple example:

mvsfunc.FilterIf(src, flt, '_Combed', prop_clip)             # is equivalent to:
core.akarin.Select([src, flt], prop_clip, 'x._Combed 1 0 ?') # when x._Combed is set and True, then pick the 2nd clip (flt) otherwise use src

The expression is still evaluated in 32-bit floating point and the final result will be rounded to nearest integer and then clamped to be within [0, len(clip_src)-1] before used as an index into clip_src.

Three new operators are introduced to simplify the common case of ranking some property and then pick the best clip:

• argminN and argmaxN: consume N elements from the stack, find the index for the max/min value in those N values. e.g. 2 1 0 3 100 argmin5 should push 2 on the stack (as the minimum value 0 is the 3rd value [index starts from 0, as in Python]).
• argsortN: in case you want a rank other than the minimum or maximum, we have argsortN which will sort the top N values and return the indices. e.g. 9 5 1 2 8 4 0 3 6 7 argsort10 should put 0 4 9 8 1 5 7 3 2 6 (top is on the right) onto the stack. If you want the index for I-th smallest value, then dupI idx! drop10 idx@ should do. Please beware the difference between sortN and argsortN.

Notes:

1. Unlike Expr, the expressions are interpreted, not JIT'ed. This shouldn't pose a big issue as it's still 10x faster than Python's interpreter.

New Feature: PropExpr filter

akarin.PropExpr(clip[] clips, dict=lambda: dict(key=val))

PropExpr is a filter to programmatically compute numeric frame properties. Given a list of clips, it will return the first clip after modifying its frame properties as specified by the dict argument. The expressions have access to the frame property of all the clips.

dict is a Python lambda that returns a Python dict, where each key specifies the expression that evaluates to the new value for the frame property of the same name. The expression supports all operators supported by Select (i.e. no support for pixel access operators.)

For each val, there are three cases:

• an integer, or a float: the property key will be set to that value.
• an empty string: the property key will be removed.
• an expression string: the result of evaluating the expression specifies the value of property key. If the result is an integer, then it will be set as an integer frame property, otherwise, a floating point frame property. (Please beware that some special properties do need to be integer, e.g. _ChromaLocation, _ColorRange, etc.)

Additionally, in v0.96e or above, val could also be an array of numbers, and each output frame will use subsequent element of the array (wrap around to the first.)

Some examples:

• PropExpr(c, lambda: dict(_FrameNumber='N')): this set the _FrameNumber frame property to the current frame number.
• PropExpr(c, lambda: dict(A=1, B=2.1, C="x.Prop 2 *")): this set property A to constant 1, B to 2.1 and C to be the value returned by the expression x.Prop 2 *, which is two times the value of the existing Prop property.
• PropExpr(c, lambda: dict(ToBeDeleted='')): this deletes the frame property ToBeDeleted (no error if it does not exist.)
• PropExpr(c, lambda: dict(A='x.B', B='x.A')): this swaps the value of property A and B as all frame property updates are performed atomically.
• PropExpr(c, lambda: dict(C=[0, 1, 2])): output frame i will have frame property C set to i%3. A common idiom of generating strength clip for specified value given a list of values (values) in Python is strength = core.std.BlankClip(format=vs.GRAYS, width=1, height=1, length=len(values)).akarin.PropExpr(lambda:dict(_val=values)).akarin.Expr('x._val'), then the i-th frame of strength clip will have values[i] as its uniform pixel values.

Notes:

1. This peculiar form of specifying the properties is to workaround a limitation of the VS API. (I know api4 can use any, but unfortunately, this plugin has to support api3 as well. It's a trivial matter to write a nicer Python wrapper for this interface, so I don't think it matters much.)
2. Unlike Expr, the expressions are interpreted, not JIT'ed. This shouldn't pose a big issue as it's still 10x faster than Python's interpreter.

New Feature: Text filter (v0.96b)

The v0.96b release also introduces an enhanced text.Text filter.

New Feature: Tmpl filter (v0.96c)

If you want a Text filter on steroid ...

New Feature: PickFrames filter (v0.96g)

akarin.PickFrames(clip clip, int[] indices)

Return a new clip with frames picked from input clip from the indices array.

PickFrames(c, indices) is functionally equivalent to c[indices[0]] + c[indices[1]] + ... + c[indices[-1]], only more efficient as you only need to create one filter instance, especially when len(indices) is very large. As long as all indices are within range [0, c.num_frames-1], there is no limit on the the indices.

New Feature: ExprTest filter (v0.96g2)

....

Enhancements

• Issue #18: DLVFX now takes a model_dir parameter and it overrides the MODEL_DIR environment variable and the builtin hardcoded path C:\Program Files\NVIDIA Corporation\NVIDIA Video Effects\models.
• Issue #19: fixed compatibility with newer VFX version (Invalid strengh parameter error for OP_SUPERRES.)
• Issue #19: DLVFX OP_DENOISE is NOT working.
• Issue #17: All expr filters now allow arbitrary number of input clips. Use srcN to access the N-th input. e.g. src0 is x, and src25 is w.
• Issue #22: allow reading the first byte of bytes-typed frame properties (v0.96b).
• Issue #15: support for LLVM 14/15 added. (v0.96d)
• Expr now fully supports 16-bit floating point formats, even when the underlying architecture doesn't support the f16c instruction set extension (std.Expr only supports 16-bit floating point formats when f16c is present.) (v0.96d)
• Now you can put akarin.dlisr.dll as akarin\dlisr.dll to avoid getting warnings during VS plugin autoload. (v0.96d2)
• Support arrays in PropExpr (e.g. PropExpr(c, lambda: dict(C=[0, 1, 2])): output frame i will have frame property C set to i%3.) (v0.96e)
• Fixes #14, logical operators with mixed int/float operands in certain rare cases (v0.96f)
• Add lerp and polyval/polyeval operators (v0.96g3).

v0.95 "With You, Forever"

This release introduces bitwise and/or/xor/not (bitand, bitor, bitxor, bitnot) operators.

Internally they will operate on 32-bit integer values, and floating point values will be implicitly converted to integers, so it should be fine for <24-bit integer clips, but if you want to process 24-32b integer clips, you have to use enable opt=1 (and beware that in this mode overflows from operators like + /* will wrap around instead of saturate.)

Even if opt=0, lexpr will try to operate on integers as much as possible (e.g. x y bitxor 0x7f8b bitand will not involve any floating point to integer conversions if both clips are integer.)

Just note that they do not operate on the raw bit patterns of (floating point) values, so you can't use z x bitnot bitand z bitnot y bitand bitor to implement z x y ?.

v0.94 "The Invisible Face is There."

This release introduces absolute pixel access: now you can write arbitrary expression to compute the coordinate of the pixel you want to access, and the coordinate can even depend on pixel values!
Of course, such flexibility is not without performance cost, so use it carefully.

For example, the following pairs of expressions are semantically equivalent:

Static Relative Access	Dynamic Absolute Access
x[2,-1]:c	X 2 + Y 1 - x[]
x[-3,10]:c	X 3.1 - Y 10.05 + x[]

Additional details:

• x[] will automatically clamp the coordinates to be within their respective ranges, so if that's what you want, you can safely return negative coordinate or larger than width/height ones.
• The coordinate is internally computed in 32-bit floating point by default (unless you use opt=1 and stick with integer constants, X/Y/width etc.), so there might some performance penalty associated with that; If the computed coordinate is not integer, it will be rounded to the nearest integer.

A (truncated) transpose of frame can be obtained with Y X x[] (a real transpose is not possible as Expr requires the input and output to have the same dimension.)

Potential uses:

• Implement arbitrary LUT (unlike core.std.Lut and core.std.Lut2 you can use arbitrary expression to compute the LUT index, as long as you represent LUT as a GRAYS/YUVS/RGBS clip of the same dimension as the other inputs). Not as flexible as dedicated 3D-LUT implementations, but it can get the job done.
• (Your ideas here.)

v0.92 "YuruYuri Summer Vacation!+ +1"

This release introduces the CAMBI banding detector. Please see the NetFlix blog post CAMBI, a banding artifact detector for an introduction to the algorithm.

Unlike VapourSynth-VMAF, which uses libvmaf C APIs, this implementation is online (i.e. you don't have to batch process the whole clip first), and it exposes internal c-score maps (which can be used as a banding mask.) I should make it clear that this is not an independent implementation, and it's still based on code from libvmaf.

Please refer to https://github.com/AkarinVS/vapoursynth-plugin/wiki/CAMBI for its docs.

An example:

src = core.lsmas.LWLibavSource(filename)  # 8-bit or 10-bit YUV/Gray video supported
cambi = src.akarin.Cambi(scores=True)     # scores=True makes it output per-scale c-score maps in addition to the per-frame CAMBI score
cambi.text.FrameProps("CAMBI").set_output(10)
for i in range(5):
    scale = cambi.std.PropToClip('CAMBI_SCALE%d' % i)
    scale.set_output(11+i)

If you're unsure which scale to use, use cambi.std.PropToClip("CAMBI_SCALE1").fmtc.resample(scale=2) to get a rough banding mask for the input.

In case you do want to batch process the whole clip and find the top CAMBI score frames, you can do this:

from vapoursynch import core
log_file = 'cambi.log'
cambi = core.lsmas.LWLibavSource(filename).akarin.Cambi()
def append_cambi(n, f): 
    with open(log_file, 'a') as fo:
        fo.write('%d %.4f\n' % (n, f.props['CAMBI']))
    return cambi
core.std.FrameEval(cambi, append_cambi, cambi).set_output(0)

And then run the script with vspipe -p cambi.vpy -a filename=INPUT.m2ts ..
Then you can sort the output file cambi.log based on the 2nd column (i.e. sort -nr -k2,2 cambi.log if you are using Unix sort.) The first column will be the corresponding frame number.

NOTES

Due to a build error, v0.92b doesn't contain DLVFX or DLISR. v0.92c fixed the issue.
v0.92d loads DLVFX dll only when actually used, and this should fix some DLL initialization issues.
v0.92e is a test release that enables JIT caching on windows (this mainly benefits vs-preview users as it can reduce expr-heavy script reload time.)
v0.92f is a test release that incorporates a fix for #3 (so DLISR can co-exist with other cuda filters in the same script now) and actually add ** alias for pow in lexpr.

v0.90 "The Akari Who Leapt Through Time"

Changes:

1. add dropN operator that drops the top N items from the stack (drop means drop1). Usually used together with sortN.
2. add sortN operator that sorts the top N items from the stack. After the operation, the smallest item is on the top. Please see https://github.com/AkarinVS/vapoursynth-plugin/wiki/Expr#using-sortn-to-implement-rank-order-filters for some examples and suggested usage.

Installation notes:

1. For DLISR, need to download https://github.com/AkarinVS/vapoursynth-plugin/releases/download/v0.70/akarin.dlisr.v1b.7z and place akarin.dlisr.dll alone side akarin.dll (this step is only necessary if you want to use DLISR)
2. For DLVFX, please see https://github.com/AkarinVS/vapoursynth-plugin/releases/tag/v0.70 on how to install required components.

v0.75b "Lazy Japanese Summer"

This version (v0.75b) can be installed by vsrepo.

Breaking change: it is discovered the v0.75b's default opt=1 (which uses int32 as intermediate type for as long as possible) might cause some issues when integer multiplication overflows int32 and wraps around, so v0.75b2 (file akarin-release-lexpr-amd64-v0.75b2-opt-0.7z) changes the default opt argument to 0 to disable automatic integer optimization. It's recommended that existing v0.75b users migrate to this patch release.

Changes:

1. @misakikasumi improved the vfx filters in the following aspects:

• performance can be improved significantly by setting appropriate num_streams>1
• support for RGB24 inputs, in addition to RGBS.
• support setting output_depth to select between 32 (RGBS) and 8 (RGB24) outputs; default to use the same format as the input clip.

2. lexpr added support for clip and clamp operators: x 16 235 clip and x 16 235 clamp are both equivalent to x 16 max 235 min.
3. It's determined that DLISR does not play well with other CUDA filters in the same script, so please make sure you only use cpu filters if you use DLISR in a script (shouldn't matter much as DLISR can saturate the highest end GPU easily so using other GPU filters will probably just slow things down)
4. (version v0.75b2) changes default opt parameter value to 0 forakarin.Expr as the default int32 optimization is known to cause issues for some of expressions.
5. (version v0.75b3) adds a unbounded cache for the compiled result, so script should reload much faster with Python based previewers. (Not ready yet, will crash at exit.)

Installation notes:

1. For DLISR, need to download https://github.com/AkarinVS/vapoursynth-plugin/releases/download/v0.70/akarin.dlisr.v1b.7z and place akarin.dlisr.dll alone side akarin.dll (this step is only necessary if you want to use DLISR)
2. For DLVFX, please see https://github.com/AkarinVS/vapoursynth-plugin/releases/tag/v0.70 on how to install required components.

v0.70 "Chocolate and Tears and Girls and Girls and Isobe Fries"

This release introduces DLISR and DLVFX filters.

• DLISR: NVidia's deep learning based image super resolution filter.
• DLVFX: NVidia's deep learning based video effect filters.
  • op=0: artefact reduction
  • op=1: super resolution
  • op=2: denoising (not working yet)

DLISR requires akarin.dlisr.dll to be placed along side akarin.dll, as the former is too large and does not change often, I will only release it when it's changed.

DLVFX requires installing appropriate Video Effects library (v0.6 beta) (it's too large and GPU dependent to be bundled together with the plugin.)

• Ampere GPUs (RTX 30 series) use https://international.download.nvidia.com/Windows/broadcast/sdk/v0.6/nvidia-video-effects-sdk-installer-ampere.exe
• Turing GPUs (RTX 20 series) use https://international.download.nvidia.com/Windows/broadcast/sdk/v0.6/nvidia-video-effects-sdk-installer-turing.exe

v0.61b "Warm Slumber Party With Everyone"

v0.61b "Warm Slumber Party With Everyone"

lexpr only release. Introduces:

• 32-bit integer sample type support;
• hexadecimal/octal constants;
• mirror boundary condition for relative pixel access, e.g.x[-1,3]:m; can also use boundary=1 parameter to set all boundary conditions to mirrored.

(If you wonder why supporting 32-bit integer formats is necessary when even builtin std.resize does not support this format:
it's to build a fast RGB24 to COMPATBGR32 converter for AkarinVS/vapoursynth-preview@d2b2d27)

v0.51 "The Great Summer Harvest", again

This legacy-only release introduces:

• Version() for better version and feature tracking.

And also fixed one issue found during testing.

1. fixed constant folding of various new operators (sin, cos, round, trunc, %)

While we're at it, also added the pi operator as it's trivial.

Again, please use the lexpr version if possible and the legacy version won't receive more feature updates.
lexpr version latest release is v0.60.

v0.60 "When Akari and the Cicadas Cry"

This lexpr-only release introduces relative pixel access feature in the LLVM (lexpr) implementation.

e.g. x[-1,1] access the pixel from clip x 1 column to the left and 1 row down to the current pixel.

• Only constant offsets are allowed.
• Out of bound accesses are clamped to the edge.

v0.50 introduces:

• Version() for better version and feature tracking.
• width and height operators to access frame width and height (for chroma planes, this also includes subsampling factors.)

You can use this to implement arbitrary convolution kernels (i.e. non-regular shapes), and my benchmark indicated that 3x3 convolution implemented this way is as fast as std.Convolution.

As there is no changes to the legacy implementation, please continue to use last release v0.51 for that.