Unbound local variable bkey_hash (from cachelib) when providing State of Store in callback that has ServersideOutput of same Store

[matthiasschuurmans]

Hi, thanks a lot for this package, it's helping me a lot with making dashboards!

Unfortunately, I'm getting UnboundLocalError: local variable 'bkey_hash' referenced before assignment from cachelib when I add a store as both ServersideOutput and State for a callback. I want to do this, so that I can first check if the store contains the data I need, before doing an expensive query to get that data and update the store. Somehow, this was working not so long ago, but I can't really track down exactly what changed, so I'm hoping you can help me.

I made a minimum reproducible example based on https://www.dash-extensions.com/transforms/serverside-output-transform:

import time
import plotly.express as px
from dash_extensions.enrich import DashProxy, Output, Input, State, ServersideOutput, html, dcc, \
    ServersideOutputTransform

app = DashProxy(transforms=[ServersideOutputTransform()])
app.layout = html.Div(
    [
        html.Button("Query data", id="btn"),
        dcc.Dropdown(id="dd"),
        dcc.Graph(id="graph"),
        dcc.Loading(dcc.Store(id="store"), fullscreen=True, type="dot"),
    ]
)

# NOTE THE `State("store", "data")` ADDITION
@app.callback(ServersideOutput("store", "data"), Input("btn", "n_clicks"), State("store", "data"), prevent_initial_call=True)
def query_data(n_clicks, data):
    time.sleep(3)  # emulate slow database operation
    return px.data.gapminder()  # no JSON serialization here

@app.callback(Output("dd", "options"),  Output("dd", "value"), Input("store", "data"), prevent_initial_call=True)
def update_dd(df):
    options = [{"label": column, "value": column} for column in df["year"]]   # no JSON de-serialization here
    return options, options[0]['value']

@app.callback(Output("graph", "figure"), [Input("dd", "value"), State("store", "data")], prevent_initial_call=True)
def update_graph(value, df):
    df = df.query("year == {}".format(value))  # no JSON de-serialization here
    return px.sunburst(df, path=["continent", "country"], values="pop", color="lifeExp", hover_data=["iso_alpha"])

if __name__ == "__main__":
    app.run_server()

The error comes from cachelib, https://github.com/pallets-eco/cachelib/blob/main/src/cachelib/file.py#L186 .For some reason, key is None, and so it does not create bkey_hash and produces the error when returning os.path.join(self._path, bkey_hash)

Versions:
dash: 2.4.1
dash-extensions: 0.1.3
cachelib: 0.7.0

Steps to reproduce:

1. Create and activate venv
2. pip install dash-extensions and pip install pandas (apparently pandas is not a requirement of dash-extensions, even though it raises ImportError: Plotly express requires pandas to be installed.)
3. Run the above code
4. Click on the button

Thank you for your time and effort!

[emilhe]

I just tried to run your example. On my systems (I tried on my OSX laptop, and on my Windows WSL2 system), it runs without any errors. What more is, I don't even have cachelib installed. I don't believe dash-extensions calls cachelib?

[matthiasschuurmans]

Thanks for your reply @emilhe . Here is my pip install dash-extensions log:

Installing collected packages: zipp, MarkupSafe, colorama, Werkzeug, Jinja2, itsdangerous, importlib-metadata, click, tenacity, Flask, brotli, six, plotly, flask-compress, editorconfig, dash-table, dash-html-components, dash-core-components, *cachelib*, more-itertools, jsbeautifier, Flask-Caching, dash, dash-extensions
Successfully installed Flask-2.1.2 Flask-Caching-1.11.1 Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.1.2 brotli-1.0.9 *cachelib-0.7.0* click-8.1.3 colorama-0.4.4 dash-2.4.1 dash-core-components-2.0.0 dash-extensions-0.1.3 dash-html-components-2.0.0 dash-table-5.0.0 editorconfig-0.12.3 flask-compress-1.12 importlib-metadata-4.11.4 itsdangerous-2.1.2 jsbeautifier-1.14.3 more-itertools-8.13.0 plotly-5.8.0 six-1.16.0 tenacity-8.0.1 zipp-3.8.0

dash-extensions uses flask-caching, which in turn uses cachelib according to https://github.com/pallets-eco/flask-caching/blob/master/src/flask_caching/backends/filesystemcache.py#L17 . See also pallets-eco/flask-caching#308 for the PR.

I don't know why your system does not use cachelib, it seems like it should no? Maybe some caching or outdated requirement versions somewhere? The PR that introduced cachelib in flask-caching is from May 28th so a while ago already, and that same day 1.11.1 was released on PyPi.

My older environment (as opposed to a newly created venv) also works and also does not use cachelib, and there it uses flask-caching version 1.10.1 as opposed to the latest 1.11.1. I can't find the exact version-bump in the code, but I'm guessing 1.11.1 introduced cachelib. It also seems like that introduced some other bugs, see https://github.com/pallets-eco/flask-caching/issues . Maybe it's a simple temporary fix to pin flask-caching version requirement to 1.10.1? I just checked in my own environment that that indeed solves the issue at least.

[emilhe]

Ah yes, now I am able to reproduce the issue. I have defined the dependency as,

Flask-Caching = "^1.10.1"

My local .lock files where all pinned at 1.10.1, similar to the repo,

https://github.com/thedirtyfew/dash-extensions/blob/a7c167818b036dd8b10befc89abbfdeb675892e2/poetry.lock#L320

so I didn't see any issues. Manually upgrading to 1.11.1, I also get the error. It looks like a bug in cachelib, so I would say the solution is to downgrade. I'll try to pin the dependency from my side.

[emilhe]

The issue should be resolved in version 0.1.4. Please let me know, if you still see any issue(s).

[kennethfungch]

Hi @emilhe
I am still experiencing the exact same problem despite upgrading to 0.1.4 with flask-caching updated to 2.0.0.

[emilhe]

@kennethfungch did you clear the cache folder(s)? The fix is not backwards compatible, so if you have old cache files laying around, they might cause issues.

[kennethfungch]

@emilhe yes I have removed the cache folder (file_system_store) and it doesn't work unfortunately.

[prokie]

I tried @matthiasschuurmans example and have the same issue with dash_extensions==0.1.4 and the prerelease 0.1.5rc2. I also tried deleting file_system_store before starting the app.

[emilhe]

Ah, yes, I can see that this error was do to a composite bug, of which only the first part was fixed in the 0.1.4 release. The latter part should be fixed in the 0.1.5 release. Please let me know, if you still see any issue(s).

[VictorCallejas]

dash-extensions

This worked for me, just updated to dash-extensions 0.1.5 , thanks!