Dictionary access issue

[habitoti]

Trying to run the UI, two layers. After finishing the move path and closing the window, before proceeding to layer two, I run into this error:

You are currently working on layer #1 of 2
Creating frames for this layer...: ETR=0:00:00.068577	ETA=0:00:00.373363	T=0:00:00.304786	Progress: 40/49Traceback (most recent call last):
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_gui.py", line 301, in <module>
    animation_output = animate_polygon(image, polygon, *animation)
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_gui.py", line 219, in animate_polygon
    return gather_vars("frames transformed_polygons")
  File "/Users/hansi/.pyenv/versions/3.13.1/lib/python3.13/site-packages/rp/r.py", line 7768, in gather_vars
    local_vars =merged_dicts(frame_locals,precedence='first')
  File "/Users/hansi/.pyenv/versions/3.13.1/lib/python3.13/site-packages/rp/r.py", line 14203, in merged_dicts
    result = first_dict if mutate else type(first_dict)()
                                       ~~~~~~~~~~~~~~~~^^
TypeError: FrameLocalsProxy expected 1 argument, got 0

Any clue? Running on Python 3.13.1, macOS (latest) on an M3 MBP.

[RyannDaGreat]

Huh, that's weird...I'm not sure why this error pops up. I designed it on my M1 Macbook so it shouldn't be that different, though I was using python 3.10.6. I'm going to update this repo in a minute...please let me know if it fixes the problem

[RyannDaGreat]

Can you please try again with the new update? @habitoti

[habitoti]

Different error now:

You are currently working on layer #1 of 2
Creating frames for this layer...: ETR=0:00:00.042738	ETA=0:00:00.349030	T=0:00:00.306292	Progress: 43/49Traceback (most recent call last):
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_gui.py", line 303, in <module>
    animation_output = animate_polygon(image, polygon, *animation)
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_gui.py", line 221, in animate_polygon
    return EasyDict(frames=frames,transformed_polygons=translated_polygons)
                                                       ^^^^^^^^^^^^^^^^^^^
NameError: name 'translated_polygons' is not defined

[RyannDaGreat]

Hi! Can you please try again? I updated it with another fix

[habitoti]

I am getting now further to the noise MPG generation.

Had to manually install torch, einops and ffmpeg. Maybe you should add them to the list of prerequisites so that they are auto installed. art and ffmpeg-python were interactively asked for and installed on "yes" during first full run.

Now it's stuck in the process of creating the noise MPG:

Warping noise for layer #2 of 2
Regaussianizing: COMPLETED IN 0:00:21.8520570:00Progress: 49/49 T=0:00:21.398962	Progress: 48/49
Traceback (most recent call last):
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_gui.py", line 402, in <module>
    polygons=as_numpy_array(layer_polygons)
  File "/Users/hansi/.pyenv/versions/3.13.1/lib/python3.13/site-packages/rp/r.py", line 27755, in as_numpy_array
    if '16' in str(x.dtype):x=x.float() #Numpy screws up with floa16
                   ^^^^^^^
AttributeError: 'list' object has no attribute 'dtype'

[dissorial]

I also had to additionally install einops, so maybe put that in the reqs

[habitoti]

So I switched to Python 3.10.6 (the version you're using) and except having to separately install einops, the preparation steps run smooth now. Seems like Python is not very backwards compatible... :-(

But even with your version, I run into another issue in the final rendering:

Traceback (most recent call last):
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/model_loading_utils.py", line 143, in load_state_dict
    file_extension = os.path.basename(checkpoint_file).split(".")[-1]
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/posixpath.py", line 142, in basename
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_inference.py", line 486, in <module>
    fire.Fire(main)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_inference.py", line 455, in main
    pipe = get_pipe(model_name, device, low_vram=low_vram)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/rp/r.py", line 23745, in memoized_function
    cache[key] = function(*args, **kwargs)
  File "/Users/hansi/projects/GenAI-Netflix-Eyeline-Research/cut_and_drag_inference.py", line 73, in get_pipe
    transformer = CogVideoXTransformer3DModel.from_pretrained(hub_model_id, subfolder="transformer", torch_dtype=torch.bfloat16)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 986, in from_pretrained
    state_dict = load_state_dict(model_file, variant=variant)
  File "/Users/hansi/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/model_loading_utils.py", line 157, in load_state_dict
    with open(checkpoint_file) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType

[RyannDaGreat]

I'm glad! I'll add Python3.10 to the reqs. It looks like the new error is related to diffusion inference, I take it you were able to get the cut-and-drag interface running?
Just to be clear, all you had to do was install einops and use Python3.10? And you're using Linux, Mac or Windows?

[RyannDaGreat]

As for the diffusion inference code...I'm not sure what's going on, maybe try pulling the newest changes and try again? The models have been moved to a new huggingface repo (the new changes in the code reflect that)