How to improve performance holistic graph on windows10?

[sewonist]

Hi All,

I've made app what is translator for deaf person with mediapipe. Origin was made with python. And I migration it to Unity for more features. I tested holistic graph with plugin. It works however performance is worst than python. I don't know why delay it. like below image.

I feel delay time almost over 1sec. So I changed code PushInput slowly.

if (delta > 1f / 20f)
{
  graph.PushInput(nextFrame).AssertOk();  
  graph.RenderOutput(webCamScreenController, nextFrame);
  delta = 0;
}

It is better than before although after several seconds. It was made delay, too. I don't need high speed. I just want to exactly same output data and video. Anyone has other idea?

Thanks,

[Joel-Ee]

How could you compile plugin in windows 10 successfully? We also try as tutorials but failed. Here is our issue: #241
maybe we can communicate more about unity performance

[sewonist]

@Joel-Ee I just follow tutorial. (https://github.com/homuler/MediaPipeUnityPlugin/#windows) I don't have any problems.

[sewonist]

I found one of solutions. I think it's not really good idea but it works for me. I don't know why it was starting slow after lost landmarks. slower and slower. My solution is reset poller every 1sec. I insert below code to OutputStreamPoller.cs

    public void Reset()
    {
      UnsafeNativeMethods.mp_OutputStreamPoller__Reset(mpPtr);
    }

And I have other question. I really wonder why value of isPoseDetectionPresent is always false? Is it normal?

[jsaon-L]

I also delay .i use 0.5.3 is ok,but 0.6.2 is faster than 0.5.3

[sewonist]

I also delay .i use 0.5.3 is ok,but 0.6.2 is faster than 0.5.3

What? It happen depend on release? Thanks you for great information. I'm going to test other release.

[jsaon-L]

google-ai-edge/mediapipe#2379

[ROBYER1]

Thanks for linking my issue with Google here, finding the same latency after a short time running the Pose detection tracking as well as Holistic and not just Holistic. Has anyone found any new fixes or optimizations for this?

For now, the hacky way @sewonist suggested worked for our demos

[ROBYER1]

I found one of solutions. I think it's not really good idea but it works for me. I don't know why it was starting slow after lost landmarks. slower and slower. My solution is reset poller every 1sec. I insert below code to OutputStreamPoller.cs

    public void Reset()
    {
      UnsafeNativeMethods.mp_OutputStreamPoller__Reset(mpPtr);
    }

And I have other question. I really wonder why value of isPoseDetectionPresent is always false? Is it normal?

Using this on device causes occasional crashing on IOS/Android :/

It also led me to discover a similar crash when trying calling stop/start graph programmatically
#250

[ROBYER1]

I just rolled back to Mediapipe 0.8.4 here and rebuilt as it was the most stable. 24576ce

Seems in versions after where they added 3d pose points, a lot of latency was added

[homuler]

Fixed by #296