HAR file partially generated

[gianlucaperna]

Hi everyone,

I already described the situation in this thread:

sitespeedio/browsertime#1664 (comment)

but, anyway, to summarize the problem, what I want to do is to visit a page for 9 minutes, and, on this page, there is a video provided using Nginx server and dash player.

Everything seems to work well in every scenario except for what concerns http3 using tc (traffic controller - Linux) with 10Mbit Up/Down constraints.
In particular, I can see on the nginx server-side that browsertime executes correctly 150 (almost) requests but on the har file I'm able to see only the first 60.
I don't know why browsertime seems to stop registering requests after the first 60.
Also looking at the pcap (that is not attached cause dimension) the experiment seems fine, the problem is only in missing data in the .har file

To conclude, I attach the result obtained in one of these cases, in the zip folder there are both .har and perfLogs of chrome.

I am really grateful if anyone can help me.
Really thanks,
Gianluca

http3.zip

[soulgalore]

Hi @GianlucaPoliTo when I drag and drop the HAR file (har_http3_bw_10_esp_1.har) into https://compare.sitespeed.io I see 141 request?

[gianlucaperna]

Hi @soulgalore,

thanks again for reply.
Yes is correct, if you open the file you are able to see 141 request. The problem is that (I attach a picture to indicate what I mean) the URL request, is a frame chunk number 60, while on the nginx side I see that browsertime does request until (almost) 150.
This is confirmed also with the pcap registered from browsertime, if I look at last http request, I can see bbb_...151.
I didn't understand why the har is cut practically.

Thanks a lot for your help and your time,
Gianluca

[soulgalore]

Ok. The trace log didn't hold any more events. When do you end your tests https://www.sitespeed.io/documentation/sitespeed.io/browsers/#choose-when-to-end-your-test ? I'm thinking like this: the tests ends, but the video continues to play when we run the JavaScript tests to collect metrics = you see them in the tcpdump? But that do not totally match because the dumb should end when the test ends.

[gianlucaperna]

Thanks @soulgalore the problem is that the experiment is set in order to finish after about 9 minutes, instead, looking at the frames it seems that finish after half of the time.

Could be a problem due to an internal crash or something else?
I'm currently trying the same experiment on another machine, here I'm able to see 82 frames, but we are still far from the result.
I just don't understand why this happens only in this case, when I set 1Mbit of bandwidth everything works well.
Also in the scenario with packet loss or round trip time, there are no problems.

Thanks a lot

[gianlucaperna]

Sorry for the double message, but, at this point, I'm not sure that the right parameter is "--pageCompleteCheckStartWait 540000" to make the experiment long 9 minutes.
Can you confirm it for me?

Thanks again!

[soulgalore]

Yep you can use that. That means that the first check will not run for a long time. Are there switches to Firefox so you can run the same test (=see if its the browser or something else)?

[gianlucaperna]

Using others browser is a bit critical, but I have a small news about chrome.
I tryed to do a set of experiment by hand, saving the har file after 9 minutes, and, only in 2 cases on 10 I have a complete set of information inside the file.
Now I'm thinking that the problem could be chrome that take too much time to write the informations on the file, have you ever encountred this problem?

Is there a flag in which I can terminate the experiment but to leaves to chrome the time to save everything correctly?
If you are courios I can attach you a zip with the 10 har , checking by eye I saw that every experiment has reached almost 135 frames (that are not present in .har).

Thanks a lot!

[soulgalore]

Hi @GianlucaPoliTo I don't think we have a flag for that, let me add that. Can you run your test from a branch? Then I can add it there and you can test.

[gianlucaperna]

Hi @soulgalore sorry for the delay in my answer, tomorrow in the morning I cloned the new branch and I tested your new feature in-depth.

What I understood is that the problem is not related to the delay in the chrome termination, but rather to the network emulator software that I'm using, in particular with the parameter BURST in tc netem controller.

Anyway, I think that the new features could be useful for someone in the future so I hope that you have not wasted your time trying to implement them.

Many many thanks for your valuable support and time!