The NuttX poll system call is slow

[pavel-kirienko]

The poll() system call takes a very long time to execute. Since it is extensively used in many high frequency data processing loops, this issue has a significant impact on overall performance of the system.

See this for more context: #6829 (comment)

[NaterGator]

@pavel-kirienko are we sure PMP™ is properly accounting for NuttX context switches? It seems logical some of the time spent in poll would be attributable to context switching and running another task.

[pavel-kirienko]

@NaterGator Yes, we can be reasonably sure that PMP™ produces sufficiently reliable data. Context switch from/into poll() is still a runtime overhead which also should be considered during optimization.

[davids5]

@NaterGator Yes, we can be reasonably sure that PMP™ produces sufficiently reliable data. Context switch from/into poll() is still a runtime overhead which also should be considered during optimization.

I have finally got to instrument this with HW, The number in the PMP™ do not match the values I am getting

A1-A2 From task ublock to the pol call is < 9 uS
B1-B2 From poll to task block is < 12uS
D1-D2 From task ublock to the pol call end is < 24 uS

I am not sure what we are seeing from the PMP™ the the rest of this it thread on the waitlist.

[pavel-kirienko]

The timing estimates you provided cannot be directly compared against the data provided by PMP™.

PMP™ assesses the contribution of every procedure/function/method to the total CPU load. The contribution per procedure is a function of the amount of time the procedure takes to execute multiplied by the number of its invocations. For example, consider a function X that takes forever to execute and invoked rarely, and a function Y that completes very quickly but is invoked frequently. From the standpoint of any sampling profiler (such as PMP™), the contribution of both could be equal.

My point is that the estimated duration of the poll() call (<36 us total if I'm reading this right) does not yet say much about its contribution to the total system load. We need to take into account how frequently the function is invoked.

[pavel-kirienko]

According to the data from the linked thread, poll() alone accounts for about 5% of the total system load. Given the 36 us estimate above we can see that poll() needs to be invoked about (1e6/36)*0.05=1389 times per second anywhere in the system in order to account for about 5% of the system load. This looks plausible to me.

[davids5]

@pavel-kirienko

I think it is good to know, as you pointed out, that when looking at the the PMP™ output there is the 'frequency of use of' and the 'amount of time' that has to be considered.

Would you agree in this case nuttx poll is not slow, but just used a lot?

[pavel-kirienko]

36 microseconds (6480 cycles) is slow. :)

Given the frequent use of this function, I would take a closer look at its implementation and see what can be sped up.

[PX4BuildBot]

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 30 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)