You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have not really spent too much time optimizing our code, but it seems like there may be opportunities to gain performance. For instance, a small change in PR #417 resulted in 50% speedup in one of the PTransforms (~10% overall speedup; or more depending on the size of the data).
However, rather than trying 'random' things, it's better to run a CPU profiler on our code to determine the places where it makes sense to optimize (e.g. I have found that sanitizing the BQ fields also takes a nontrivial amount of time). Beam has some flags to setup profiling (see thread).
We can also consider using Cython once we know the areas to optimize.
The text was updated successfully, but these errors were encountered:
We have not really spent too much time optimizing our code, but it seems like there may be opportunities to gain performance. For instance, a small change in PR #417 resulted in 50% speedup in one of the PTransforms (~10% overall speedup; or more depending on the size of the data).
However, rather than trying 'random' things, it's better to run a CPU profiler on our code to determine the places where it makes sense to optimize (e.g. I have found that sanitizing the BQ fields also takes a nontrivial amount of time). Beam has some flags to setup profiling (see thread).
We can also consider using Cython once we know the areas to optimize.
The text was updated successfully, but these errors were encountered: