Possible Performance Optimisations

[Zentrik]

I'm just creating this as a reference.

There are various problems with this package from a performance standpoint, largely relating to dynamic dispatches and heap allocations. I investigated this a while ago so I might be wrong on some counts but the main problems are

• The default parameters is a dictionary with values being Any, this introduces type instabilities all over the place;
• Unnecessary allocations, e.g. see Zentrik@34d9f82 for two specific examples. There are various other places that would benefit from pre-allocating memory.

I made some improvements on my fork, which eliminate almost all dynamic dispatches and heap allocations, in these two commits Zentrik@34d9f82, Zentrik@f842769. Note my fork is specialised specifically for my case, i.e. it hardcodes the optimisation algorithm that is used and forces Float32's in various places and might use a fork of various packages that speeds them up as well. However, I don't see any reason that these optimisations require these specialisations, it was just easier to do that.