Move aKNN limits enforcement into the default Codec's KnnVectorsFormat implementation

[mikemccand]

Description

[Spinoff from #12306]

There have been many discussions and polls about what to do about the existing (weakly enforced) limit of aKNN vector dimensionality in Lucene.

This issue represents Option 3 in @alessandrobenedetti's recent poll thread.

Since it is this Codec component (currently Lucene95HnswVectorsFormat) that is implementing the HNSW approach for approximate KNN, it makes sense that it should be the one to enforce any limits (dimensionality, max connections, beam width, etc.). In fact, it already seems to enforce some limits -- I see MAXIMUM_BEAM_WIDTH = 3200 and MAXIMUM_MAX_CON = 512. Once we do this, users can still fork their own Codec to change limits, or implement a different aKNN algorithm, etc., and it will be clear that they are no longer using Lucene's default Codec so index format backwards compatibility is no longer ensured.

Version and environment details

No response

[bruno-roustant]

In the same work, or in a separate work, we could create the extension of the HNSW implementation in the codecs package to provide it to users, so they don't have to have their own fork for that. This alternative codec would just support more dimensions to play with, without backwards compatibility constraints.