ValueError: numpy.ndarray size changed

[edogab33]

I'm running Top2Vec in Databricks and this error appears:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Code:

from top2vec import Top2Vec
model = Top2Vec(messages_lst, embedding_model='distiluse-base-multilingual-cased')

Environment:
python3.8.8
numpy==1.20.0 (also 1.21.x doesn't work)
scipy==1.5.2
hdbscan==0.8.27

Seems to be related to hdbscan but doing !pip install hdbscan --no-cache-dir --no-binary :all: --no-build-isolation (also with numpy), as suggested in other issues, doesn't solve the problem.

[jcarletgo]

Same issue seems to be coming from gensim

  File "/opt/conda/lib/python3.9/site-packages/gensim/matutils.py", line 1104, in <module>
    from gensim._matutils import logsumexp, mean_absolute_difference, dirichlet_expectation
  File ".eggs/Cython-0.29.14-py2.7-linux-x86_64.egg/Cython/Includes/numpy/__init__.pxd", line 242, in init gensim._matutils
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

But really the issue seems to be tensorflow 2.5.0 requires numpy~=1.19.2 but in top2vec numpy>=1.20.0
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/release/requirements_common.txt

[gxavier38]

I'm having the same issue running in a Kaggle notebook. I found these relevant issues, but I've tried the suggestions and they didn't work for me. Seems like an old version of top2vec had this issue too.

#125
scikit-learn-contrib/hdbscan#457

[edogab33]

Just an update: I was running Top2Vec in Databricks (which was giving me the same problem with BERTopic). Now I migrated everything to Google Colab and it works fine. Probably there is one package which is giving some troubles.

[gxavier38]

@edogab33 thanks! Colab also worked for me. That's strange

[ddangelov]

This sounds like an environment issue.

[edogab33]

This sounds like an environment issue.

Yes, it is. The problem is that I checked each external library version and they were all correct.