Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast Insert #151

Open
Kencho5 opened this issue Nov 30, 2021 · 2 comments
Open

Fast Insert #151

Kencho5 opened this issue Nov 30, 2021 · 2 comments

Comments

@Kencho5
Copy link

Kencho5 commented Nov 30, 2021

as you mentioned in the readme, it states that it can easily scale up to billions of images, but i'm having a hard time inserting a lot of images in elasticsearch. i've tried to do it with multiprocessing, threading and even async too, but given that i have around 160 million images it's still slow. any suggestions on how i can insert those images a bit faster?

@trashmit
Copy link

did you come up with a solution?

@Kencho5
Copy link
Author

Kencho5 commented Apr 13, 2022

yes. made bulk insert script, but don't have it anymore. you have to split functions and use multiprocessing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants