Followers crawling

[InesArous]

Hi guys, I was able to crawl followers on Instagram after modifying some lines in the intagramcrawler.py. Now, I am facing a problem when the number of followers is over 1000: The crawler scrolls down the followers page but the page freezes in a loading phase so after some time the crawler simply quits the query.
Have you faced this issue before? If so how to fix it?

[AbeerArora123]

Could I ask what you altered in the code to get it working?

[markusMM]

Do you mean, you can crawl for followers as constraint?
Actually, if this script would allow pure SQL or Hadoop, it would be much more useful.

e.g. -q "select top 100000 from instagram.photos order by num_follows instargram.follower"

Or something like that.

Anyways maybe that's not how a crawler works, but it would be very cool B-)