Skip to content
This repository has been archived by the owner on May 3, 2019. It is now read-only.

#168 loadmore.click() seems to be unstable #19

Open
wangchust opened this issue Oct 26, 2017 · 0 comments
Open

#168 loadmore.click() seems to be unstable #19

wangchust opened this issue Oct 26, 2017 · 0 comments

Comments

@wangchust
Copy link

wangchust commented Oct 26, 2017

When I crawle diffrent users, I only get 13 photos in many users wherever my '-n' is set to 200 and these user in fact have more than 13 photos.
So I think loadmore.click() in line 168 may not work well. Dose anyone suffer the same problem

some info:
posts: 148, number: 200
Scraping photo links...
Number of photo_links: 13
Saving...
Downloading 12 images to ta/linyaudavy
Quitting driver...
headless mode on
dir_prefix: ./data/, query: hecs_510, crawl_type: photos, number: 200, caption: False, authentica
tion: auth.json
posts: 273, number: 200
Scraping photo links...
Number of photo_links: 13
Saving...
Downloading 12 images to ta/hecs_510
Quitting driver...
headless mode on
dir_prefix: ./data/, query: da1sun, crawl_type: photos, number: 200, caption: False, authenticati
on: auth.json
posts: 2140, number: 200
Scraping photo links...
Number of photo_links: 13
Saving...
Downloading 12 images to ta/da1sun
Quitting driver...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant