ensure to retrieve all pages when using #files with given block

[jgigault]

we should take "page_next_token" into account while listing pages through #files
this reproduces the behaviour we have with GoogleXML files collection

I guess that this should fix known issue #42

Example of usage:

connection =
  ::Fog::Storage.new(
    provider:               'Google',
    google_project:        '...',
    google_json_key_string: '...'
  )

bucket_name = '...'

directory = connection.directories.get(bucket_name)

files = directory.files.each_with_object([]) { |file, results| results << file }

files.size # can now be greater than 1000

[geemus]

At a glance at least this seems reasonable enough, but I don't have a test account and haven't used this directly in a while.

@Temikus any chance you could take a peek and review? Thanks!

[Temikus]

@geemus yeah, of course! Just a busy week so I’ll take a look on a Saturday.

[geemus]

@Temikus no rush, just happened to see it and wanted to make sure it was on your radar. Thanks!

[geemus]

Thanks!

[jgigault-pa]

Thank you!