Verify that the "Total collected files" number from our collections table reflects the number of CDX entries with status 200

[VascoRatoFCCN]

On the collections table we have the following entry for total collected files for the collection IA:

However, counting the number of status 200 CDXJ entries for collection IA we get 132113441

$ grep -a 'status": "200"' IA.cdxj | wc -l
132113441

This discrepancy could be due to duplication, but more likely it has to do with the numbers being outdated due to shifting from NutchWax to pywb for URL indexing.

We need confirm for all collections whether this number is correct or not, and rectify it when needed.

[dcgomes]

The "Total collected files" usually is counted from the crawl logs. However, the IA collection was not crawled, it was donated. Therefore probably this number was obtained from the initial Nuthwax indexing. The CDX count is more reliable.

@vitgou change Replay tab on https://docs.google.com/spreadsheets/d/1hfLyq9N2ZJEd1hra2V0OPI24jTy42ByR6M667rG-oHo/edit?gid=1778054497#gid=1778054497:

1. Replace column "Estimated number of files indexed on CDXJ" by "Number of indexed files from CDXJ (HTTP 200)"
2. Add "Total number of files indexed on CDXJ"