Error: [scrap] page.goto() Error

[MrOrz]

View details in Rollbar: https://rollbar.com/mrorz/url-resolver/items/35/

Error: net::ERR_SPDY_PROTOCOL_ERROR at http://sg.news-republic.com/article/i6786616295023444481?app_id=1239&c=sys&gid=6786616295023444481&impr_id=6786695957527513346&language=zh-hant&region=tw&user_id=6751299899164673025
  File "/srv/www/node_modules/puppeteer/lib/Page.js", line 622, in navigate
            return response.errorText ? new Error(`${response.errorText} at ${url}`) : null;
  File "<anonymous>", line unknown, in runMicrotasks
  File "internal/process/task_queues.js", line 94, in processTicksAndRejections
  File "/srv/www/node_modules/puppeteer/lib/Page.js", line 595, in async Page.goto
        let error = await Promise.race([
  File "/srv/www/src/lib/scrap.js", line 131, in async scrap
        response = await page.goto(url, {
  File "/srv/www/src/resolvers/resolveUrls.js", line 25, in async
            const fetchResult = await fetcher;
  File "index 0", line unknown, in async Promise.all

[MrOrz]

Related: puppeteer/puppeteer#2377

• ERR_CERT_AUTHORITY_INVALID is not reproducible in dev mode, but reproducible by docker image built on Mac.
• ERR_SPDY_PROTOCOL_ERROR is not reproducible in both dev mode and docker images built on Mac.
• On staging machine, sending pages that reported ERR_SPDY_PROTOCOL_ERROR will cause page crash, which is not reproducible on local machine.

[MrOrz]

實測：
On staging machine, ettoday url is still crashing the page, without triggering so-called SPDY_PROTOCOL_ERROR.

URLs used: https://cofacts.hacktabl.org/reply/a19d7W8BNefAB_hw4jdp

Error:

url-resolver_1_d44e6628b5aa | Encountered error while handling an uncaught exception.
url-resolver_1_d44e6628b5aa | Error: ignored identical item
url-resolver_1_d44e6628b5aa |     at Rollbar._log (/srv/www/node_modules/rollbar/src/rollbar.js:106:19)
url-resolver_1_d44e6628b5aa |     at Rollbar.error (/srv/www/node_modules/rollbar/src/rollbar.js:73:8)
url-resolver_1_d44e6628b5aa |     at Rollbar._uncaughtError (/srv/www/node_modules/rollbar/src/server/rollbar.js:207:15)
url-resolver_1_d44e6628b5aa |     at Rollbar.<anonymous> (/srv/www/node_modules/rollbar/src/server/rollbar.js:490:10)
url-resolver_1_d44e6628b5aa |     at process.fn (/srv/www/node_modules/rollbar/src/server/rollbar.js:503:5)
url-resolver_1_d44e6628b5aa |     at process.emit (events.js:223:5)
url-resolver_1_d44e6628b5aa |     at processPromiseRejections (internal/process/promises.js:201:33)
url-resolver_1_d44e6628b5aa |     at processTicksAndRejections (internal/process/task_queues.js:95:32) {
url-resolver_1_d44e6628b5aa |   item: {
url-resolver_1_d44e6628b5aa |     message: undefined,
url-resolver_1_d44e6628b5aa |     err: Error: Page crashed!
url-resolver_1_d44e6628b5aa |         at Page._onTargetCrashed (/srv/www/node_modules/puppeteer/lib/Page.js:211:24)
url-resolver_1_d44e6628b5aa |         at CDPSession.<anonymous> (/srv/www/node_modules/puppeteer/lib/Page.js:122:56)
url-resolver_1_d44e6628b5aa |         at CDPSession.emit (events.js:223:5)
url-resolver_1_d44e6628b5aa |         at CDPSession._onMessage (/srv/www/node_modules/puppeteer/lib/Connection.js:200:12)
url-resolver_1_d44e6628b5aa |         at Connection._onMessage (/srv/www/node_modules/puppeteer/lib/Connection.js:112:17)
url-resolver_1_d44e6628b5aa |         at WebSocket.<anonymous> (/srv/www/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)
url-resolver_1_d44e6628b5aa |         at WebSocket.onMessage (/srv/www/node_modules/ws/lib/event-target.js:120:16)
url-resolver_1_d44e6628b5aa |         at WebSocket.emit (events.js:223:5)
url-resolver_1_d44e6628b5aa |         at Receiver.receiverOnMessage (/srv/www/node_modules/ws/lib/websocket.js:789:20)
url-resolver_1_d44e6628b5aa |         at Receiver.emit (events.js:223:5),
url-resolver_1_d44e6628b5aa |     custom: undefined,
url-resolver_1_d44e6628b5aa |     timestamp: 1580235352427,
url-resolver_1_d44e6628b5aa |     uuid: 'c7f3d1fe-82e0-42c0-f58b-2385a0c88962',
url-resolver_1_d44e6628b5aa |     _originalArgs: [Arguments] {
url-resolver_1_d44e6628b5aa |       '0': Error: Page crashed!
url-resolver_1_d44e6628b5aa |           at Page._onTargetCrashed (/srv/www/node_modules/puppeteer/lib/Page.js:211:24)
url-resolver_1_d44e6628b5aa |           at CDPSession.<anonymous> (/srv/www/node_modules/puppeteer/lib/Page.js:122:56)
url-resolver_1_d44e6628b5aa |           at CDPSession.emit (events.js:223:5)
url-resolver_1_d44e6628b5aa |           at CDPSession._onMessage (/srv/www/node_modules/puppeteer/lib/Connection.js:200:12)
url-resolver_1_d44e6628b5aa |           at Connection._onMessage (/srv/www/node_modules/puppeteer/lib/Connection.js:112:17)
url-resolver_1_d44e6628b5aa |           at WebSocket.<anonymous> (/srv/www/node_modules/puppeteer/lib/WebSocketTransport.js:44:24)
url-resolver_1_d44e6628b5aa |           at WebSocket.onMessage (/srv/www/node_modules/ws/lib/event-target.js:120:16)
url-resolver_1_d44e6628b5aa |           at WebSocket.emit (events.js:223:5)
url-resolver_1_d44e6628b5aa |           at Receiver.receiverOnMessage (/srv/www/node_modules/ws/lib/websocket.js:789:20)
url-resolver_1_d44e6628b5aa |           at Receiver.emit (events.js:223:5),
url-resolver_1_d44e6628b5aa |       '1': [Function]
url-resolver_1_d44e6628b5aa |     },
url-resolver_1_d44e6628b5aa |     _isUncaught: true
url-resolver_1_d44e6628b5aa |   }
url-resolver_1_d44e6628b5aa | }

[MrOrz]

But we are pushing new url-resolver to production anyway, since it (a) has smaller image size and (b) latest Google Chrome and (c) ignores HTTPS cert error. Hopefully it will reduce errors, at least the certificate ones.

[MrOrz]

After changing to new Puppeteer, the error changes to "ERR_HTTP2_PROTOCOL_ERROR". Quantity is almost the same.
https://rollbar.com/mrorz/url-resolver/items/42/
garris/BackstopJS#1082

[MrOrz]

Use #52 to track future development.