Not possible to redeem tickets with Redeemer on Arbitrum

[leszko]

It's not possible to redeem tickets with a separate Redeemer, because it returns the following error reported in Discord:

I0221 23:10:46.788973   20060 queue.go:134] Ticket recipient is not active in this round, cannot redeem ticket recipient=0xBe8770603dAf200b1Fa136aD354BA854928e602B

Problem

1. Redeemer checks if the ticket recipient is an active orchestrator in the local SQLite (in orchestrator table).
2. The orchestrator table is a cache of the current on-chain orchestrators, which is refreshed:
   • When on-chain block logs are propagated with orchestrator registration
   • (Broadcaster-only) during the orchestrator discovery
   • (Orchestrator-only) during the orchestrator setup
3. Redeemer may not have the correct orchestrator cache in the following cases
   • It was first started after orchestrators were already registered (we backfill blocks only until last block found in SQLite DB)
   • Logs has crashed (probably because of this bug: Nil Pointer in watchers while running in Arbitrum #2265

Solution

The solution is to either just allow redeeming tickets if the given orchestrator is not found in the local SQLite DB or refresh orchestrator if the given orchestrator is not found.

[yondonfu]

IIUC this problem was introduced with #2253 because:

• That PR added a requirement in pm/queue.go for the recipient of the ticket to be active and the active check was implemented using a DB query against the orchestrators table
• The orchestrators table could be missing the recipient of the ticket if the node is running in redeemer mode so the table is empty for the DB query and as a result the active check in pm/queue.go fails

Does that sound right to you?

If so, could a solution be to ensure that the address that the node expects to receive tickets for is included in the orchestrators table? This is what we do when the node is running in orchestrator mode using the setupOrchestrator() function. This function will add the specified address to the orchestrators table ensuring that it exists in the DB. Could we do something similar when the node is running in redeemer mode i.e. call setupOrchestrator() when the node is running in redeemer mode and when it is running in orchestrator mode?

[leszko]

IIUC this problem was introduced with #2253 because:

• That PR added a requirement in pm/queue.go for the recipient of the ticket to be active and the active check was implemented using a DB query against the orchestrators table
• The orchestrators table could be missing the recipient of the ticket if the node is running in redeemer mode so the table is empty for the DB query and as a result the active check in pm/queue.go fails

Does that sound right to you?

Yes, that is correct.

If so, could a solution be to ensure that the address that the node expects to receive tickets for is included in the orchestrators table? This is what we do when the node is running in orchestrator mode using the setupOrchestrator() function. This function will add the specified address to the orchestrators table ensuring that it exists in the DB. Could we do something similar when the node is running in redeemer mode i.e. call setupOrchestrator() when the node is running in redeemer mode and when it is running in orchestrator mode?

Ahh, right, I missed the point that Redeemer is always running for one eth orch address only. I thought that it was possible to run a Redeemer for multiple orchestrators, but that actually not the case.

Updated the related PR: https://github.com/livepeer/go-livepeer/pull/2284/files

PTAL. Note that I kept the change with easing the condition for checking if the orchestrator is active. I think that if the given orchestrator cannot be found in DB, then we should try to redeem the ticket.