[RAG] Fix ReGReT

[klshuster]

Patch description
Turns out the test I wrote for ReGReT was broken, so I didn't catch the bugs here. ReGReT should work now.

CC #3927
Testing steps
Fixed the test and confirmed everything works

$ pytest -k TestRegret
====== test session starts ======
platform linux -- Python 3.7.9, pytest-6.2.1, py-1.10.0, pluggy-1.0.0.dev0
rootdir: /private/home/kshuster/ParlAI, configfile: pytest.ini
plugins: hydra-core-1.0.7, requests-mock-1.8.0, regressions-2.1.1, datadir-1.3.1
collected 666 items / 664 deselected / 2 selected

nightly/gpu/test_rag.py ..                                                                                                                                                     [100%]

====== slowest 10 durations ======
56.77s call     tests/nightly/gpu/test_rag.py::TestRegret::test_rag_regret_sep
11.50s call     tests/nightly/gpu/test_rag.py::TestRegret::test_rag_regret_same

(4 durations < 0.005s hidden.  Use -vv to show these durations.)
===== 2 passed, 664 deselected, 2 warnings in 76.86s (0:01:16) =====

[mojtaba-komeili]

LGTM.

[spencerp]

Nice! How broken was this? Are there any examples in docs that we need to update with this new flag for them to work?

also, plz teach me how to fix regret more generally thx

[klshuster]

How broken was this?

So broken it literally didn't work.

Are there any examples in docs that we need to update with this new flag for them to work?

I haven't included any examples in the docs for this, as it was not a highlighted model within the project; however, if more people are interested in the future this is something I can revisit.

plz teach me how to fix regret more generally thx

Some code pointers, if you are indeed interested:

1. ReGReT only has three (well, now four) flags: --regret controls using the model; --regret-model-file can be a separate retrieve+generate model; and --regret-intermediate-maxlen controls the maximum generation length of the intermediate generation (used in the second round of retrieval)
2. We build the ReGReT model in build_regret_model. This is essentially a no-op if we don't specify the model file, as the regret model is just the same as the normal, full model. Otherwise, we load up and create a new RagModel with the specified model file. There are a couple optimizations here that make this faster/more RAM-efficient; first, if the separate regret model has the same options for its index and passages, we share them between the two models so we don't load it twice; and, with this new flag I'm adding, you can now manually specify to share the same index and passages.
3. During training and eval, there are two functions required for using ReGReT: _regret_generate and _regret_rebatchify. The former simply swaps around agent attributes and runs a full generation phase to get the outputs from the first round of ReGReT; the latter then creates a new batch where the query vector is no longer the human response but the generation from the first round of ReGReT; note that the final generation is still dependent on the input text, and it is just the retrieval component that receives augmented input