Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Update Hallucination README.md #3888

Merged
merged 7 commits into from
Aug 4, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion projects/hallucination/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ The following commands demonstrate how to train some of the models above; we int
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a T5-Large RAG-Turn Doc-Then-Turn model with DPR Retrieval on WoW

Expand All @@ -98,6 +99,7 @@ The following commands demonstrate how to train some of the models above; we int
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a BlenderBot-2.7B RAG Sequence Model with DPR-Poly Retrieval on WoW

Expand All @@ -113,6 +115,7 @@ For the BlenderBot model, we add extra positions to the encoder, so that we can
--model-parallel True --optimizer adam \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a BART-Large FiD Model, with a DPR Retriever initialized from a DPR Model trained with RAG.

Expand All @@ -127,6 +130,7 @@ This is the **BART FiD RAG** model specified above.
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a T5-Base FiD Model, using a smaller index for debug purposes.

Expand All @@ -141,4 +145,5 @@ We provide a smaller FAISS index comprising ~3k documents, which encompasses all
--log-every-n-secs 30 --lr-scheduler reduceonplateau --lr-scheduler-patience 1 \
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model