[Style-Controlled Generation] Open-source a second style classifier

[EricMichaelSmith]

Patch description

By request, open-source a second classifier of dialogue style from Controlling Style in Generated Dialogue. Unlike the first classifier, this one takes as context only the specific utterance to be classified, omitting any previous turns of the conversation. Add a teacher for testing this new classifier, as well as a unit test.

Testing steps

Unit test

Command:

pytest tests/nightly/gpu/test_style_gen.py

Output:

==== 4 passed, 32 warnings in 159.13s (0:02:39) ====

Evaluating the classifier on a pre-labeled version of BlendedSkillTalk

Command:

parlai eval_model \
--task style_gen:CurrUttOnlyStyle \
--wrapper-task style_gen:LabeledBlendedSkillTalk \
--model-file zoo:style_gen/curr_only_classifier/model \
--model projects.style_gen.classifier:ClassifierAgent \
--classes-from-file image_chat_personalities_file

Output:

16:46:41 | Finished evaluating tasks ['style_gen:CurrUttOnlyStyle'] using datatype valid
    accuracy  bleu-4  <PER_CLASS_METRICS_SNIPPED>  clen  ctpb  ctps  ctrunc  ctrunclen  exps  exs    f1  gpu_mem  llen  loss    lr  ltpb  ltps  ltrunc  ltrunclen   tpb   tps  \
       .4311 .004642                              19.18 19.18 425.9       0          0  22.2 5651 .4363    .1586 5.633 2.658 5e-10 5.633 125.1       0          0 24.82 550.9
    weighted_f1
          .4319

The accuracy is low here because the task was labeled using a different classifier, zoo:style_gen/prev_curr_classifier/model.

Displaying test predictions from the classifier

Command:

parlai display_model \
--task style_gen:CurrUttOnlyStyle \
--wrapper-task style_gen:LabeledBlendedSkillTalk \
--model-file zoo:style_gen/curr_only_classifier/model \
--model projects.style_gen.classifier:ClassifierAgent \
--classes-from-file image_chat_personalities_file \
--num-examples 10

Output:

I received on-the-job training when i first started
    labels: Businesslike
     model: Businesslike
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
For a good number of years now.
    labels: Conservative (Traditional, Conventional)
     model: Relaxed
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
That it is,especially if you dont take the proper measures
    labels: Critical
     model: Critical
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
Thats true,especially in this economy
    labels: Realistic
     model: Honest
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
Do you enjoy it?
    labels: Considerate
     model: Questioning
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
I feel you. Stretch along the halls
    labels: Sympathetic
     model: Empathetic
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
Sure, I have friends who I am always free to share with
    labels: Relaxed
     model: Warm
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
I have never done that.  Not really the physical activity type, but I'd be willing to give it a try, I guess
    labels: Open
     model: Open
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
Now that is more my speed. It tastes kind of strange, but I like it
    labels: Erratic
     model: Odd
- - - NEW EPISODE: style_gen:LabeledBlendedSkillTalk- - -
Yea, it's just one of those things. Once you start on it, you can't stop
    labels: Ordinary
     model: Realistic