Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Add BERT QA Scala/Java example #14592

Merged
merged 5 commits into from
Apr 5, 2019
Merged

Add BERT QA Scala/Java example #14592

merged 5 commits into from
Apr 5, 2019

Conversation

lanking520
Copy link
Member

@lanking520 lanking520 commented Apr 2, 2019 •
edited
Loading

Description

As title explained, this example shows how Scala/Java can do the inference with BERT model.
@zachgk @andrewfayres @gigasquid @aaronmarkham

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@lanking520 lanking520 requested review from nswamy and yzhliu as code owners April 2, 2019 05:10
@lanking520 lanking520 self-assigned this Apr 2, 2019
@lanking520 lanking520 added Example Java Label to identify Java API component labels Apr 2, 2019
@gigasquid
Copy link
Member

Great work! This is awesome.

piyushghai
piyushghai suggested changes Apr 2, 2019
View reviewed changes
Copy link
Contributor

@piyushghai piyushghai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments, otherwise this is good!

aaronmarkham
aaronmarkham suggested changes Apr 3, 2019
View reviewed changes
...ckage/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertDataParser.java Show resolved Hide resolved
...ckage/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertDataParser.java Outdated Show resolved Hide resolved
...ckage/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertDataParser.java Outdated Show resolved Hide resolved
...ckage/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertDataParser.java Outdated Show resolved Hide resolved
...ckage/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertDataParser.java Outdated Show resolved Hide resolved
scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/README.md Outdated Show resolved Hide resolved
scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/README.md Outdated Show resolved Hide resolved
scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/README.md
By default, this model are using `bert_12_768_12` model with extra layers for QA jobs.

After that, to be able to use it in Java, we need to export the dictionary from the script to parse the text
to actual indexes. Please add the following lines after [this line](https://github.com/dmlc/gluon-nlp/blob/master/scripts/bert/staticbert/static_finetune_squad.py#L262).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dangerous... if that file changes, this link goes bad... another way?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently no, since they did not provide vocabulary extraction

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok what if you quote the line - that way if it moves a bit, people still know where to work.

scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/README.md Outdated Show resolved Hide resolved
...examples/src/test/java/org/apache/mxnetexamples/javaapi/infer/predictor/BertExampleTest.java Show resolved Hide resolved
@lanking520
Copy link
Member Author

@aaronmarkham @piyushghai @zachgk Could you please double check

aaronmarkham
aaronmarkham approved these changes Apr 4, 2019
View reviewed changes
Copy link
Contributor

@aaronmarkham aaronmarkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content LGTM!

piyushghai
piyushghai reviewed Apr 4, 2019
View reviewed changes
scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertQA.java
BertQA inst = new BertQA();
CmdLineParser parser = new CmdLineParser(inst);
parser.parseArgument(args);
BertDataParser util = new BertDataParser();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : util --> dataparser just a more meaningful variable name :)

piyushghai
piyushghai approved these changes Apr 4, 2019
View reviewed changes
Copy link
Contributor

@piyushghai piyushghai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small comment and then we're good to go :)

@lanking520 lanking520 merged commit d5d1d7a into apache:master Apr 5, 2019
nswamy pushed a commit that referenced this pull request Apr 5, 2019
@lanking520 @nswamy
Add BERT QA Scala/Java example (#14592)
6b800e4 
* add BertQA major code piece

* add scripts and bug fixes

* add integration test

* address comments

* address doc comments
@lanking520 lanking520 deleted the bertQA branch April 8, 2019 18:28
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
@lanking520
Add BERT QA Scala/Java example (apache#14592)
ee5e17e 
* add BertQA major code piece

* add scripts and bug fixes

* add integration test

* address comments

* address doc comments
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@zachgk zachgk zachgk requested changes

@piyushghai piyushghai piyushghai approved these changes

@aaronmarkham aaronmarkham aaronmarkham approved these changes

@nswamy nswamy Awaiting requested review from nswamy nswamy is a code owner

@yzhliu yzhliu Awaiting requested review from yzhliu yzhliu is a code owner

Assignees

@lanking520 lanking520

Labels
Example Java Label to identify Java API component
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lanking520 @gigasquid @zachgk @aaronmarkham @piyushghai