Skip to content
This repository has been archived by the owner on Dec 4, 2023. It is now read-only.

[SDK] Add bot-ai-qna library #907

Conversation

Batta32
Copy link
Contributor

@Batta32 Batta32 commented Jan 15, 2021

Fixes #740

  • Migration of source code
  • Migration of tests
  • Testing

Description

We are adding the SDK bot-ai-qna library in Java having into account the C# Microsoft.Bot.Builder.AI.QnA present in botbuilder-dotnet repository.

Specific Changes

  • Generate main structure of the library
  • Migrate dialogs folder
  • Migrate models folder
  • Add icon.png in resources folder
  • Migrate utils folder
  • Migrate root files
  • Add testData folder
  • Add QnAMakerRecognizerTests unit tests
  • Add QnAMakerTests unit tests
  • Add QnAMakerTraceInfoTests unit tests

Migrated structure
image

Migrated test structure
image

Testing

QnA tests passing correctly 48/48
image

mvn clean install passing correctly
image

Feature Plan

-

@Batta32 Batta32 self-assigned this Jan 15, 2021
@tracyboehrer
Copy link
Member

Parts of the QnAMakerDialog have been updated to be Adaptive friendly. This won't apply for this version on the Java SDK since Adaptive won't be part of the GA release.

For example, in QnAMakerDialog: StringExpression, IntegerExpression, etc...

Other:

  1. QnAMakerComponentRegistration is not needed (can't actually be ported, so delete the file)
  2. schemas folder can be deleted.

@Batta32
Copy link
Contributor Author

Batta32 commented Jan 29, 2021

Thanks @tracyboehrer for the feedback! We will be applying the changes as soon as we can 😊.

Batta32 and others added 7 commits February 5, 2021 16:01
* Delete QnAMakerComponentRegistration class

* Delete schema folder

* Fix ref to AdaptiveExpressions in QnAMakerDialog

* Fix ref adaptiveExpressions in QnAMakerRecognizer

* Apply internal feedback

* Add dialogs library, fix references

* Fix PMD and checkstyles errors

* Add description in javadocs for IOException

* Apply internal feedback
* Add files into the TestData folder

* Add QnAMakerTraceInfoTests

* Add MyTurnContext class

* Change tests location

* Add MyTurnContext class

* Add returns in set and get Locale methods

* Fix consistency multi-line methods

* Add default inputHint

Co-authored-by: Martin Battaglino <[email protected]>

* Add QnAMakerCardEqualityComparer

* Change tests location

* Add QnAMakerCardEqualityComparer class

Co-authored-by: Martin Battaglino <[email protected]>

* Move location of QnAMakerTraceInfoTests

* Remove TestData

* Add testData

* Add QnAMakerTest class

* Add files into the TestData folder

* Change location of tests

* Add Mockito dependency

* Add more test to QnAMakerTests class

* Migrate more tests

* Remove TestData

* Add testData

* Add QnAMakerTraceInfoTests

* Add MyTurnContext class

* Change tests location

* Add MyTurnContext class

* Add returns in set and get Locale methods

* Fix consistency multi-line methods

* Add default inputHint

Co-authored-by: Martin Battaglino <[email protected]>

* Add QnAMakerCardEqualityComparer

* Change tests location

* Add QnAMakerCardEqualityComparer class

Co-authored-by: Martin Battaglino <[email protected]>

* Add private methods in QnAMakerTests

* Fix issues

* Add MockWebServer to tests

* replace ref to interceptHttp

* Add try/catch/finally

* Replace thenApply with thenAccept

* Add mocked OkHttpClient

* Fix errors

* Fix delta

* Remove merge files

* Fix QnAMakerTest

* Clean-up pom, remove QnAMakerDialogTest, update QnAMakerTests

Co-authored-by: Martin Battaglino <[email protected]>

Co-authored-by: Franco Alvarez <[email protected]>
* Add QnAMakerRecognizer, tests and fix some issues

* Apply internal feedback
* Fix QnAMakerRecognizerTests unit tests

* Fix mock issues and internal issues

* Fix QnAMakerRecognizerTests unit tests

* Rollback unused changes

* Add body in http execution

* Fix tests execution adding the correct Test import

* Apply internal feedback

* Fix QnA unit tests

* Fix generate answer utils

* Fix errors

* Fix mock issues and internal issues

* Fix issues with the url

* Add remaining join

* Fix qnaMakerTestUnsuccessfulResponse test

* Refactor of QnAMakerTests

* Import correct json and fix string format

* Fix qnaMakerTraceActivity test

* Fix tests with request object

* Fix qnaMakerLowScoreVariation

* Fix qnaMakerTestThresholdInQueryOption test

* Fix telemetryOverride test

* Enable qnaMakerTestOptionsHydration failing test

* Fix qnaMakerTestOptionsHydration test

* Fix telemetry tests

* Fix disparity in GenerateAnswerUtils

* Fix remaining unit tests

* Change jupiter to junit

* Remove mockito-junit-jupiter dependency

* Fix checkstyle errors

* Apply internal feedback

Co-authored-by: Franco Alvarez <[email protected]>
Co-authored-by: Federico Bernal <[email protected]>

* Rollback id to Id

* Improve readFileContent method and remove comments

Co-authored-by: Franco Alvarez <[email protected]>
Co-authored-by: Federico Bernal <[email protected]>
@Batta32 Batta32 marked this pull request as ready for review February 19, 2021 17:41
@Batta32
Copy link
Contributor Author

Batta32 commented Feb 19, 2021

Hi @tracyboehrer , we finished with the migration of the bot-ai-qna library having all the unit tests passing correctly so this is ready to be reviewed and merged 😊.

Copy link
Member

@tracyboehrer tracyboehrer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Drop "I" for IQnAMakerClient and ITelemetryQnAMaker to be consistent with other Java SDK interfaces
  • Drop QnAMaker.kind and associated getter. This is an Adaptive thing that isn't used yet.
  • I believe we can drop QnAMaker constructor "sourceFilePath" and "sourceLineNumber". This is part of DotNets debugging that we don't currently have in Java.
  • I believe we can delete resources/icon.png. I think this is just for NuGet packaging in DotNet.
  • Is QnAMaker.httpClient (and accessors) used? DotNet support passing in a custom client that we haven't supported anywhere else.

Bug

QnAMaker.getAnswerRaw

if (messageActivity == null || messageActivity.getType() != ActivityTypes.MESSAGE) {

Activity.getType is a string so that comparison won't work. Use:

Activity.isType(ActivityTypes.MESSAGE)

Recommend searching for "getType()" and see if that happens elsewhere.

Bug

QnAMakerDialog.callTrain, line 934

QueryResult qnaResult = trainResponses
                .stream().filter(kvp -> kvp.getQuestions()[0] == reply).findFirst().orElse(null);

Performing string comparison with ==. We typically use StringUtils.equals.

Recommend maybe searching for "==", "!=", etc... and checking for other string comparisons. This would also help you find the previous bug.

Minor:

QnAMaker.fillQnAEvent:

return CompletableFuture.completedFuture(new Pair(telemetryPropertiesResult, telemetryMetricsResult));

Produces an unchecked assignment warning. Should be:

return CompletableFuture.completedFuture(new Pair<>(telemetryPropertiesResult, telemetryMetricsResult));

@tracyboehrer
Copy link
Member

@Batta32 Lee said there is a CheckStyle rule you can add that will flag string comparisons. That might be the quickest way to find those. Check with him on that. We were going to add it as part of the work Lee is doing, but its just as well if you do.

@LeeParrishMSFT
Copy link
Contributor

LeeParrishMSFT commented Feb 22, 2021

@Batta32 Lee said there is a CheckStyle rule you can add that will flag string comparisons. That might be the quickest way to find those. Check with him on that. We were going to add it as part of the work Lee is doing, but its just as well if you do.

This is what you can add as a child to the section:

    <!-- check for usage of literal equality "==" between string -->
   <module name="StringLiteralEquality"/>

@Batta32
Copy link
Contributor Author

Batta32 commented Feb 23, 2021

Thanks @tracyboehrer and @LeeParrishMSFT for your feedback! We will be implementing the requested changes in order to update the PR.

  • httpClient is only used in HttpRequestUtils in C# and Java as well. We created the client where it's used instead of adding unused properties in QnAMaker, TrainUtils and GenerateAnswerUtils. We can add the property on these classes too if you want 😊.
  • We will add the StringLiteralEquality rule in the bot-checkstyle.xml file and fix the issues that will appear in the entire repository.

* Remove I from ITelemetryQnAMaker IQnAMakerClient

* Remove kind property

* Drop unused constructor in QnAMakerDialog

* Remove resources/icon.png

* Replace getType with isType

* Replace == comparison

* Fix unchecked assignment warning in fillQnAEvent

* Add StringLiteralEquality

* Remove sourceFilePath and sourceLineNumber

* Use activity.getType in tests

* Remove unnecesary whitespace

Co-authored-by: Martin Battaglino <[email protected]>
@Batta32
Copy link
Contributor Author

Batta32 commented Feb 23, 2021

Hi @tracyboehrer & @LeeParrishMSFT, we already applied the requested changes 😊.

@tracyboehrer
Copy link
Member

@Batta32 Awesome! Can you merge the latest from main?

@Batta32
Copy link
Contributor Author

Batta32 commented Feb 23, 2021

Done!

@tracyboehrer tracyboehrer merged commit 7b2d3a4 into microsoft:main Feb 23, 2021
@Batta32 Batta32 deleted the feature/southworks/library/bot-ai-qna/base branch February 23, 2021 18:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bot-ai-qna package
4 participants