[Service Bus] StressTests scenarios #11546

HarshaNalluru · 2020-09-29T09:48:41Z

"stress-tests-track-2" folder is an independent npm project with a package.json. Currently, testing preview.6 version of the SDK.
The following is a brief introduction to the contents.

stressTestsBase

Creates a new service-bus queue
Provides methods that would call into each of the main methods of the service bus API
Tracks the info related to successes, failures and errors corresponding to the operations
Snapshots at every 5 seconds, logs into a file

scenario{...}.ts

Instantiates StressTestsBase, and uses the methods provided by the class and tailors the scenario
Each of the scenarios can be executed without any setup, just by running through ts-node
Takes command-line arguments which allows granular control over the scenario
Example - ts-node scenario{...}.ts --delayBetweenSendsInMs=100

setupAndGuide.md

As the name says, it has the setup and the commands to run to test a specific scenario

richardpark-msft

Looking like a great start to me.

I'm wondering if you can lean on any of the work we've got for our normal live tests (like creating/recreating entities, etc..)

sdk/servicebus/service-bus/test/stress-test-track-2/stressTestsBase.ts

richardpark-msft · 2020-09-29T18:37:28Z

Also want to mention that you've got a lot of TODOs about making the limits configurable, which is great. I would suggest we get that in place sooner rather than later or else we'll end up baking them in unintentionally and with these kinds of tests you want to be clear about what your measurements/intervals/timings are.

HarshaNalluru · 2020-09-29T21:58:09Z

I'm wondering if you can lean on any of the work we've got for our normal live tests (like creating/recreating entities, etc..)

Good idea.. I'll re-use wherever I can.

…e entry too

sdk/servicebus/service-bus/test/stress-test-track-2/SetupAndGuide.md

ramya-rao-a · 2020-10-08T18:46:44Z

sdk/servicebus/service-bus/test/stress-test-track-2/package.json

+  "author": "",
+  "license": "ISC",
+  "dependencies": {
+    "@azure/service-bus": "^7.0.0-preview.6",


Suggested change

"@azure/service-bus": "^7.0.0-preview.6",

"@azure/service-bus": "^7.0.0-preview.7",

Currently, the code is written based on preview.6, I'll update both the code and the version to incorporate the changes that we recently did to sessions.

Forgot to update that I've updated this! :P

ramya-rao-a · 2020-10-08T18:53:02Z

sdk/servicebus/service-bus/test/stress-test-track-2/scenarioBatchReceive.ts

+  let receiver: ServiceBusReceiver<ServiceBusReceivedMessage>;
+
+  if (receiveMode === "receiveAndDelete") {
+    receiver = sbClient.createReceiver(stressBase.queueName, {
+      receiveMode: "receiveAndDelete"
+    });
+  } else {
+    receiver = sbClient.createReceiver(stressBase.queueName);
+  }


Why not use the receiveMode directly?

Suggested change

let receiver: ServiceBusReceiver<ServiceBusReceivedMessage>;

if (receiveMode === "receiveAndDelete") {

receiver = sbClient.createReceiver(stressBase.queueName, {

receiveMode: "receiveAndDelete"

});

} else {

receiver = sbClient.createReceiver(stressBase.queueName);

}

const receiver = sbClient.createReceiver(stressBase.queueName, {

receiveMode: "receiveAndDelete"

});

That's a typescript limitation with the string literal union types - example to try out.

In a gist, it tries to match the union type with the string literal type from each of the overloads in the function definition, and none of them match individually.

sdk/servicebus/service-bus/test/stress-test-track-2/scenarioPeekMessages.ts

ramya-rao-a · 2020-10-08T19:00:09Z

sdk/servicebus/service-bus/test/stress-test-track-2/scenarioRenewMessageLock.ts

+      );
+      elapsedTime = new Date().valueOf() - startedAt.valueOf();
+      messages.map((msg) =>
+        stressBase.renewMessageLockUntil(


It is not clear to me what our expectation here is...
The renewMessageLockUntil will keep renewing the lock... what are we looking for? That no error occurs?
Where will the error surface?

And when does the program end? We are awaiting on receiveMessages() but not really for the renewMessageLockUntil to finish?

The time duration for renewMessageLockUntil is less than the testDuration always, and hence would end before, and lock renewals are being done through setTimeouts.

The intention behind the test is to have many messages for which we'd be renewing the locks, note down the errors if any, observe the limits.. as in number of messages can we handle, etc.

ramya-rao-a · 2020-10-08T19:02:33Z

sdk/servicebus/service-bus/test/stress-test-track-2/scenarioRenewSessionLock.ts

+    while (elapsedTime < testDurationInMs) {
+      let receiver;
+      try {
+        receiver = await sbClient.createSessionReceiver(stressBase.queueName, {


Why create a new receiver in each iteration of the loop?

The intention behind this test to keep renewing the locks for multiple sessions and observe any unexpected behaviour with maintaining a huge number of sessions at the same time.

…ide.md Co-authored-by: Ramya Rao <[email protected]>

…om/HarshaNalluru/azure-sdk-for-js into harshan/sb/service-bus-stress-base

…to harshan/sb/service-bus-stress-base

…ent fixes

### Original issue - #11108 Follow-up of the stress tests PR #11546, here are the changes - Uses the latest preview.8 version of the service-bus SDK - Fixes the boolean options being passed as command-line args - Multiple sessions support in the renew session lock test - Adds (some)new options that can be passed as command-line args to the tests - And maybe minor edits

HarshaNalluru added 2 commits September 29, 2020 02:46

package.json and tsconfig.json

09ce3da

samples

7c9fa2f

ghost added the Service Bus label Sep 29, 2020

Add TODOs

22d7226

HarshaNalluru changed the title ~~[Service Bus] StressTests framework and starter scenarios~~ [Service Bus] StressTests starter scenarios Sep 29, 2020

HarshaNalluru changed the title ~~[Service Bus] StressTests starter scenarios~~ [Service Bus] StressTests scenarios Sep 29, 2020

HarshaNalluru added 4 commits September 29, 2020 09:58

init.. cration and deletion on resources

68251f6

Add a console.log

5058a2a

Grouping the metrics and info accordingly

5b2d840

update TODO

2b41b67

richardpark-msft reviewed Sep 29, 2020

View reviewed changes

sdk/servicebus/service-bus/test/stress-test-track-2/stressTestsBase.ts Outdated Show resolved Hide resolved

HarshaNalluru added 6 commits September 29, 2020 14:35

Added interfaces for grouping

fc29a98

Address Richar'd feedback

2fc740d

minor todo

1366e52

Simplified messageLockRenewalTimers

2810c23

JS Docs

2100a7b

minor changes to logs

f0cd9ef

HarshaNalluru added 11 commits September 29, 2020 15:25

renewLock -> renewMessageLockUntil

8835660

formatting

7704d75

readme

49d8faa

single sender - command line arguments

f383458

More TODOs

827ac0d

messagesSent filling - missed

69e7e9e

batch receive scenario args

6470326

Add minimist package for command line args

d4d19e1

Update stress tests with command line args from minimist

bf2b89c

file renames

9b5452d

renew message lock - scenario completed - command line args and readm…

5e56406

…e entry too

HarshaNalluru added 4 commits October 5, 2020 03:02

omit readme in stress folder

104657f

update ommitted path

24141a5

close-open scenario draft

cb31810

close open scenario done

bbbac0a

HarshaNalluru marked this pull request as ready for review October 8, 2020 16:50

HarshaNalluru requested review from bterlson, chradek, danieljurek, KarishmaGhiya, mikeharder, mitchdenny, ramya-rao-a and weshaggard as code owners October 8, 2020 16:50

ramya-rao-a reviewed Oct 8, 2020

View reviewed changes

HarshaNalluru and others added 11 commits October 8, 2020 19:34

Update sdk/servicebus/service-bus/test/stress-test-track-2/SetupAndGu…

9f59851

…ide.md Co-authored-by: Ramya Rao <[email protected]>

pass fromSeqNum to peekMessages

85f92ba

Merge branch 'harshan/sb/service-bus-stress-base' of https://github.c…

6c7199f

…om/HarshaNalluru/azure-sdk-for-js into harshan/sb/service-bus-stress-base

Take in changes from preview.7

5cc5ab7

Merge branch 'master' of https://github.com/Azure/azure-sdk-for-js in…

6df0f74

…to harshan/sb/service-bus-stress-base

pick up current generated package - for stress tests - to get the rec…

0fe826e

…ent fixes

Add initializer options to the report

92a17fb

Move logging to the top in init

b38461a

stress base updates - take settleOnMessageReceived

8fe7cfd

settleMessageOnReceive option for batch receive

f8796e0

formatting

8a46d91

ramya-rao-a approved these changes Oct 30, 2020

View reviewed changes

ramya-rao-a merged commit b6cb91d into Azure:master Oct 30, 2020

HarshaNalluru mentioned this pull request Nov 10, 2020

[Service Bus] Stress Tests - followup PR #12229

Merged

HarshaNalluru mentioned this pull request Nov 11, 2020

[Service Bus] Stress/Perf testing for track 2 API #11108

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Service Bus] StressTests scenarios #11546

[Service Bus] StressTests scenarios #11546

HarshaNalluru commented Sep 29, 2020 •

edited

Loading

richardpark-msft left a comment

richardpark-msft commented Sep 29, 2020

HarshaNalluru commented Sep 29, 2020

ramya-rao-a Oct 8, 2020

HarshaNalluru Oct 9, 2020

HarshaNalluru Oct 20, 2020

ramya-rao-a Oct 8, 2020

HarshaNalluru Oct 9, 2020 •

edited

Loading

ramya-rao-a Oct 8, 2020

HarshaNalluru Oct 9, 2020

ramya-rao-a Oct 8, 2020

HarshaNalluru Oct 9, 2020

	"@azure/service-bus": "^7.0.0-preview.6",
	"@azure/service-bus": "^7.0.0-preview.7",

[Service Bus] StressTests scenarios #11546

[Service Bus] StressTests scenarios #11546

Conversation

HarshaNalluru commented Sep 29, 2020 • edited Loading

stressTestsBase

scenario{...}.ts

setupAndGuide.md

richardpark-msft left a comment

Choose a reason for hiding this comment

richardpark-msft commented Sep 29, 2020

HarshaNalluru commented Sep 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HarshaNalluru Oct 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HarshaNalluru commented Sep 29, 2020 •

edited

Loading

HarshaNalluru Oct 9, 2020 •

edited

Loading