Skip to content

Commit

Permalink
implemented being able to select tag when calling feature ref #471
Browse files Browse the repository at this point in the history
this now makes it possible to compose gatling tests out of existing suites more efficiently
  • Loading branch information
ptrthomas committed Jul 26, 2018
1 parent e5faab7 commit 4944fd7
Show file tree
Hide file tree
Showing 18 changed files with 141 additions and 53 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2823,6 +2823,17 @@ So you get the picture, any kind of complicated 'sign-in' flow can be scripted a
Do look at the documentation and example for [`configure headers`](#configure-headers) also as it goes hand-in-hand with `call`. In the above example, the end-result of the `call` to `my-signin.feature` resulted in the `authToken` variable being initialized. Take a look at how the [`configure headers`](#configure-headers) example uses the `authToken` variable.

### Call Tag Selector
You can "select" a single `Scenario` (or `Scenario`-s or `Scenario Outline`-s) by appending a "tag selector" at the end of the feature-file you are calling. For example:

```cucumber
call read('classpath:my-signin.feature@name=someScenarioName')
```

While the tag does not need to be in the `@key=value` form, it is recommended for readability when you start getting into the business of giving meaningful names to your `Scenario`-s.

This "tag selection" capability is designed for you to be able to "compose" flows out of existing test-suites when using the [Karate Gatling integration](karate-gatling). Normally we recommend that you keep your "re-usable" features lightweight - by limiting them to just one `Scenario`.

### Data-Driven Features
If the argument passed to the [call of a `*.feature` file](#calling-other-feature-files) is a JSON array, something interesting happens. The feature is invoked for each item in the array. Each array element is expected to be a JSON object, and for each object - the behavior will be as described above.

Expand Down
30 changes: 22 additions & 8 deletions karate-core/src/main/java/com/intuit/karate/FileUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,20 @@ private static String removePrefix(String text) {
int pos = text.indexOf(':');
return pos == -1 ? text : text.substring(pos + 1);
}

private static StringUtils.Pair parsePathAndTags(String text) {
int pos = text.indexOf(':');
text = pos == -1 ? text : text.substring(pos + 1); // remove prefix
pos = text.indexOf('@');
if (pos == -1) {
text = StringUtils.trimToEmpty(text);
return new StringUtils.Pair(text, null);
} else {
String left = StringUtils.trimToEmpty(text.substring(0, pos));
String right = StringUtils.trimToEmpty(text.substring(pos));
return new StringUtils.Pair(left, right);
}
}

private static enum PathPrefix {
NONE,
Expand All @@ -93,25 +107,25 @@ private static enum PathPrefix {
public static ScriptValue readFile(String text, ScriptContext context) {
text = StringUtils.trimToEmpty(text);
PathPrefix prefix = isClassPath(text) ? PathPrefix.CLASSPATH : (isFilePath(text) ? PathPrefix.FILE : PathPrefix.NONE);
String fileName = removePrefix(text);
fileName = StringUtils.trimToEmpty(fileName);
StringUtils.Pair pair = parsePathAndTags(text);
text = pair.left;
if (isJsonFile(text) || isXmlFile(text) || isJavaScriptFile(text)) {
String contents = readFileAsString(fileName, prefix, context);
String contents = readFileAsString(text, prefix, context);
ScriptValue temp = evalKarateExpression(contents, context);
return new ScriptValue(temp.getValue(), text);
} else if (isTextFile(text) || isGraphQlFile(text)) {
String contents = readFileAsString(fileName, prefix, context);
String contents = readFileAsString(text, prefix, context);
return new ScriptValue(contents, text);
} else if (isFeatureFile(text)) {
String contents = readFileAsString(fileName, prefix, context);
FeatureWrapper feature = FeatureWrapper.fromString(contents, context.env, text);
String contents = readFileAsString(text, prefix, context);
FeatureWrapper feature = FeatureWrapper.fromString(contents, context.env, text, pair.right);
return new ScriptValue(feature, text);
} else if (isYamlFile(text)) {
String contents = readFileAsString(fileName, prefix, context);
String contents = readFileAsString(text, prefix, context);
DocumentContext doc = JsonUtils.fromYaml(contents);
return new ScriptValue(doc, text);
} else {
InputStream is = getFileStream(fileName, prefix, context);
InputStream is = getFileStream(text, prefix, context);
return new ScriptValue(is, text);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public class AsyncFeature implements AsyncAction<ScriptValueMap> {
public AsyncFeature(FeatureWrapper feature, KarateBackend backend) {
this.feature = feature;
this.backend = backend;
this.iterator = feature.getSections().iterator();
this.iterator = feature.getSectionsByCallTag().iterator();
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ public static Map<String, Object> runFeature(File file, Map<String, Object> vars
}

public static Map<String, Object> runFeature(File file, CallContext callContext, KarateReporter reporter) {
FeatureWrapper featureWrapper = FeatureWrapper.fromFile(file, reporter);
FeatureWrapper featureWrapper = FeatureWrapper.fromFile(file, null, reporter);
ScriptValueMap scriptValueMap = CucumberUtils.callSync(featureWrapper, callContext);
return scriptValueMap.toPrimitiveMap();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -148,9 +148,9 @@ public static ScriptValueMap callSync(FeatureWrapper feature, CallContext callCo
return result.vars;
}

public static void callAsync(String filePath, CallContext callContext) {
public static void callAsync(String filePath, String callTag, CallContext callContext) {
File file = FileUtils.getFeatureFile(filePath);
FeatureWrapper feature = FeatureWrapper.fromFile(file);
FeatureWrapper feature = FeatureWrapper.fromFileAndTag(file, callTag);
callAsync(feature, callContext);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,21 @@
*/
package com.intuit.karate.cucumber;

import gherkin.formatter.model.Tag;
import java.util.List;
import java.util.stream.Collectors;

/**
*
* @author pthomas3
*/
public class FeatureSection {

private final int index;
private final FeatureWrapper feature;
private final ScenarioWrapper scenario;
private final ScenarioOutlineWrapper scenarioOutline;
private final ScenarioOutlineWrapper scenarioOutline;

public FeatureSection(int index, FeatureWrapper feature, ScenarioWrapper scenario, ScenarioOutlineWrapper scenarioOutline) {
this.index = index;
this.feature = feature;
Expand All @@ -46,13 +50,23 @@ public FeatureSection(int index, FeatureWrapper feature, ScenarioWrapper scenari
}
}

public List<String> getTags() {
List<Tag> tags;
if (isOutline()) {
tags = scenarioOutline.getScenarioOutline().getGherkinModel().getTags();
} else {
tags = scenario.getScenario().getGherkinModel().getTags();
}
return tags.stream().map(t -> t.getName()).collect(Collectors.toList());
}

public int getIndex() {
return index;
}
}

public FeatureWrapper getFeature() {
return feature;
}
}

public ScenarioWrapper getScenario() {
return scenario;
Expand All @@ -61,17 +75,17 @@ public ScenarioWrapper getScenario() {
public ScenarioOutlineWrapper getScenarioOutline() {
return scenarioOutline;
}

public boolean isOutline() {
return scenarioOutline != null;
}

public int getLine() {
if (isOutline()) {
return scenarioOutline.getLine();
} else {
return scenario.getLine();
}
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
public class FeatureWrapper {

private final String path;
private final String callTag;
private final String text;
private final List<String> lines;
private final CucumberFeature feature;
Expand All @@ -52,36 +53,40 @@ public ScriptEnv getEnv() {
return scriptEnv;
}

public String getCallTag() {
return callTag;
}

public void setEnv(ScriptEnv scriptEnv) {
this.scriptEnv = scriptEnv;
}

public static FeatureWrapper fromFile(File file) {
return fromFile(file, Thread.currentThread().getContextClassLoader(), null);
}

public static FeatureWrapper fromFile(File file, KarateReporter reporter) {
return fromFile(file, Thread.currentThread().getContextClassLoader(), reporter);

public static FeatureWrapper fromFileAndTag(File file, String callTag) {
return fromFile(file, callTag, Thread.currentThread().getContextClassLoader(), null);
}

public static FeatureWrapper fromFile(File file, ClassLoader classLoader, KarateReporter reporter) {
public static FeatureWrapper fromFile(File file, String callTag, KarateReporter reporter) {
return fromFile(file, callTag, Thread.currentThread().getContextClassLoader(), reporter);
}

public static FeatureWrapper fromFile(File file, String callTag, ClassLoader classLoader, KarateReporter reporter) {
String text = FileUtils.toString(file);
ScriptEnv env = new ScriptEnv(null, file.getParentFile(), file.getName(), classLoader, reporter);
return new FeatureWrapper(text, env, file.getPath());
return new FeatureWrapper(text, env, file.getPath(), callTag);
}

public static FeatureWrapper fromFile(File file, ScriptEnv env) {
String text = FileUtils.toString(file);
return new FeatureWrapper(text, env, file.getPath());
return new FeatureWrapper(text, env, file.getPath(), null);
}

public static FeatureWrapper fromString(String text, ScriptEnv scriptEnv, String path) {
return new FeatureWrapper(text, scriptEnv, path);
public static FeatureWrapper fromString(String text, ScriptEnv scriptEnv, String path, String callTag) {
return new FeatureWrapper(text, scriptEnv, path, callTag);
}

public static FeatureWrapper fromStream(InputStream is, ScriptEnv scriptEnv, String path) {
String text = FileUtils.toString(is);
return new FeatureWrapper(text, scriptEnv, path);
return new FeatureWrapper(text, scriptEnv, path, null);
}

public String joinLines(int startLine, int endLine) {
Expand All @@ -104,7 +109,7 @@ public String joinLines() {
public List<String> getLines() {
return lines;
}

public String getFirstScenarioName() {
if (featureSections == null || featureSections.isEmpty()) {
return null;
Expand All @@ -115,7 +120,7 @@ public String getFirstScenarioName() {
} else {
return fs.getScenario().getScenario().getGherkinModel().getName();
}
}
}

public CucumberFeature getFeature() {
return feature;
Expand All @@ -125,6 +130,25 @@ public List<FeatureSection> getSections() {
return featureSections;
}

public List<FeatureSection> getSectionsByCallTag() {
if (callTag == null) {
return featureSections;
}
List<FeatureSection> filtered = new ArrayList(featureSections.size());
for (FeatureSection fs : getSections()) {
List<String> tags = fs.getTags();
if (tags == null || tags.isEmpty()) {
continue;
}
for (String tag : tags) {
if (callTag.equals(tag)) {
filtered.add(fs);
}
}
}
return filtered;
}

public String getPath() {
return path;
}
Expand All @@ -135,7 +159,7 @@ public String getText() {

public FeatureWrapper addLine(int index, String line) {
lines.add(index, line);
return new FeatureWrapper(joinLines(), scriptEnv, path);
return new FeatureWrapper(joinLines(), scriptEnv, path, callTag);
}

public FeatureSection getSection(int sectionIndex) {
Expand Down Expand Up @@ -166,16 +190,17 @@ public FeatureWrapper replaceLines(int start, int end, String text) {
lines.remove(start);
}
lines.set(start, text);
return new FeatureWrapper(joinLines(), scriptEnv, path);
return new FeatureWrapper(joinLines(), scriptEnv, path, callTag);
}

public FeatureWrapper removeLine(int index) {
lines.remove(index);
return new FeatureWrapper(joinLines(), scriptEnv, path);
return new FeatureWrapper(joinLines(), scriptEnv, path, callTag);
}

private FeatureWrapper(String text, ScriptEnv scriptEnv, String path) {
private FeatureWrapper(String text, ScriptEnv scriptEnv, String path, String callTag) {
this.path = path;
this.callTag = callTag;
this.text = text;
this.scriptEnv = scriptEnv;
this.feature = CucumberUtils.parse(text, path);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ public void testMultiLineEdit() {
public void testIdentifyingStepWhichIsAnHttpCall() {
String text = "Feature:\nScenario:\n* method post";
ScriptEnv env = getEnv();
FeatureWrapper fw = FeatureWrapper.fromString(text, env, "dummy.feature");
FeatureWrapper fw = FeatureWrapper.fromString(text, env, "dummy.feature", null);
printLines(fw.getLines());
StepWrapper step = fw.getSections().get(0).getScenario().getSteps().get(0);
logger.debug("step name: '{}'", step.getStep().getName());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ private ScriptValueMap getRequest(String name) {
@Test
public void testServer() {
File file = FileUtils.getFileRelativeTo(getClass(), "server.feature");
FeatureWrapper featureWrapper = FeatureWrapper.fromFile(file);
FeatureWrapper featureWrapper = FeatureWrapper.fromFileAndTag(file, null);
FeatureProvider provider = new FeatureProvider(featureWrapper);
ScriptValueMap vars = provider.handle(getRequest("Billie"));
Match.equals(vars.get("response").getAsMap(), "{ id: 1, name: 'Billie' }");
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
@ignore
Feature:

@foo=bar1
Scenario:
* def step1 = 'step1'
* def step2 = 'step2'
* print 'bar1-1'
* print 'bar1-2'

@foo=bar2
Scenario:
* print 'bar2-1'
* print 'bar2-2'

Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ Feature:
Scenario:
* def before = 'before'
* print 1 + 2
* def callresult = call read('called_2.feature')
* def callresult = call read('called_2.feature@foo=bar2')
* def after = 'after'
14 changes: 11 additions & 3 deletions karate-gatling/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ class CatsSimulation extends Simulation {
)

val create = scenario("create").exec(karateFeature("classpath:mock/cats-create.feature"))
val delete = scenario("delete").exec(karateFeature("classpath:mock/cats-delete.feature"))
val delete = scenario("delete").exec(karateFeature("classpath:mock/cats-delete.feature@name=delete"))

setUp(
create.inject(rampUsers(10) over (5 seconds)).protocols(protocol),
Expand All @@ -62,6 +62,14 @@ class CatsSimulation extends Simulation {

}
```
### `karateProtocol`
This piece is needed because Karate is responsible for making HTTP requests while Gatling is only measuring the timings and managing threads. In order for HTTP requests to "aggregate" correctly in the Gatling report, you need to declare the URL patterns involved in your test. For example, in the example above, the `{id}` would be random - and Gatling would by default report each one as a different request. You also need to group requests by the HTTP method (`get`, `post` etc.) and you can also set a pause time (in milliseconds) if needed. We recommend you set that to `0` for everything unless you really need to artifically limit the requests per second. Make sure you wire up the `protocol` in the Gatling `setUp`.

### `karateFeature`
This executes a whole Karate feature as a "flow". Note how you can have concurrent flows in the same Gatling simulation.

#### Tag Selector
In the code above, note how a single `Scenario` (or multiple) can be "chosen" by appending the tag name to the `Feature` path. This allows you to re-use only selected tests out of your existing functional or regression test suites for composing a performance test-suite.

> The tag does not need to be in the `@key=value` form and you can use the plain "`@foo`" form if you want to. But using the pattern `@name=someName` is arguably more readable when it comes to giving your various `Scenario`-s meaningful names.
* `karateProtocol` - this piece is needed because Karate is responsible for making HTTP requests while Gatling is only measuring the timings and managing threads. In order for HTTP requests to "aggregate" correctly in the Gatling report, you need to declare the URL patterns involved in your test. For example, in the example above, the `{id}` would be random - and Gatling would by default report each one as a different request. You also need to group requests by the HTTP method (`get`, `post` etc.) and you can also set a pause time (in milliseconds) if needed. We recommend you set that to `0` for everything unless you really need to artifically limit the requests per second. Make sure you wire up the `protocol` in the Gatling `setUp`.
* `karateFeature` - this executes a whole Karate feature as a "flow". Note how you can have concurrent flows in the same Gatling simulation.
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ class KarateActor extends Actor {
}
}

class KarateAction(val name: String, val protocol: KarateProtocol, val system: ActorSystem, val statsEngine: StatsEngine, val next: Action) extends ExitableAction {
class KarateAction(val name: String, val callTag: String, val protocol: KarateProtocol, val system: ActorSystem, val statsEngine: StatsEngine, val next: Action) extends ExitableAction {

def getActor(): ActorRef = {
val actorName = new File(name).getName + "-" + protocol.actorCount.incrementAndGet()
Expand Down Expand Up @@ -100,7 +100,7 @@ class KarateAction(val name: String, val protocol: KarateProtocol, val system: A
val asyncNext: Runnable = () => next ! session
val callContext = new CallContext(null, 0, null, -1, false, true, null, asyncSystem, asyncNext, stepInterceptor)

CucumberUtils.callAsync(name, callContext)
CucumberUtils.callAsync(name, callTag, callContext)

}

Expand Down
Loading

0 comments on commit 4944fd7

Please sign in to comment.