Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for JavaScript String interpolation in Scenario Names a… #1416

19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4037,6 +4037,25 @@ Scenario Outline: inline json

For another example, see: [`examples.feature`](karate-demo/src/test/java/demo/outline/examples.feature).

If you're looking for more complex ways of naming your scenarios you can use JavaScript String interpolation by wrapping the name of the scenario in backticks.

```cucumber
Scenario Outline: `name is ${name.first} ${name.last} \
and age is ${age}`
* match name.first == "#? _ == 'Bob' || _ == 'Nyan'"
* match name.last == "#? _ == 'Dylan' || _ == 'Cat'"
* match title == karate.info.scenarioName

Examples:
| name! | age | title |
| { "first": "Bob", "last": "Dylan" } | 10 | name is Bob Dylan and age is 10 |
| { "first": "Nyan", "last": "Cat" } | 5 | name is Nyan Cat and age is 5 |
```

String interpolation will support variables in scope and/or Examples (including functions defined globally, but not functions defined in the background), operators and even the Java Interop access and access to the Karate API.

For some more examples check [`test-outline-name-js.feature`](karate-core/src/test/java/com/intuit/karate/core/parser/test-outline-name-js.feature).

### The Karate Way
The limitation of the Cucumber `Scenario Outline:` (seen above) is that the number of rows in the `Examples:` is fixed. But take a look at how Karate can [loop over a `*.feature` file](#data-driven-features) for each object in a JSON array - which gives you dynamic data-driven testing, if you need it. For advanced examples, refer to some of the scenarios within this [demo](karate-demo): [`dynamic-params.feature`](karate-demo/src/test/java/demo/search/dynamic-params.feature#L70).

Expand Down
34 changes: 32 additions & 2 deletions karate-core/src/main/java/com/intuit/karate/StringUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,13 @@
import java.io.BufferedReader;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
Expand Down Expand Up @@ -178,16 +180,44 @@ public static StringUtils.Pair splitByFirstLineFeed(String text) {
String right = "";
if (text != null) {
int pos = text.indexOf('\n');
// use backslash to continue in the same line
String pattern = "(.|\\n)+?[^\\\\\\s*](\\n|$)";
Pattern r = Pattern.compile(pattern);
Matcher m = r.matcher(text);
if (m.find( )) {
pos = m.end();
}

if (pos != -1) {
left = text.substring(0, pos).trim();
right = text.substring(pos).trim();
left = trimBetweenNewLinesWithEscape(text.substring(0, pos)).trim();
right = trimBetweenNewLinesWithEscape(text.substring(pos)).trim();
} else {
left = text.trim();
}
}
return StringUtils.pair(left, right);
}

public static String trimBetweenNewLinesWithEscape(String text) {
AtomicBoolean previousLineHasBackslash = new AtomicBoolean(false);
return Arrays.stream(text.split("\\n")).map(s -> {
if(s.matches(".*\\\\\\s*$")) {
// ends with a backslash
String trimmed = s.trim(); // in Java 11 this should be strip() instead of trim()
previousLineHasBackslash.set(true);
return '\\' == trimmed.charAt(trimmed.length() - 1) ? trimmed.substring(0, trimmed.length() - 1) : trimmed;
} else {
String returnStr = s;
if(previousLineHasBackslash.get()) {
returnStr = returnStr.trim();
}
previousLineHasBackslash.set(false);
return returnStr;
}

}).collect(StringBuilder::new, StringBuilder::append, StringBuilder::append).toString();
}

public static List<String> toStringLines(String text) {
return new BufferedReader(new StringReader(text)).lines().collect(Collectors.toList());
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ public boolean tryAdvance(Consumer<? super ScenarioRuntime> action) {
}
if (currentScenario.isDynamic()) {
if (background == null) {
background = new ScenarioRuntime(featureRuntime, currentScenario);
background = new ScenarioRuntime(featureRuntime, currentScenario, true);
background.run();
if (background.result.isFailed()) { // karate-config.js || background failed
currentScenario = null;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,19 +23,22 @@
*/
package com.intuit.karate.core;

import com.intuit.karate.RuntimeHook;
import com.intuit.karate.ScenarioActions;
import com.intuit.karate.FileUtils;
import com.intuit.karate.KarateException;
import com.intuit.karate.LogAppender;
import com.intuit.karate.Logger;
import com.intuit.karate.RuntimeHook;
import com.intuit.karate.ScenarioActions;
import com.intuit.karate.http.ResourceType;
import com.intuit.karate.shell.FileLogAppender;

import java.io.File;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
*
Expand All @@ -56,12 +59,21 @@ public class ScenarioRuntime implements Runnable {
public final Map<String, Object> magicVariables;
public final boolean selectedForExecution;
public final boolean dryRun;
public final boolean isBackgroundRuntime;

public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario) {
this(featureRuntime, scenario, null);
this(featureRuntime, scenario, null, false);
}

public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario, boolean backgroundRuntime) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need the extra boolean won't it be a property of the ScenarioRuntime

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running Scenario Outlines the Background runs wrapped in a ScenarioRuntime - while when running a scenario the background section is just included in the steps. That background ScenarioRuntime doesn't have a scenario associated with thus I had a nullpointer.

I think there was a way to figure out (like null scenario maybe or something) that I can use instead of the boolean to avoid evaluating the scenario name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try scenario.isDynamic() you can see it used elsewhere in the same file

this(featureRuntime, scenario, null, backgroundRuntime);
}

public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario, ScenarioRuntime background) {
this(featureRuntime, scenario, background, false);
}

public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario, ScenarioRuntime background, boolean backgroundRuntime) {
logger = new Logger();
this.featureRuntime = featureRuntime;
this.caller = featureRuntime.caller;
Expand Down Expand Up @@ -93,6 +105,7 @@ public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario, Scenari
}
dryRun = featureRuntime.suite.dryRun;
selectedForExecution = isSelectedForExecution(featureRuntime, scenario, tags);
isBackgroundRuntime = backgroundRuntime;
}

public boolean isFailed() {
Expand Down Expand Up @@ -349,6 +362,11 @@ public void beforeRun() {
}
ScenarioEngine.set(engine);
engine.init();
if(!this.isBackgroundRuntime) {
// don't evaluate names when running the background section
this.evaluateScenarioName();
this.evaluateScenarioDescription();
}
result.setExecutorName(Thread.currentThread().getName());
result.setStartTime(System.currentTimeMillis() - featureRuntime.suite.startTime);
if (!dryRun) {
Expand Down Expand Up @@ -484,4 +502,23 @@ public String toString() {
return scenario.toString();
}

public void evaluateScenarioName() {
String scenarioName = this.scenario.getName();
boolean wrappedByBackTick = scenarioName != null && scenarioName.length() > 1 && '`' == scenarioName.charAt(0) && '`' == scenarioName.charAt((scenarioName.length() - 1));
if (wrappedByBackTick) {
String evaluatedScenarioName = this.engine.evalJs(scenarioName).getAsString();
this.scenario.setName(evaluatedScenarioName);
}
}

public void evaluateScenarioDescription() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • is fixing the description really necessary (then I have to include this in the JSON when we serialize a test result)
  • is supporting the \ to spill over 2 lines really necessary

I'd like to keep it simple :|

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I meant when I said we need to care about multiple lines is I believe the first line always goes into the name and any extra lines go into the description. if that is the case, I propose we only interpolate the name

for those who want extra / dynamic docs - the new doc keyword is what I'd like users to use, and this can go into any Step

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I thought about 2 lines I was thinking about wrapping lines - different developers might have different views in terms of the size of a line / line wrap. I agree that anything past 2 lines is probably bad practice (and reports won't look pretty at all) but what do you think about still supporting any lines and leaving the "good practice" of keeping the name short and simple to the test scripters?

I'm just thinking that might need to throw an exception if we want to limit to 2 lines and one day someone will want "just" the 3 lines :)

The description was already in the code (saw it in the replace). Are you planning on removing it all together? I agree with not evaluating as the scenario outline should always do the same validations so a static description should suffice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear ya. I'd be okay if this wasn't cucumber, the syntax is biased towards "one line at a time", the only way to span multiple lines is to use the triple-quotes """

so I'd really like to stick to one-line for the name. I also made this commit to verify what I was saying, that the parser already gives you the name and description separate: 77310ac

not planning to remove the description but we have now created a JSON for representing "an executed scenario" - see the toKarateJson() methods in ScenarioResult etc. right now I'm not including the scenario description because it can always be derived from the original feature-file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so yes, good point - if the description is already being included in the replace - we can decide to not do it anymore. and one reason is we have a new way of adding fancy descriptions in-line. the name is important to stand out in reports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it - will remove the usage of \ altogher

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove the option of using string interpolation in the description but for now will leave the replace on the description - can probably be marked as deprecated or something but might be important to keep it for backwards compability

String scenarioDescription = this.scenario.getDescription();
boolean wrappedByBackTick = scenarioDescription != null && scenarioDescription.length() > 1 && '`' == scenarioDescription.charAt(0)
&& '`' == scenarioDescription.charAt((scenarioDescription.length() - 1));
if (wrappedByBackTick) {
String evaluatedScenarioDescription = this.engine.evalJs(scenarioDescription).getAsString();
this.scenario.setDescription(evaluatedScenarioDescription);
}
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,14 @@ void testSplitByFirstLineFeed() {
StringUtils.splitByFirstLineFeed("foo"));
assertEquals(new Pair("foo", "bar"),
StringUtils.splitByFirstLineFeed("foo\nbar"));
assertEquals(new Pair("foo foo2", "bar"),
StringUtils.splitByFirstLineFeed("foo \\\nfoo2\nbar"));
assertEquals(new Pair("foo foo2", "bar test"), // leave the space, it's not an error in the unit test
StringUtils.splitByFirstLineFeed("foo \\\nfoo2\nbar\n test"));
assertEquals(new Pair("foo foo2", "bar test"),
StringUtils.splitByFirstLineFeed("foo \\\nfoo2\nbar \\\n test"));
assertEquals(new Pair("multi line left", "right side can also be multiline and note the concatenated multiline word"),
StringUtils.splitByFirstLineFeed("multi \\\nline \\\n left\nright side \\\n can also be multi\\\nline \\\n and note the concatenated multiline word"));
}

@Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,13 @@ class FeatureParserTest {
static final Logger logger = LoggerFactory.getLogger(FeatureParserTest.class);

static FeatureResult execute(String name) {
return execute(name, null);
}

static FeatureResult execute(String name, String env) {
Feature feature = Feature.read("classpath:com/intuit/karate/core/parser/" + name);
Runner.Builder builder = Runner.builder();
builder.karateEnv(env);
builder.tags("~@ignore");
FeatureRuntime fr = FeatureRuntime.of(new Suite(builder), feature);
fr.run();
Expand Down Expand Up @@ -146,6 +151,12 @@ void testOutlineName() {
match(map.get("title"), "name is Nyan and age is 5");
}

@Test
void testOutlineNameJs() {
FeatureResult result = execute("test-outline-name-js.feature", "unit-test");
assertFalse(result.isFailed());
}

@Test
void testTagsMultiline() {
FeatureResult result = execute("test-tags-multiline.feature");
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
Feature:

Background:
* def sum = function(x,y){ return x + y; }
* def js_data =
"""
[
{
"name": "Bob",
"age": 10,
"title": "name is Bob and age is 10"
},
{
"name": "Nyan",
"age": 5,
"title": "name is Nyan and age is 5"
}
]
"""

* def nested_js_data =
"""
[
{
"name": {
"first": "Bob",
"last": "Dylan"
},
"age": 10,
"title": "name is Bob and age is 10"
},
{
"name": {
"first": "Nyan",
"last": "Cat"
},
"age": 5,
"title": "name is Nyan and age is 5"
}
]
"""

Scenario Outline: `name is ${name} and age is ${age}`
* def name = '<name>'
* match name == "#? _ == 'Bob' || _ == 'Nyan'"
* match title == karate.info.scenarioName

Examples:
| name | age | title |
| Bob | 10 | name is Bob and age is 10 |
| Nyan | 5 | name is Nyan and age is 5 |


Scenario Outline: `name is ${name} and age is ${age}`
* def name = '<name>'
* match name == "#? _ == 'Bob' || _ == 'Nyan'"
* match title == karate.info.scenarioName

Examples:
| js_data |


Scenario Outline: `name is ${name.first} and age is ${age}`
* match name.first == "#? _ == 'Bob' || _ == 'Nyan'"
* match title == karate.info.scenarioName

Examples:
| nested_js_data |


Scenario Outline: `name is ${name.first} ${name.last} \
and age is ${age}`
* match name.first == "#? _ == 'Bob' || _ == 'Nyan'"
* match name.last == "#? _ == 'Dylan' || _ == 'Cat'"
* match title == karate.info.scenarioName

Examples:
| name! | age | title |
| { "first": "Bob", "last": "Dylan" } | 10 | name is Bob Dylan and age is 10 |
| { "first": "Nyan", "last": "Cat" } | 5 | name is Nyan Cat and age is 5 |


# String interpolation allows you to use operators
Scenario: `one plus one equals ${1 + 1}`
* match karate.info.scenarioName == "one plus one equals 2"

# String interpolation allows you to use operators
Scenario: if name is not entirely wrapped in backticks... won't be evaluated `one plus one equals ${1 + 1}`
* match karate.info.scenarioName == "if name is not entirely wrapped in backticks... won't be evaluated `one plus one equals ${1 + 1}`"

# can even access the karate object
Scenario: `scenario execution (env = ${karate.env})`
# the env is set on the unit test in FeatureParserTest.java
* match karate.info.scenarioName == "scenario execution (env = unit-test)"

# functions can also be used, including access to the Java Interop API
Scenario: `math scenario: should return ${java.lang.Math.pow(2, 2)}`
* def powResult = java.lang.Math.pow(2, 2)
* match karate.info.scenarioName == "math scenario: should return " + powResult
* match karate.info.scenarioName == "math scenario: should return 4"

# and if you really really have a need... you can wrap your scenario name with backtick to have a multi-line name
# note that by default any content after the first new line that does not include a backslah will be set as the scenario description
Scenario: `math scenario: should return ${java.lang.Math.pow(2, 2)} \
because 2 is the base and 2 is the exponent \
and 2^2=${java.lang.Math.pow(2, 2)}`
and the next new line will be the description of your scenario... \
which can also be multi-line
* def powResult = java.lang.Math.pow(2, 2)
* match karate.info.scenarioName == "math scenario: should return 4 because 2 is the base and 2 is the exponent and 2^2=4"


# if you don't add the backslash at the end of first line, the second line onwards will be the scenario description so in this case
# there's nothing to evaluate as there's no closing backslash
Scenario: `math scenario: should return ${java.lang.Math.pow(2, 2)}
because 2 is the base and 2 is the exponent
and 2^2=${java.lang.Math.pow(2, 2)}`
* def powResult = java.lang.Math.pow(2, 2)
* match karate.info.scenarioName == "`math scenario: should return ${java.lang.Math.pow(2, 2)}"
Original file line number Diff line number Diff line change
@@ -1,5 +1,22 @@
Feature:

Background:
* def js_data =
"""
[
{
"name": "Bob",
"age": 10,
"title": "name is Bob and age is 10"
},
{
"name": "Nyan",
"age": 5,
"title": "name is Nyan and age is 150"
}
]
"""

Scenario Outline: name is <name> and age is <age>
* def name = '<name>'
* match name == "#? _ == 'Bob' || _ == 'Nyan'"
Expand All @@ -9,3 +26,21 @@ Examples:
| name | age | title |
| Bob | 10 | name is Bob and age is 10 |
| Nyan | 5 | name is Nyan and age is 5 |


Scenario Outline: name is <name> and age is <age>
* def name = '<name>'
* match name == "#? _ == 'Bob' || _ == 'Nyan'"
* def title = karate.info.scenarioName

Examples:
| js_data |

Scenario Outline: name is <name> \
and age is <age>
* def name = '<name>'
* match name == "#? _ == 'Bob' || _ == 'Nyan'"
* def title = karate.info.scenarioName

Examples:
| js_data |