Can't get provider to recognize matchers in pact

[edeandrea]

This could totally be user error as I'm somewhat new to this, but for the life of me I can not figure out how to get the provider to verify a pact without it trying to do exact matching.

I have a consumer which generates this pact:

{
  "consumer": {
    "name": "rest-fights"
  },
  "interactions": [
    {
      "comments": {
        "testname": "io.quarkus.sample.superheroes.fight.client.VillainConsumerContractTests.helloVillains()",
        "text": [

        ]
      },
      "description": "A hello request",
      "key": "2ec6e2e8",
      "pending": false,
      "request": {
        "headers": {
          "Accept": [
            "text/plain"
          ]
        },
        "method": "GET",
        "path": "/api/villains/hello"
      },
      "response": {
        "body": {
          "content": "Hello villains!",
          "contentType": "text/plain",
          "contentTypeHint": "DEFAULT",
          "encoded": false
        },
        "headers": {
          "Content-Type": [
            "text/plain"
          ]
        },
        "matchingRules": {
          "body": {
            "$": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "type"
                }
              ]
            }
          },
          "header": {
            "Content-Type": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "regex",
                  "regex": "text/plain"
                }
              ]
            }
          }
        },
        "status": 200
      },
      "type": "Synchronous/HTTP"
    }
  ],
  "metadata": {
    "pact-jvm": {
      "version": "4.3.14"
    },
    "pactSpecification": {
      "version": "4.0"
    }
  },
  "provider": {
    "name": "rest-villains"
  }
}

It is generated by a JUnit 5 test (au.com.dius.pact.consumer:junit5:4.3.14)

@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(
  providerName = "rest-villains",
  pactVersion = PactSpecVersion.V4,
  hostInterface = "localhost",
  // Make an assumption and hard-code the Pact MockServer to be running on port 8081
  // I don't like it but couldn't figure out any other way
  port = "8081"
)
public class VillainConsumerContractTests {
  @Pact(consumer = "rest-fights")
  public V4Pact helloPact(PactDslWithProvider builder) {
    return builder
      .uponReceiving("A hello request")
        .path(VILLAIN_HELLO_URI)
        .method(HttpMethod.GET)
        .headers(HttpHeaders.ACCEPT, MediaType.TEXT_PLAIN)
      .willRespondWith()
        .matchHeader(HttpHeaders.CONTENT_TYPE, MediaType.TEXT_PLAIN, MediaType.TEXT_PLAIN)
        .status(Status.OK.getStatusCode())
        .body(PactDslRootValue.stringType(DEFAULT_HELLO_RESPONSE))
      .toPact(V4Pact.class);
  }

  @Test
  @PactTestFor(pactMethod = "helloPact")
  void helloVillains() {
    runHelloVillains(); // What this does is unimportant here. It works and generates the pact pasted above
  }
}

The pact is published to pactflow.io.

On the provider side I have this (au.com.dius.pact.provider:junit5:4.3.14):

@Provider("rest-villains")
@PactBroker(url = "https://myname.pactflow.io")
@EnabledIfSystemProperty(named = "pactbroker.auth.token", matches = ".+", disabledReason = "pactbroker.auth.token system property not set")
public class ContractVerificationTests {
  @TestTemplate
  @ExtendWith(PactVerificationInvocationContextProvider.class)
  void pactVerificationTestTemplate(PactVerificationContext context) {
    context.verifyInteraction();
  }

  @BeforeEach
  void beforeEach(PactVerificationContext context) {
    context.setTarget(new HttpTestTarget("localhost", 8081));
  }

  @PactBrokerConsumerVersionSelectors
  public static SelectorBuilder consumerVersionSelectors() {
    return new SelectorBuilder()
      .branch(System.getProperty("pactbroker.consumer.branch", "main"));
  }
}

When I run the verification I'm getting this:

Verifying a pact between rest-fights (1.0) and rest-villains

  Notices:
    1) The pact at https://myname.pactflow.io/pacts/provider/rest-villains/consumer/rest-fights/pact-version/ed93e91f94fe8ea4404bb1c08318b77005f1f432 is being verified because the pact content belongs to the consumer version matching the following criterion:
    * latest version from branch 'main' (1.0)

  [from Pact Broker https://myname.pactflow.io/pacts/provider/rest-villains/consumer/rest-fights/pact-version/ed93e91f94fe8ea4404bb1c08318b77005f1f432/metadata/c1tdW2JdPW1haW4mc1tdW2xdPXRydWUmc1tdW2N2XT0xNg]
  A hello request

  Test Name: io.quarkus.sample.superheroes.fight.client.VillainConsumerContractTests.helloVillains()

  Comments:

08:35:29 DEBUG [io.qu.sa.su.vi.re.VillainResource] (executor-thread-0) Hello Villain Resource
    returns a response which
      has status code 200 (OK)
      includes headers
        "Content-Type" with value "text/plain" (FAILED)
      has a matching body (FAILED)

Failures:

1) Verifying a pact between rest-fights and rest-villains - A hello request includes headers "Content-Type" with value "[text/plain]"

    1.1) header: Expected 'text/plain;charset=UTF-8' to match 'text/plain'

    1.2) body: / Expected body 'Hello villains!' to match 'Hello Villain Resource' using equality but did not match

It looks like its doing exact matching on the header and body, even though the pact has matching rules where

1. The Content-Type header should contain text/plain, not be equal to text/plain
2. The body should match any String and shouldn't be trying to verify an exact String

Am I doing something wrong here or missing something? Or is something not working the way it should?

On the consumer I also tried using PactSpecVersion.V3 instead of PactSpecVersion.V4 but got the same result.

The entire source code of the consumer & provider is available:

• Consumer: https://github.com/edeandrea/quarkus-super-heroes/tree/pact/rest-fights
  • Contract test class is https://github.com/edeandrea/quarkus-super-heroes/blob/pact/rest-fights/src/test/java/io/quarkus/sample/superheroes/fight/client/VillainConsumerContractTests.java
• Provider: https://github.com/edeandrea/quarkus-super-heroes/tree/pact/rest-villains
  • Contract verification test class is https://github.com/edeandrea/quarkus-super-heroes/blob/pact/rest-villains/src/test/java/io/quarkus/sample/superheroes/villain/ContractVerificationTests.java

[edeandrea]

Hi - I know I only created this late last week, but it is somewhat time sensitive.

@holly-cummins and I are doing a talk at Devoxx Belgium in a couple of weeks (https://devoxx.be/talk/?id=6661) and we wanted to use the Quarkus Sample application (https://github.com/quarkusio/quarkus-super-heroes) as a demo for "showing off" Pact as well as its integration with pactflow.io.

This issue is a blocker for that. Neither she nor I can figure out how to get the provider tests to pass using the matchers specified in the Pact generated from the consumers.

Any help would be greatly appreciated!

[rholshausen]

matchHeader sets up a regex to match the header value. With regexes, they need to match the whole string. So setting .matchHeader(HttpHeaders.CONTENT_TYPE, MediaType.TEXT_PLAIN, MediaType.TEXT_PLAIN) won't work. You should try something like .matchHeader(HttpHeaders.CONTENT_TYPE, "text/plain.*", MediaType.TEXT_PLAIN)

However, Pact understands how to match content-type headers, so you don't need to use matchHeader at all. If you specify just the base header value without the attributes, it will ignore the charset attribute fro the provider. However, if you set a specific attribute (say charset=US-ASCII) it will check the charset value matches correctly.

Coming to the body, most of the "type" matchers will only work with structured documents (like JSON and XML). Think about what you are trying to so. You body is set as text/plain, and you want to ensure it is always the same type as a string. But a plain text body can only ever be represented as a string.

The only matcher that works with plain text is the regex one. Remember the regex will need to match the entire body contents. You may also want to consider using a JSON body instead of plain text.

[edeandrea]

Thank you for the comments. I will give it a try tomorrow.

Unfortunately I can't change the APIs. They are what they are.

So for matching a text/plain body my matcher should essentially be .+?

[rholshausen]

That will match everything except the empty string, looking at your example you could also try something like Hello villains.*

[edeandrea]

But then wouldn't that make it so the consumer understood what the provider's test data was? In a real world situation I'm not sure the consumer would know what text to expect other than "some text". The consumer has no control over what the provider might send back (and honestly, this consumer really doesn't care as long as it's something.

I'm trying to be as loose as possible and not have the provider and consumer not know too much about each other, other than the contract.

[rholshausen]

That makes sense, essentially you are stating the body must not be empty.

[edeandrea]

Thank you. I will play with this more in the morning!

[edeandrea]

@rholshausen I did get the body to validate using the regex!

However, Pact understands how to match content-type headers, so you don't need to use matchHeader at all. If you specify just the base header value without the attributes, it will ignore the charset attribute fro the provider. However, if you set a specific attribute (say charset=US-ASCII) it will check the charset value matches correctly.

I'm not sure I quite understand? If I don't use matchHeader then no matching seems to take place on the header.

In my consumer test I do

@Pact(consumer = "rest-fights")
  public V4Pact helloPact(PactDslWithProvider builder) {
    return builder
      .uponReceiving("A hello request")
        .path(VILLAIN_HELLO_URI)
        .method(HttpMethod.GET)
        .headers(HttpHeaders.ACCEPT, MediaType.TEXT_PLAIN)
      .willRespondWith()
        .headers(Map.of(HttpHeaders.CONTENT_TYPE, MediaType.TEXT_PLAIN))
        .status(Status.OK.getStatusCode())
        .body(PactDslRootValue.stringMatcher(".+", DEFAULT_HELLO_RESPONSE))
      .toPact(V4Pact.class);
  }

which generates this Pact

{
  "consumer": {
    "name": "rest-fights"
  },
  "interactions": [
    {
      "comments": {
        "testname": "io.quarkus.sample.superheroes.fight.client.VillainConsumerContractTests.helloVillains()",
        "text": [

        ]
      },
      "description": "A hello request",
      "key": "2ec6e2e8",
      "pending": false,
      "request": {
        "headers": {
          "Accept": [
            "text/plain"
          ]
        },
        "method": "GET",
        "path": "/api/villains/hello"
      },
      "response": {
        "body": {
          "content": "Hello villains!",
          "contentType": "text/plain",
          "contentTypeHint": "DEFAULT",
          "encoded": false
        },
        "headers": {
          "Content-Type": [
            "text/plain"
          ]
        },
        "matchingRules": {
          "body": {
            "$": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "regex",
                  "regex": ".+"
                }
              ]
            }
          }
        },
        "status": 200
      },
      "type": "Synchronous/HTTP"
    }
  ],
  "metadata": {
    "pact-jvm": {
      "version": "4.3.14"
    },
    "pactSpecification": {
      "version": "4.0"
    }
  },
  "provider": {
    "name": "rest-villains"
  }
}

When I verify on the provider I get

  [from Pact Broker https://quarkus-super-heroes.pactflow.io/pacts/provider/rest-villains/consumer/rest-fights/pact-version/e7b9f3ef4a9a6651d81db46e546d2b19902a75d5/metadata/c1tdW2JdPW1haW4mc1tdW2xdPXRydWUmc1tdW2N2XT0xNg]
  A hello request

  Test Name: io.quarkus.sample.superheroes.fight.client.VillainConsumerContractTests.helloVillains()

  Comments:

07:48:19 DEBUG [io.qu.sa.su.vi.re.VillainResource] (executor-thread-0) Hello Villain Resource
    returns a response which
      has status code 200 (OK)
      has a matching body (OK)

It doesn't seem to do any kind of verification on the Content-Type header.

Or do you mean to say that if the Content-Type header is not text/plain (or any derivative of it), then the verification test will automatically fail?

[edeandrea]

I think i've moved on past the "hello world" one to something a little more complicated.

I have a GET endpoint where the provider returns a JSON payload and a 200 in some conditions but a 404 in other conditions.

I've gotten the provider to verify the 200 condition fine. For the 404 condition, I used a state in the Pact. In the provider test I am doing this:

@State("No random villain found")
public void clearData() {

}

It doesn't seem, though, that I have access to any of the beans in my application when that method fires. I tried to use mocking and I also tried to inject my application's datasource as a CDI bean and that doesn't seem to work either. Any class-level attribute that I've injected into my test class seems to be null. I'm not sure how I can alter the state of the application without being able to interact with some of the beans in the application.

See https://github.com/edeandrea/quarkus-super-heroes/blob/pact/rest-villains/src/test/java/io/quarkus/sample/superheroes/villain/ContractVerificationTests.java#L55-L59

When I go to run the tests I end up with a NullPointerException because the bean injection is null at the time the @State method runs.

[edeandrea]

I was actually able to get this work by doing some stuff in the @BeforeEach method. At that point the CDI context is fully available.

@QuarkusTest
@Provider("rest-villains")
@PactBroker(url = "https://quarkus-super-heroes.pactflow.io")
@EnabledIfSystemProperty(named = "pactbroker.auth.token", matches = ".+", disabledReason = "pactbroker.auth.token system property not set")
public class ContractVerificationTests {
  @ConfigProperty(name = "quarkus.http.test-port")
  int quarkusPort;

  @InjectSpy
  VillainService villainService;

  @TestTemplate
  @ExtendWith(PactVerificationInvocationContextProvider.class)
  void pactVerificationTestTemplate(PactVerificationContext context) {
    context.verifyInteraction();
  }

  @BeforeEach
  void beforeEach(PactVerificationContext context) {
    context.setTarget(new HttpTestTarget("localhost", this.quarkusPort));

    var isNoRandomVillainFoundState = Optional.ofNullable(context.getInteraction().getProviderStates())
      .orElseGet(List::of)
      .stream()
      .filter(state -> "No random villain found".equals(state.getName()))
      .count() > 0;

    if (isNoRandomVillainFoundState) {
      when(this.villainService.findRandomVillain())
        .thenReturn(Optional.empty());
    }
  }

  @PactBrokerConsumerVersionSelectors
  public static SelectorBuilder consumerVersionSelectors() {
    return new SelectorBuilder()
      .branch(System.getProperty("pactbroker.consumer.branch", "main"));
  }

  @State("No random villain found")
  public void clearData() {
    // Already handled in beforeEach
  }
}