Testing Camel K integrations

[nicolaferraro]

An attendee at a recent conf asked for a way to test Camel K integrations.

Tests that should be done on a Camel K integration are not unit tests, they should involve external systems as well. Maybe they will run on a special environment where external services are mocked, but we should define a way to declare testing together with the integration.

I'm thinking to the following scenario:

• user writes the integration file
• user writes the test file (same name with -test is recognized by default)
• user runs kamel test integration.groovy
• integration is deployed together with test code
• test is executed, results are collected and appended at the end of the stream

Questions are:

• can it work?
• is camel able to externalize test code completely from integration code?
• which test framework?
• test code is not a DSL script... do we need some integration with IDE?

@davsclaus, @oscerd, @lburgazzoli, @astefanutti, @valdar, @jamesnetherton

[heiko-braun]

@christophd FYI

[bouskaJ]

@davsclaus, @oscerd, @lburgazzoli, @astefanutti, @valdar, @jamesnetherton any ideas/suggestions?

[christophd]

I have started to give some thought to it.

I really like the idea of test code to be part of the deployment, because then the test framework is able to interact with other services as part of the K8s platform. This is especially useful when the test code itself needs to provide a mocked service as part of the test. This mocked service could be provided as part of K8s as any other service.

can it work?

I once put a little POC together (as part of the test framework Citrus https://github.com/citrusframework/citrus-remote-demo) where the integration tests (e.g. based on JUnit and Citrus) get deployed to Docker/K8s as part of a small web application. The web application (built as fat jar) provided a REST API to execute tests and to get its reports. The tests can also auto start upon deployment.

The POC also included a Maven plugin to create the test web application, execute the tests as part of the Maven build lifecycle and collect the reports.

is camel able to externalize test code completely from integration code?

I think the answer is yes. You loose the ability to invoke the integration via direct endpoint as well as using the mock endpoint though as you would probably do with unit tests. But in general the integration should be invoked either via timer or some accessible endpoint I believe. All outbound calls to 3rd party services can be mocked with frameworks like Citrus.

test code is not a DSL script... do we need some integration with IDE?

I can imagine the test code to be a Camel DSL, too. That would be most comfortable for the user because it is not yet another DSL (e.g. Citrus) to learn. Integration into the IDE is quite important in my eyes. You get a lot of integration with tools and IDE for free when being compliant with frameworks like JUnit. JUnit5 provides a brilliant way to incorporate own frameworks and extensions.

which test framework?

I think Citrus (https://github.com/citrusframework/citrus) can help to mock messaging transports and verify message content with Json, XML, plaintext validators. As already mentioned JUnit5 is a wonderful base framework with wide spread tools and IDE support. In general BDD (Behavior Driven Development) principles can help as testers and managers love it

[nicolaferraro]

After some brainstorming here and there I'd proceed with a POC about that.

We want to remain without project structure for camel k integrations, to keep everything simple and portable. For this reason, testing frameworks requiring a structure and complex dependencies are not well suited.
A possible format for test files is the Gherkin language from Cucumber. It requires a simple .feature file that can contain BDD statements.

We want to focus on integration testing, so a framework like Citrus is well suited for this and Citrus already supports many kind of Gherking statements. We'll add some of ours.

E.g. taken from Citrus doc:

Feature: Voting Http REST API

  Background:
    Given URL: http://localhost:8080/rest/services
    Given variables
      | id      | citrus:randomUUID()  |
      | title   | Do you like Mondays? |
      | options | [ { "name": "yes", "votes": 0 }, { "name": "no", "votes": 0 } ] |
      | report  | true                 |

  Scenario: Clear voting list
    When send DELETE /voting
    Then receive status 200 OK

  Scenario: Get empty voting list
    Given Accept: application/json
    When send GET /voting
    Then Response: []
    And receive status 200 OK

  Scenario: Create voting
    Given Request:
    """
    {
      "id": "${id}",
      "title": "${title}",
      "options": ${options},
      "report": ${report}
    }
    """
    And Content-Type: application/json
    When send POST /voting
    Then receive status 200 OK

  Scenario: Get voting list
    When send GET /voting
    Then validate $.size() is 1
    Then validate $..title is ${title}
    Then validate $..report is ${report}
    And receive status 200 OK

Tests can be very simple or also complex with this approach. Although it won't reach the complexity allowed in a Java test class.

We can create a "IntegrationTest" custom resource to drive the test. Tests will be still using Citrus under the hood and they'll be driven by the operator. This way the test is independent on any CLI but we can still obtain the state of the tests from CLI. Having them as resources also help integrating tests with a CI.

Gherkin will be the first "format" we support, but we can add others later.

[lburgazzoli]

We could then report test results through conditions

[nicolaferraro]

All this ended up in YAKS