karate-demo

Karate Demo

This is a sample Spring Boot web-application that exposes some functionality as web-service end-points. And includes a set of Karate examples that test these services as well as demonstrate various Karate features and best-practices.

Example	Demonstrates
greeting.feature	Simple GET requests and multiple scenarios in a test.
headers.feature	Multiple examples of header management including dynamic setting of headers for each request using a JS file (classpath:headers.js). Also demonstrates handling of cookies, and path / query parameters. There are also examples of how to set up a re-usable *.feature file for per-request secure / auth headers after a sign-in. An OAuth 2 sample is also available.
sign-in.feature	HTML form POST example. Typically you use the response to get an authentication token that can be used to build headers for subsequent requests. This example also demonstrates getting past an end-point protected against CSRF.
cats.feature	Great example of embedded-expressions (or JSON / XML templating). Also shows how to set the Accept header for getting XML from the server.
kittens.feature	Reading a complex payload expected response from a file. You can do the same for request payloads as well.
read-files.feature	The above example reads a file with embedded expressions and this one reads normal JSON and XML for use in a match. Also a good example of using the set and table keywords to build JSON (or XML) payloads from scratch.
graphql.feature	GraphQL example showing how easy it is to prepare queries and deal with the highly dynamic and deeply nested JSON responses - by focusing only on the parts you are interested in
upload.feature	Multi-part file-upload example, as well as comparing the binary content of a download. Also shows how to assert for expected response headers. Plus an example of how to call custom Java code from Karate.
dogs.feature	How to easily use Java interop to make a JDBC / database call from a Karate test. Here is the utility-class code - which depends on Spring JDBC: DbUtils.java. The same approach can be used to mix things like Selenium or WebDriver into Karate.
cats-java.feature	Another example of how to call Java code showing how you can pass JSON data around.
schema.feature	Karate's simpler approach to schema-validation compared with an actual JSON-schema example taken from json-schema.org. If you really want to perform JSON-schema validation, this example shows you how you can easily do so using Java interop and the json-schema-validator, but should you really ?
first.feature	Along with second.feature shows how you can use tags to selectively run tests or groups of tests from the command-line.
soap.feature	Examples for both SOAP 1.1 and 1.2, also showing how you can read request payloads and even expected response XML from files. Note that examples of how to manipulate XML are linked from the main documentation - for example this one: xml.feature
dynamic-params.feature	Multiple examples of data-driven testing including using a Scenario Outline and Examples - so that you can compare approaches. Since the params keyword takes JSON (and keys with null values are ignored), you can easily script different permutations of query parameters. This example also uses a JavaScript function (to simplify a custom assertion), which is defined in a separate file.
call-feature.feature	How you can re-use a sequence of HTTP calls in a *.feature file from other test scripts. This is hugely useful for those common authentication or 'set up' flows that create users, etc. Refer to the main documentation on how you can pass parameters in and get data back from the 'called' script.
call-json-array.feature	This example loads JSON data from a file and uses it to call a *.feature file in a loop. This approach can enable very dynamic data-driven tests, since there are a variety of ways by which you can create the JSON data, for example by calling custom Java code.
call-table.feature	This is a great example of how Karate combines with Cucumber and JsonPath to give you an extremely readable data-driven test. Karate's table keyword is a super-elegant and readable way to create JSON arrays, perfect for setting up all manner of data-driven tests.
call-dynamic-json.feature	Shows how to dynamically create a JSON array and then use it to call a *.feature file in a loop. In this example, the JSON is created using a JavaScript function, but it can very well be the response from an HTTP call, the result of a JsonPath expression or even a List of HashMap-s acquired by calling Java. This test actually calls a second *.feature file in a loop to validate a 'get by id'. Using JsonPath and match each to validate all items within a JSON array is also demonstrated.
call-once.feature	Cucumber has a limitation where Background steps are re-run for every Scenario and even for every Examples row within a Scenario Outline. This is a problem when you have expensive and time-consuming HTTP calls in your 'set-up' routines. Fortunately you have an elegant work-around with Karate's callonce keyword.
polling.feature	If you need to keep polling until the response changes to something you expect, you can achieve this by combining JavaScript functions with calling another *.feature file.
JavaApiTest.java	If you need to call a Karate test from Java code you can do so using the Java API. This is useful in some situations, for example if you want to mix API-calls into a Selenium / WebDriver test.
CatsUiRunner.java	(Experimental) You can use the Karate UI to debug and step-through (even replay) each step of a test. Here is a video that shows the possibilities: link

Configuration and Best Practices

File	Demonstrates
karate-config.js	Shows how the demoBaseUrl property is injected into all the test scripts on startup. Notice how JavaScript allows you to perform simple conditional logic and string manipulation, while still being a 'devops-friendly' plain-text file. It is good practice to set the connectTimeout and readTimeout so that your tests 'fail fast' if servers don't respond. For advanced users - you can even run a 'global' init routine using karate.callSingle().
karate-config-contract.js	When your project gets complex, you can over-ride config per environment (karate.env). Refer to the documentation on environment specific config.
TestBase.java	This is specific to Spring Boot, but this code takes care of starting the embedded app-server and dynamically chooses a free port. The chosen port value is passed to the above config routine via a Java System.setProperty() call.
DemoTest.java	This Java class is strategically placed at the root of the directory structure containing *.feature files. Note how the @CucumberOptions annotation allows you to skip any *.feature files if they have @ignore at the start. The plugin option specifies the reports and formats desired, which can be over-ridden on the command-line or by the maven config described below.
DemoTestParallel.java	Karate has a utility to run tests in parallel and this does not depend on JUnit, TestNG or even Maven. A JUnit XML report file and Cucumber JSON report file would be generated for each feature executed. You can easily configure your CI with the location of these files so that you get proper test-reports after a build. This is the recommended way of running Karate as part of an automated build or CI pipeline. Here, the (optional) third-party cucumber-reporting library is being used (see details below).
pom.xml	Line 131 shows how the maven-surefire-plugin can be configured to point to what is basically your 'test-suite'. You may not even need to do this if you follow the recommended naming conventions and folder structure, and then Maven defaults would work as you would expect. Note that this demo application has many dependencies that you will not need for a typical Karate project.

Gradle

This project contains an example build.gradle that can be used as a reference by Gradle users who want to use the recommended directory structure.

To generate the Gradle wrapper (as is usual for Gradle):

gradle wrapper

After that use the Gradle wrapper to run the tests:

./gradlew test

Example Report

This is optional and if you use the parallel runner as described above, the JUnit XML emitted is sufficient for most CI tools (e.g. Jenkins) to generate test reports and determine whether the build passed or failed. But the advantage of the approach below is that it includes HTTP request and response logs in-line with the report (see video).

Since the maven-cucumber-reporting plugin has an issue where reports will not be generated if the build fails, we recommend that you directly use the cucumber-reporting library programmatically in combination with the Karate parallel runner. Here is how:

Maven Dependency

Add the net.masterthought:cucumber-reporting jar as a dependency in test scope

<dependency>
    <groupId>net.masterthought</groupId>
    <artifactId>cucumber-reporting</artifactId>
    <version>3.8.0</version>
    <scope>test</scope>
</dependency>

Log4j Config File

If you don't already have log4j (v2) in the mix, place this minimal config on the classpath as log4j2.properties (in the same folder as karate-config.js).

log4j.rootLogger = INFO, CONSOLE
log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout

Generate Report

Refer to the code in the demo: DemoTestParallel.java, specifically the generateReport() method.

And here is the output, which goes into target/cucumber-html-reports if you follow the above steps:

This report is recommended especially because the HTTP request and response payloads are embedded. You can even see the results of print statements in-line.

Code Coverage using Jacoco

In the pom.xml, code coverage using Jacoco is also demonstrated. Since this is set-up as a Maven profile, instrumentation and code-coverage reporting would be performed only when you use the coverage profile. Note that code-coverage data (binary) would be saved to this file: target/jacoco.exec.

So to run tests and perform code-coverage:

mvn clean test -Pcoverage

And the HTML reports would be output to target/site/jacoco/index.html.

As this demo example shows - if you are able to start your app-server and run Karate tests in the same JVM process, code-coverage reports for even HTTP integration tests will be very easy to generate. This is even easier with the karate-mock-servlet as you don't even need to boot an app-server.