Tutorial
Welcome! This tutorial was designed those who are completely new to the field of automata learning and Model-Based Testing and prefer a quick guided tour through the workings of HermieLab and see what it has to offer.
In this tutorial, we will create a behavioral model of a simple music player. If you haven't got any experience with automata learning, please first read this short introduction. At the end of this tutorial, you'll understand the HermieLab and will be able to use it effectively in your projects.
Download the Music Box Example.
During this tutorial, we're going to learn a behavioral model of a simple MVC-style music player. It has 3 Buttons, a Song button which when pressed starts a song, a Rhythm button which when pressed starts a rhythm and a Quit button which stops what is currently playing. A rhythm and a song can be played at the same time.
Credits of this example go to Mariëlle Stoelinga at the University of Twente, she introduced this example in the course Testing Techniques.
As a model, we want to capture which actions lead to which state of the music box. Given a certain combination of button presses, for example Song, Song, Rhythmn we want to know in which state the Music Box is.
To generate a learner setup which can learn this model, we need to provide it with 5 critical pieces of information. We need to indicate:
- which kind model we want it to learn (DFA/Mealy Machine),
- how the Learner can start the program,
- how the Learner can reset the program,
- which methods the learner should use as inputs of the model,
- how we want to measure the effects of the method (output of the method, what state the program or certain variables are in or any other criteria)
We will provide this information to the learning setup and it's different components through annotations. Let's start!
First, we need to indicate what kind of model we want to learn and how the Learner can start the program. We can do this with the Start Annotation. Annotate the start(Stage) method in Main with @Start(automaton = "MealyMachine"). Make sure to put the annotation above the @Override annotation. HermieLab annotations always need to be on top of other annotations.
@Start(automaton = "MealyMachine")
@Override
public void start(Stage primaryStage) throws Exception {
// code
}Now we will indicate how the Learner can reset this program. In this demo example we have included a reset() method for convenience. Annotate the reset() method in Controller with @PostQuery(). This tells the Learner that this method should be executed after every query.
@PostQuery()
public void reset(){
// code
}Often applications don't have such a handy function and some parts have to be reset manually. For more information how to do this, please take a look at the adapter documentation //TODO Link.
Lastly, we will indicate which methods the learner should use as inputs of the model and how we want to measure the effects of the method.
Whenever a button is pressed in a JavaFX application, a pressed event is generated and it will be captured by the event handlers of these buttons. Therefore, we will use the methods which are called on button pushes as input symbols. Annotate the event handler methods in Controller with the @FunctionSymbol annotation.
@FunctionSymbol(symbolID = "rhythmPushed", outputID = "myState")
public void rhythmPushed(ActionEvent actionEvent) {
//code
}
@FunctionSymbol(symbolID = "songPushed", outputID = "myState")
public void songPushed(ActionEvent actionEvent) {
//code
}
@FunctionSymbol(symbolID = "quitPushed", outputID = "myState")
public void quitPushed(ActionEvent actionEvent) {
//code
}Note how we have set the outputID to "myState". This is because we don't want to observe the output of this function but instead observe the state of the Music Box. OutputID tells the learner which method to call whenever this function has been executed to observe the state of the program. Annotate the getState() function in the Controller.
@OutputFunction(ID = "myState")
public String getState() {
//code
}Now the learner knows that it should call this method to observe the state of the program. If we would've left the default value of outputID or used "this" as outputID we would've taken the output of the methods as the output of the symbol.
Congratulations, we have now provided all information our learning setup needs to successfully learn a model of our program. We can start generating the setup!
To generate the setup, we have to perform 2 steps. First, we have to generate a settings file containing all the information necessary to generate the setup. Secondly we have to generate the components from this settings file. This can be done using annotation Processors.
First, add the hermielab.processor.AnnotationsProcessor to your annotations processors under build > plugins > plugin > configuration > annotationProcessors in the POM in your compile plugin.
<plugin>
[...]
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
<showWarnings>true</showWarnings>
<annotationProcessors>
<annotationProcessor>hermielab.processor.AnnotationsProcessor</annotationProcessor>
</annotationProcessors>
</configuration>
[...]
</plugin>Run the following maven command: mvn clean compile -X. A settings file called hermielabsettings.xml will be generated. This settings file contains all information necessary for the next annotation processor to generate all the components that make up the learning setup.
Swap out the first annotation processor with the other annotation processor:
hermielab.processor.BasicLearningSetupProcessor. Rerun mvn clean compile -X. Afterward, a package named 'hermielab' will appear in your target > generated-sources > annotations folder. This contains all the components necessary to create and run a learning experiment together with a small demo experiment.
Create a new package called experiment and move the ApplicationAdapter, ApplicationInjector and the 3 generated xml files from the hermielab package to this new package. This is to prevent the generated code being removed after you recompile the project.
Additionally, copy and paste this template experiment into the folder as well
Experiment.java
```java package experiment;import experiment.hermielab.ApplicationAdapter; import experiment.hermielab.ApplicationInjector; import hermielab.core.adapter.Adapter; import hermielab.core.instancemanager.InstanceManager; import hermielab.core.learningsetup.BasicMealyMachineLearningSetup; import hermielab.core.utils.DotWriter; import javafx.application.Application; import javafx.stage.Stage; import net.automatalib.automata.transout.MealyMachine;
import java.nio.file.Paths; import java.util.Random;
public class Experiment extends Application {
@Override
public void start(Stage primaryStage) throws Exception {
ApplicationInjector injector = new ApplicationInjector();
injector.inject();
Adapter adapter = new ApplicationAdapter(InstanceManager.getInstance());
BasicMealyMachineLearningSetup setup = new BasicMealyMachineLearningSetup(adapter,
Paths.get("src","main","java","experiment"),
"experiment1",
1,
15,
1000,
new Random(42));
setup.initializeOracles();
MealyMachine result = setup.executeExperiment();
System.out.println(DotWriter.toDot(setup.getAlphabet(),result));
}
}
</details>
A small explanation about the generated classes and components:
* Experiment is a dummy experiment. It will start a learning experiment with some pre-determined variables and print out the learned model to the console. The model will be printed in DOT format, to visualize DOT files, one can use tools such as GraphViz, JUNG or this [online viewer](http://www.webgraphviz.com/). For more information about experiments, please refer to //TODO
* ApplicationInjector is a component which makes sure all classes which are of interested to the experiment can be accessed at any point in time. For more information regarding Injectors, please refer to //TODO.
* The Adapter is a component which allows the Learner to communicate with our SUT. For more information about the adapter, please refer to //TODO.
* The checkers file contains information necessary for the checker. To see an example where the checker is needed, please try out //TODO.
* The experiments file contains all information regarding the experiments. Namely, it contains the experiment ID, which kind of automaton we're going to learn, which oracles are going to be used and the experiments input alphabet.
* The mappings file contains information on how to map abstract symbols to concrete symbols and vice versa.
Now we're almost ready to perform the learning experiment, they're just a few manual tweaks necessary before we can start. Often the learning experiment requires a few manual additions before an experiment can be started.The `quitpushed()`, `rhythmPushed()` and `songPushed()` functions are lacking arguments. This is because in step 4 we didn't specify any parameters for these functions. Since ActionEvent argument doesn't affect the workings of the function at all, we can just use null as a parameter for now.
```java
public void quitPushed() {
((Controller) instance.getClassInstance(Controller.class)).quitPushed(null);
}
public void rhythmPushed() {
((Controller) instance.getClassInstance(Controller.class)).rhythmPushed(null);
}
public void songPushed() {
((Controller) instance.getClassInstance(Controller.class)).songPushed(null);
}
Furthermore, Before we can actually invoke start() on Main at least one instance of the Main class has to be instantiated so we can retrieve it with the instance.getClassInstance() method. Add new Main() before this.start0() in the start() method of the Adapter. Lastly, add a new Stage() parameter to the start method and catch the exceptions it might throw.
@Override
public void start() {
new Main();
this.start0();
}
public void start0() {
try {
((Main) instance.getClassInstance(Main.class)).start(new Stage());
} catch (Exception e) {
e.printStackTrace();
}
}Congratulations, you now have a working learning setup! Comment out the and elements from your POM, run the experiment and observe the learned model. You can copy and paste the learned model into GraphViz to visualize it.
TBA
TBA