FMIMOBSTER

FMIMOBSTER

FMIMOBSTER (Functional Mockup Interface Model Based Tester) is a tool that applies the principles of model based testing to improve the conformance of Functional Mockup Units to the Functional Mockup Interface Standard.

This is a command line application that tests an FMU. It uses a slightly modified version of the tool modbat to generate random walks in a graph that represents all possible interactions with an FMU.

Such graph looks like the following.

Each edge refers to a method that is implemented in a Scala class.

The graph is enriched with state machines from other files. For example:

You can see the files used to created the state machine in [graphs].

If, during a test, an assertion fails, or an uncaught exception happens, the tool produces a error trace and a seed, that allows for the reproducibility of the test.

Installing and Running the App

Make sure to have Java 11 installed. See this issue.

Download the app jar and example from the releases.

Run the following, for the list of commands:

java -jar FMIMOBSTER-assembly-X.Y.jar --help

where X.Y denotes the app's version.

Running an Example and General Workflow

The release comes with an FMU (courtesy of the INTO-CPS Association). To run 1000 tests on it, type:

java -D"org.slf4j.simpleLogger.defaultLogLevel=trace"  -jar .\FMIMOBSTER-assembly-X.Y.jar mbt -d threewatertank1.fmu -n 1000 -l 10 -o "results"

where X.Y denotes the app's version.

This will produce a results folder with the logs of each individual test. The log file of each test is named with the seed that was used for the test. You can use the seed to reproduce the test.

For instance, if a test produced the files 1a0379c62d09e045.err and 1a0379c62d09e045.log, you can reproduce the same test using the command:

java -D"org.slf4j.simpleLogger.defaultLogLevel=trace"  -jar .\FMIMOBSTER-assembly-X.Y.jar mbt -d threewatertank1.fmu -s 1a0379c62d09e045 -o "results"

In general, to run the model based testing on an FMU in path path/to/fmu.fmu, the following command is used:

java -D"org.slf4j.simpleLogger.defaultLogLevel=trace"  -jar .\FMIMOBSTER-assembly-X.Y.jar mbt -d path/to/fmu.fmu -n 10 -l 10 -o "results"

where

-n is the number of random walks to produce
-l is the maximum number of self-loops executed
-o is the results folder
-D"org.slf4j.simpleLogger.defaultLogLevel=trace" sets the log level to trace
X.Y denotes the app's version

While the app supports running a command in a directory of FMUs, it is advised to run the app for each FMU individually, and in a separate process. This is because some FMUs might crash the java process, which means the other FMUs did not get tested.

Unfortunately, the app does not spawn separate processes for each walk.

Setup Development Environment

This is a simple sbt project.

See build.properties and build.sbt for sbt and scala versions.

The following will produce the jar of the app.

sbt assembly

Building and Running App in Docker Containner

1. Install Docker
2. Build image: docker build -t fmimobster:1.0 .
3. Open interactive shell: docker run --rm --interactive --tty fmimobster:1.0 bash
4. Now build and run tool within the container. Follow the instructions above.
5. Optional: remove the image built: docker image rm fmimobster:1.0

Release Procedure

• Update version (search for current version to find all places where it needs to be updated. Usually in the FMUAnalyzerApp and build.sbt)
• Run the build.py script using python 3.9.

Acknowledgements

We are thankful to:

• modbat for the test case generation
• INTO-CPS Association for the FMI library.
• FWO for financing part of the development of this tool.
• https://acronymify.com for generating the acronym of this tool.