Testing state machines

itemis CREATE supports you with testing of statecharts and generated state machines. Learn more about test-driven statechart modelling and how to use SCTUnit to develop your statechart model step by step using a test-driven approach.

In test-driven development, before writing any code the developer creates a unit test that checks the to-be-established implementation for correctness. Only after that, the developer codes the implementation and executes the test. If the test fails, the developer has to fix the implementation – perhaps repeatedly – until the test succeeds.

Tests are not written for the complete software product in advance, but rather on a per-feature basis. That is, the developer writes tests only for the one single feature he is going to implement next. Having successfully finished that implementation, the developer should refactor his code and his tests. The most recent additions might have opened opportunities to restructure one or the other, thus reducing complexity and/or adding readability. Refactoring must never change the semantics of the code. To ensure that, tests come in handy and should be run after each refactoring step.

Test-driven statechart modelling

In model-driven software development (MDSD), specific parts of the software are created by a code generator instead of being written manually. This also pertains to code generated from statechart models.

The nice thing about generated code is that we can safely assume it to be correct – at least as long as the code generator developers have tested their product thoroughly enough. However, what about the model? While the generated code might be correct, this won’t help at all if the underlying model is broken in the first place.

SCTUnit is a scripting and testing framework for writing unit tests for statechart models. You can validate the behaviour of your statechart by running these tests. Each test, written in the SCTUnit language, consists of a well-defined sequence of instructions. When running a test, these instructions are applied to a state machine under control and possibly changes that state machine in a specific way. You put down the expected effects of these changes as assertions. An assertion is used to check whether a specific condition is fulfilled, for example certain states being active or not, variables having specific values, operations having been executed, etc. SCTUnit allows test-driven development of statechart models on the semantic level.

The subsequent sections will first introduce SCTUnit by example and then explain the SCTUnit language in full.

SCTUnit by example

This section introduces SCTUnit by a simple statechart example that models a light switch. We will develop this statechart step by step using a test-driven approach. That is, for each single development step, we will first write a test, then change our statechart model, then run all the tests we created so far until they succeed. After that, we will proceed with the next development step.

Light switch requirements

Before implementing any tests or models, let’s write down the requirements for our light switch statechart model:

1. When the state machine starts, the model is in the Off state (initially the light is off).
2. If the Off state is active and the operate event occurs (a user operates the light switch), the state machine changes from the Off state to the On state (the light is switched on).
3. If the On state is active and the operate event occurs (a user operates the light switch), the state machine changes from the On state to the Off state (the light is switched off).

Specifying the initial behaviour as a test

Now we need an Eclipse project to host our statechart model. So let’s create one, naming it light_switch. Within the project, we create two folders, model and test, for – you guessed it – the model and the tests, respectively.

In the test directory, proceed as follows to create a SCTUnit test file:

1. In the project explorer view, right-click on the test folder. The context menu opens.
2. In the context menu, select New → Other…. The New wizard opens.
3. In the wizard, select itemis CREATE → SCTUnit test case.
4. Click Next >. The wizard switches to the New SCTUnit Test Class page.
5. In the Filename text field, specify the name of the file to contain the SCTUnit tests, say, light_switch.sctunit. The filename extension must always be .sctunit.
6. Click Finish.
7. If a dialog appears asking whether you want to add the Xtext nature to the project, please click Yes.

As a result, the empty file light_switch.sctunit is created in the test folder and is opened in an editor.

In the spirit of test-driven development, create a test for the light switch model’s initial behaviour by writing the following in the light_switch.sctunit file:

testclass light_switch_tests for statechart light_switch {

    @Test
    operation isStateOffActiveAfterInit () {
        enter
        assert active (light_switch.main_region.Off)
    }

}

Let’s walk through this example line by line and see what all of this means.

• testclass light_switch_tests for statechart light_switch {
  • The keyword testclass introduces a test class, a collection of tests. In this example, it is named light_switch_tests.
  • A test class always refers to a statechart that is under control and is to be tested. This relationship is established by the for statechart clause. The keywords for statechart are followed by the name of the statechart, here: light_switch. However, since there is no light_switch statechart yet, an error marker is attached to the statechart name, as shown in the following screenshot:
    
  • The body of the test class is bracketed in braces, i.e., between { and }.
• @Test
operation isStateOffActiveAfterInit () {
  • Each test is an operation that is annotated by @Test. This example defines the operation isStateOffActiveAfterInit. It has no parameters, which is indicated by (). As we will see later, an operation does not necessarily have to be a test, but can be a mere subroutine. Such operations can have parameters. The operation’s body is bracketed in braces.
• enter
  • The enter statement initializes and starts the state machine; it “enters” it. As a consequence, the state machine activates the state that is pointed to by the initial state.
• assert active (light_switch.main_region.Off)
  • The assert statement expects that a specified condition is fulfilled. Using the built-in active function, our example’s assertion expects that the state Off is active. The state is addressed in a fully-qualified manner, i.e., statechart. region. state. However, since we don’t have a statechart model yet and less than ever any state, the Off state in the assertion is flagged as an error.

Implementing a minimal light switch model

Since the test has error markers, we can say that it fails even without being run. So let’s fix that with minimal effort and create a model that is called light_switch and has a state Off. Also, change the annotation EventDriven to CycleBased(200). Figure "First step of creating a light switch model" shows the resulting model.

First step of creating a light switch model

Running the test

Figure "First step of creating a light switch model" also shows that the error markers of the test class have been cleared, because our minimal statechart model has the name that is specified in the test and it also contains the Off state.

Now we can execute the test class. In order to do so, proceed as follows:

• Right-click on the light_switch.sctunit file in the test directory. The context menu opens.
• In the context menu, select Run As → SCTUnit
  
• The test in the test class is executed.
• The results are displayed in the JUnit tab. A green bar is shown, which means our model fulfills the expectations expressed in the test class.
  

Adding more tests

Writing a test for the second requirement

The model by now implements the first requirement specified in section "Light switch requirements".

So let’s take care of the second requirement, formulate it as a SCTUnit test, and add it to the test class.

The whole test class now looks like this:

testclass light_switch_tests for statechart light_switch {

    @Test
    operation isStateOffActiveAfterInit () {
        enter
        assert active (light_switch.main_region.Off)
    }

    @Test
    operation isStateOnActiveAfterOperateInOffState () {
        enter
        raise operate
        proceed 1 cycle
        assert active (light_switch.main_region.On)
    }
}

The raise operate instruction raises the operate event. However, aside from this event having been raised, nothing will happen yet. The next instruction will change that.

The proceed 1 cycle instruction orders the state machine to execute one run-to-completion step (RTC). During this RTC, the state machine can react to the raised operate event.

Processing the operate event should cause the On state to become active. This is checked by the following assertion:
assert active (light_switch.main_region.On)

To make the test class syntactically correct, the statechart model must be amended by the operate event and by the On state.

Detecting a semantically wrong implementation

However, in order to demonstrate what happens if an SCTUnit test fails, let’s have a look at a wrong implementation, as shown in figure "Erroneous light switch model". Here the operate event does not trigger a transition from Off to On but from Off back to Off.

Erroneous light switch model

The test operation isStateOnActiveAfterOperateInOffState should detect this semantic error and alert us. Running the amended test class as an SCTUnit again indeed leads to the following result:

The large bar is red now instead of green, meaning that at least one test failed. Below the bar the test class and its tests are listed and it is indicated which tests succeeded and which tests failed. We can see that our isStateOffActiveAfterInit test still succeeds; so while we didn’t correctly implement the second requirement, we at least didn’t break the first one.

Fixing the model as follows turns the test green:

Calling operations

Okay, so let’s write a test for the third and final requirement:

    @Test
    operation isStateOffActiveAfterOperateInOnState () {
        isStateOnActiveAfterOperateInOffState
        raise operate
        proceed 1 cycle
        assert active (light_switch.main_region.Off)
    }

A prerequisite for dealing with an operate event in the On state is to get into that state in the first place. However, we have implemented the necessary actions before, namely in the test isStateOnActiveAfterOperateInOffState. The great thing is that we can just use that test by calling it as subroutine. This is like calling a function or method in any other programming language.

We know that after executing isStateOnActiveAfterOperateInOffState the On state is active – we even assert that in the called test operation. So now we just have to raise operate again, cycle the statechart once and check whether we are in the Off state after that.

Looping

By going from Off to On and back to Off, we have ensured now that the light switch model behaves as specified. However, we might want to check whether this is still the case if we go through this cycle a number of times, say twice or 20,000 times. To support scenarios like this, the SCTUnit language provides a couple of advanced features, see section The SCTUnit language. Here we use the while loop to iterate over the off-on-off cycle a couple of times:

    @Test
    operation isStateOffActiveAfter10Cycles () {
        enter
        var i: integer = 0
        while (i < 10) {
            raise operate
            proceed 1 cycle
            assert active (light_switch.main_region.On)
            raise operate
            proceed 1 cycle
            assert active (light_switch.main_region.Off)
            i = i + 1
        }
    }

Test Coverage

Test coverage is a metric that describes to which extent your code is covered by your tests. Translated to statechart modelling, a test coverage describes which parts of your state machine were activated during test execution. This information allows to identify missing tests and thereby to increase test quality.

Whenever a test set is executed, a test coverage is computed on the fly. The test coverage metrics can be examined in the Coverage View. The coverage value is given in percent for each model element and is defined as follows:

• Transition coverage is 100% if the transition is executed at least once, and 0% otherwise.
• A state is covered if it is entered and all of its outgoing transitions are covered.
• Region coverage is the weighted average coverage of all contained states.
• Statechart coverage is the weighted average coverage of its top level regions.

In the following example you can see that state On has a coverage of 50%. Although it is entered by the tests, its outgoing transition On -> Off is not covered.

The coverage view contains the following toolbar buttons:

• : Shows / hides fully covered elements in the tree
• : Clears the view and removes the coverage highlighting in the model

The coverage highlighting feature transports the coverage values directly onto the statechart model by coloring all elements in the colors:

• green (fully covered),
• yellow (not fully covered), and
• red (not covered at all).

You can enable the highlighting by selecting an arbitrary element in the coverage view.

With the latest 5.2.0 update, we’ve introduced a useful feature: coverage files that are now generated and saved on your computer. These have the extension “.cov” and can be saved alongside the corresponding SCTUnit file in your workspace. This allows you to easily compare your current coverage results with past ones to see how test coverage is improving over time.

We’ve also made it convenient for you to set your SCTUnit Coverage preferences. Just head over to the SCTUnit “Run Configurations” and look for the new “Coverage” tab right next to the “Common” tab. This lets you adjust coverage settings according to your preferences.

Switching between SCTUnit Editors will now automatically load the most recent previously calculated coverage. This avoids the need to recalculate any coverage information.

HTML Coverage Report

Also, it is now possible to export a HTMLreport of your test coverage. To do this, click the save icon once your coverage has been calculated.

A dialog will open allowing you to save the .zip file in the location of your choice. The HTML file contained can then be opened in your browser to view the report. The report contains all the information about the coverage of your tests as well as a highlighted image of each statechart covered by your SCTUnit tests.

The SCTUnit language

While section "SCTUnit by example" gave an overview of test classes and operation, the section at hand provides you with a complete description of the SCTUnit language. Since the SCTUnit language is an extension of the statechart language, this section will focus on language elements that are specific to the SCTUnit language. Please see section "The statechart language" for everything else.

An important extension of the SCTUnit language over the statechart language is that you can write your own operations (subroutines) and control statements, like if and while . They make it possible to “script” complex procedures, for example by raising different events under different conditions or by looping over a sequence of statements multiple times.

Please remember that you can always hit [Ctrl]+[Space] when editing SCTUnit language files within itemis CREATE. The editor will show you all possible input choices that are valid at your cursor location.

Test classes

A test class contains one or more operations or test cases. It is contained in a file with a .sctunit filename extension.

testclass NameOfTheTestClass for statechart NameOfTheStatechart {

    const pi : real = 3.141592654
    var sum : integer = 0

    @Test
    operation isFeatureAvailable () {
        /* … */
    }
}

The header of a test class consists of the keyword testclass, followed by the name of the test class, followed by the keywords for statechart, followed by the name of the statechart this test class relates to.

The test class imports the statechart, so all variables, operations, events, etc. that are defined in an interface in the statechart’s definition section, as well as the statechart’s states, are available in and accessible by the test class.

Please note: Entities defined in a statechart’s internal scope are not visible from the outside, including test classes controlling the statechart.

The body of a test class is enclosed in braces ({ … }). It consists of an optional part with variable and constant definitions, followed by at least one operation.

Figure "Test class grammar" summarizes the structure of a test class:

Test class grammar

Test classes are namespace-aware: Using the package statement , you can insert a test class into a specific namespace.

Test classes can be grouped into a test suite.

Operations and tests

Operations are comparable to methods, functions, or subroutines in other programming languages. Operations are defined in test classes.

A test is parameterless operation without a return type that is annotated with @Test.

    @Test
    operation isFeatureAvailable () {
        enter
        var i: integer = 0
        var countValue: integer = count
        while (i < 10) {
            raise do_cycle
            assert active (myStatechart.main_region.State_A)
            assert count == countValue + 1
            i = i + 1
        }
    }

The header of an operation consists of the keyword operation, followed by the name of the operation, followed by a parenthesized list of parameters, optionally followed by a colon and a return type. If no return type is specified, it is inferred from the operation’s return statement(s) .

Operations can also be annotated. Operations annotated with Test are automatically executed when the SCTUnit test is executed.

The body of an operation is enclosed in braces ({ … }). It consists of a sequence of statements, which may be empty. If the operation’s header specifies a return type different from void, the operation must return a value of that type, using the return statement .

Figure "Operation grammar" summarizes the structure of an operation:

Operation grammar

Scopes

An operation can define its own variables. Aside from these

• local variables

it also has access to

• variables and operations defined in the test class,
• variables, operations, events, and states defined in an interface of the statechart controlled by the operation’s test class,
• entities imported by the statechart controlled by the operation’s test class.

An operation can call other operations. This is like subroutine calls in other programming languages.

Operations that are declared in the statechart cannot be called from a test class.

Annotations

When running a test class or a test suite as an SCTUnit, only those operations are executed as tests that are annotated with @Test. Operations annotated with @Ignore or not annotated at all will not be regarded as tests. However, they can be called by test operations.

While the @Ignore annotation and an omitted annotation have the same effect functionally, you should use the @Ignore annotation to mark operations that are intended as tests, but are (temporarily) disabled for one or the other reason. This differentiates them from other operations that are not tests, but mere subroutines called from elsewhere.

Statements and expressions

The body of an operation consists of statements. Many types of statements, like variable definitions, assignments, event raisings, or operation calls, are already defined in the statechart language and are described in section "Statements" of the statechart language documentation.

Statement types that are specific to the SCTUnit language are described in the following subsections.

Returning an operation’s result

The return statement terminates the execution of the current operation. It either returns nothing or the value of an expression to the caller. If an expression is returned, its type must match the operation’s return type. If the operation returns nothing, its return type must be void, either implicitly or explicitly.

    operation getCircleArea (radius: real): real {
        const pi: real = 3.141592654
        return pi * pi * radius
    }

Figure "Return statement grammar" summarizes the structure of the return statement:

Return statement grammar

Active

active is a built-in function of the statechart language. In the context of testing, it can be used to determine a state’s state at a given time. It returns a logic value, if the state in question is active it returns true, otherwise false.

assert active (myStatechart.main_region.State_A)

Assertions

When it comes to testing, the most important statement is the assertion. It evaluates a condition that must be fulfilled for the test to not fail.

assert sum == 42

assert active (myStatechart.main_region.State_A)
assert !active (myStatechart.main_region.State_B) message "State_B must not be active here."

Generally, an assertion consists of the keyword assert, followed by a boolean expression, optionally followed by the keyword message and an error message text (string literal). The assertion expects the boolean expression to be true to continue the test. If it evaluates to false the test fails. The optional message can be used to clarify what went wrong.

Asserting an operation call

A special assertion variant uses the called keyword (" assert called statement"). It checks whether a certain operation has been called (executed), typically by some action in the statechart.

assert called myOperation
assert called myOperation(42, 815)
assert called myOperation 4 times
assert called myOperation(42, 815) 1 times
assert ! called myError

Asserting an outgoing event

Finally, you can use the assert statement to check whether the state machine has raised an outgoing event. To do so, the assert keyword is followed by the name of the desired event. It is also possible to assert the opposite, i.e., that the event has not been raised. In this case, insert the negation operator ! between assert and the event name.

Here’s an example:

testclass outgoingEventTest for statechart raiseOutgoingEvent {

    @Test
    operation test_e1 () {
        enter
        raise e1
        proceed 1 cycle
        assert e3
        exit
    }

    @Test
    operation test_e2 () {
        enter
        raise e2
        proceed 1 cycle
        assert ! e3
        exit
    }
}

Assertion grammar

Figure "Assertion grammar" summarizes the structure of an assertion:

Assertion grammar

Entering a state machine

The enter statement serves to enter the statechart associated with this test class. The state machine is initialized and started. The state that is denoted by the initial state becomes active.

A test must execute the enter statement before it can perform any sensible testing on the statechart. Unless the state machine is entered, all states are inactive.

Please see section "SCTUnit by example" for examples on how the enter statement is used. Please also see the section on the exit statement .

Exiting a state machine

The exit statement exits and quits a state machine. You can re-initialize and re-enter it using the enter statement .

testclass enter_exit_tests for statechart myStatechart {

    @Test
    operation checkState () {

        /* Before entering the state machine, all states are inactive: */
        assert !active (myStatechart.main_region.State_A)

        /* Now we are entering the state machine. The state that the
         * initial state points to becomes active: */
        enter
        assert active (myStatechart.main_region.State_A)

        /* The "exit" statement leaves the state machine. All of the 
         * latter's states are thus inactive:
         */
        exit
        assert !active (myStatechart.main_region.State_A)

        /* It is possible to re-enter the state machine or to be precise:
         * to enter a new instance of the state machine. As above,
         * "State_A" should be active now: */
        enter
        assert active (myStatechart.main_region.State_A)
    }
}

Raising an event

The raise statement raises one of the state machine’s incoming events.

raise operate

raise valueChanged : 3

raise user.click

Since state machine internals are inaccessible to SCTUnit tests and outgoing events cannot be raised in general, only incoming events can be raised. Internal events, defined in the internal scope, and outgoing events, defined in interfaces, cannot be raised.

Please note: The raise statement is not specific to the SCTUnit language, but is (also) part of the statechart language, see section "Raising an event".

Proceeding a state machine

The proceed statement allows to proceed the state machine either in terms of time or in terms of run-to-completion steps (RTC). Cycle-based state machines can be explicitly told to perform an RTC step, while event-driven state machines run an RTC step implicitly when an incoming event is raised.

The proceed statement consists of the keyword proceed and an indication by what to proceed. Two variants are available:

• proceed number time_unit – This variant instructs the state machine to proceed by the specified time, e.g., proceed 30 s proceeds by 30 seconds. For cycle-based statecharts, RTC steps will be performed based on the defined cycle time. For example, proceed 400 ms on a statechart with @CycleBased(200) annotation, two RTC steps will be performed. Supported time units are:
  • s – seconds
  • ms – milliseconds
  • us – microseconds
  • ns – nanoseconds
• proceed number cycle – This variant is only available for cycle-based statemachines and instructs the state machine to perform number run-to-completion steps. Proceeding cycles also implies proceeding of time based on the cycle period defined by the @CycleBased() annotation.

raise operate
proceed 100 ms
raise user.click
proceed 1 cycle

When running an SCTUnit test, the state machine does not run in real time, but in virtual time instead. That is, a statement like proceed 3600 s does not have to wait for one hour of real time to elapse. Instead the state machine “leaps” by one hour in an instant, raises all affected time events, and processes them.

Figure "Proceed statement grammar" summarizes the structure of the proceed statement:

Proceed statement grammar

Defining variables and constants

Variables and constants (for brevity we’ll summarize both as “variables”) can be defined as specified in the statechart language, please see sections "Variables" and "Constants" for all the details.

However, variables in the SCTUnit language must always be initialized. For example, while a definition like

var sum: integer

is fine in the statechart language, it is an error in the SCTUnit language. You would rather have to write something like

var sum: integer = 0

Variables can be defined in the scope of the test class or in operations.

Conditional statements

The if statement executes a sequence of statements depending on a condition.

        if (i < 5) {
            raise do_cycle
        }

        if (i < 5) {
            raise do_cycle
        } else {
            raise button5
        }

The if statement starts with the keyword if, followed by a boolean expression in parenthesis, followed by a sequence of statements in braces. The sequence of statements must contain at least one statement. It is executed if and only if the boolean expression evaluates to true.

And optional else clause may follow. It consists of the keyword else, followed by a sequence of statements in braces. The sequence of statements must contain at least one statement. It is executed if and only if the boolean expression evaluates to false.

Figure "If statement grammar" summarizes the structure of the if statement:

If statement grammar

Loops

The while statement executes a sequence of statements repeatedly, as long as a condition is fulfilled.

        var i : integer = 0
        while (i < 10) {
            raise do_cycle
            proceed 1 cycle
            i = i + 1
        }

The while statement starts with the keyword while, followed by a boolean expression in parenthesis, followed by a sequence of statements in braces, the loop’s body. The loop’s body must contain at least one statement. It is executed repeatedly if and only if the boolean expression evaluates to true. The expression is evaluated before the first execution of the loop body and after each execution of the loop body.

Figure "While statement grammar" summarizes the structure of the while statement:

While statement grammar

Retrieving the state machine’s status

The keywords active, is_active and is_final make it possible to retrieve certain aspects of the state machine’s status as boolean values and e.g., use them in assertions.

assert is_active

assert is_final

assert active(main_region.StateA)

Mocking an operation call

The mock statement allows you to mock operations defined in the statechart. You can specify what should be returned when the operation is called, even depending on the given input parameters.

However, how could you test the behaviour of your statechart if you cannot call an actual operation and retrieve its results?

That’s what the mock statement is for. It is a makeshift that mimics an actual operation call by two mechanisms:

1. It creates a static mapping from a specific operation call with a specific list of parameter values to a specific return value.
2. If that operation is called with exactly that specific list of parameter values the caller receives the mapped return value as a result.

A test can create an arbitrary number of such mappings, i.e., it can use the mock statement multiple times for multiple operations, or for multiple parameter list of the same operation.

    @Test
    operation testNeutronFlux() {
        mock getNeutronFlux(12.0) returns (1000000000.0)
        mock getNeutronFlux(18.0) returns (4.3)
        enter
        p = 18.0
        proceed 1 cycle
        assert active(neutronFlux.main_region.State_A)
        p = 12.0
        proceed 1 cycle
        assert active(neutronFlux.main_region.State_B)
    }

In order to avoid an exception to be thrown, you can define a mock statement with a default return value. This value will be returned from all calls of the mocked operation that are not explicitly overridden by mock statements with specific parameters.

        mock getNeutronFlux returns (-1.0)
        mock getNeutronFlux(12.0) returns (1000000000.0)
        mock getNeutronFlux(18.0) returns (4.3)

The type of the default value specified in the mock statement must match the return type of the mocked operation.

Figure "Mock statement grammar" summarizes the structure of the mock statement:

Mock statement grammar

Test suites

A test suite aggregates a set of test classes into a logical unit. It is contained in a file with a .sctunit filename extension.

testsuite MyTestSuite {
    TestClassA,
    TestClassB,
    TestClassC
}

The nice thing about test suites is that you can run all tests of all the test classes at once. Right-click on the test suite file, say, mytestsuite.sctunit, and select Run As → SCTUnit in the context menu. All tests in all test classes referenced in the test suite will be executed.

You can put test classes that are testing different statecharts into a single test suite. However, all test classes within a test suite must pertain to statecharts using the same language domain. For example, if you have a statechart using itemis CREATE' default domain and another statechart using the C domain, you cannot put their respective test classes into the same test suite. Instead, you would have to write two different test suites: one for the test classes testing your “normal” statecharts, the another one for testing your “C” statecharts.

The header of a test suite consists of the keyword testsuite, followed by the name of the test suite.

The body of a test suite is enclosed in braces ({ … }). It consists of one or more names of test classes, separated by comma.

Figure "Test suite grammar" summarizes the structure of a test suite:

Test suite grammar

Test suites are namespace-aware: Using the package statement , you can insert a test suite into a specific namespace.

Namespaces

You can organise your test classes and test suites in different namespaces. Each test class or test suite can assign itself to a namespace by the package statement.

The package statement

Use the package statement to determine a namespace for the test class or test suite in the current .sctunit file. The package statement is optional, but if you use it, it must be the first statement of your .sctunit file. Test classes and test suites without a preceeding package statement will be put into the default namespace.

package foo.light_switch.test

testclass light_switch_tests for statechart light_switch {
…
}

The package statement consists of the keyword package, followed by the package name (namespace). The package name is fully-qualified, i.e., in dot notation.

Please see section "Test units" for a summary of the package statement’s and related statements' grammar.

Test units

A test unit is either a test class or a test suite. It is contained in a file with a .sctunit filename extension.

Generating SCTUnit source code

Executing SCTUnit tests within the Eclipse development environment as described in section "Running the test" is a nice and handy feature for the developers. However, it all happens on the modeling level, a higher abstraction level than the actual source code you are likely generating from your statechart. You will probably want to test the generated source code as well, especially in any safety critical environments.

Generating unit tests as source code comes as a solution. Source code can be inspected, compiled, and integrated wherever needed.

This section explains how you can generate your SCTUnit tests as source code with itemis CREATE. Supported programming languages are C, C++, Java, Python as well as SCXML for Qt. The SCTUnit code generators translate the SCTUnit tests into a unit test framework of the target language. For example, the generated Java test code uses JUnit as its testing framework and Mockito for mocked methods. C and C++ sources rest upon the Google Test unit testing library.

In order to use the SCTUnit code generators effectively, you should already be familiar with generating source code from statecharts, as explained in chapter "Generating state machine code". In fact, you have to generate your statechart as source code first, so that the SCTUnit tests have something to be executed upon. Below we are assuming that you have your statechart code generator model and the generated code handy. Generating SCTUnit test source code follows the same principles as generating state machine source code, so we can focus on SCTUnit specifics here.

Creating a generator model

The first step is to create a generator model (SGen) for the SCTUnit test, similar to explanation in section "Generating state machine code".

1. In the main menu, select File → New → Other…. The New dialog opens.
2. In the New dialog, select itemis CREATE → Code generator model.
3. Click on Next. The itemis CREATE generator model wizard opens.
4. Select a project and a folder as the generator model’s location. The filename must end with .sgen.
5. Click on Next.
6. In the Generator drop-down menu, select one of the source code generators whose names begin with “SCTUnit”, e.g., “SCTUnit C Code Generator”.
7. In the list below, check the SCTUnit files you want to generate as source code.
8. Click on Finish.

As a result, the generator model file is created. Figure "A sample SCTUnit generator model" shows a generator model for the SCTUnit C generator. The target language is reflected in the generator ID “sctunit::c”. Each SCTUnit test unit is contained in a section marked with the test keyword, followed by the respective fully-qualified test unit name.

A sample SCTUnit SGen model

SCTUnit C Code Generator

The SCTUnit C code generator transforms SCTUnit tests into C++ code that makes use of the Google Test unit testing library.

The test code generator can be controlled by the following features:

Outlet

The Outlet feature is required and determines where the generated artifacts will be placed. Meaning is same as for the common outlet feature.

SGenModel

The SGenModel feature allows to generate a basic generator model file for the statechart under test. The generated .sgen file will take over the FunctionInlining and IdentifierSettings features.

FunctionInlining

In this context, the FunctionInlining feature is only relevant in combination with the SGenModel feature.

IdentifierSettings

In order to include the proper statechart header and use the proper statechart APIs, the test code generator needs to know which identifier settings are used in the statechart’s generator model. This means, whenever the statechart generator model uses specific IdentifierSettings, the same values need to be set in the test generator model.

In addition, it is possible to specify the file extension of the generated test files with the option testFilenameExtension. The default is ‘cc’.

LicenseHeader

The LicenseHeader features adds a license header to the generated test files. See also the common LicenseHeader feature.

OutEventAPI

The OutEventAPI feature is not yet used. It is expected that the statechart uses getters for its out event API. The usage of observers is not yet supported. See also the common OutEventAPI feature.

JUnitWrapper

The JUnitWrapper creates an additional JUnit test that simply runs the generated GTest. This is intended for Java based CI environments.

SCTUnit C++ Code Generator

The SCTUnit C++ code generator transforms SCTUnit tests into C++ code that makes use of the Google Test unit testing library.

The test code generator can be controlled by the following features:

Outlet

The Outlet feature is required and determines where the generated artifacts will be placed. Meaning is same as for the common outlet feature.

SGenModel

The SGenModel feature allows to generate a basic generator model file for the statechart under test. The generated .sgen file will take over the FunctionInlining and IdentifierSettings features.

FunctionInlining

In this context, the FunctionInlining feature is only relevant in combination with the SGenModel feature.

IdentifierSettings

In order to include the proper statechart header and use the proper statechart APIs, the test code generator needs to know which identifier settings are used in the statechart’s generator model. This means, whenever the statechart generator model uses specific IdentifierSettings, the same values need to be set in the test generator model.

In addition, it is possible to specify the file extension of the generated test files with the option testFilenameExtension. The default is “cc”.

LicenseHeader

The LicenseHeader features adds a license header to the generated test files. See also the common LicenseHeader feature.

OutEventAPI

The OutEventAPI feature is not yet used. It is expected that the statechart uses getters for its out event API. The usage of observers is not yet supported. See also the common OutEventAPI feature.

JUnitWrapper

The JUnitWrapper creates an additional JUnit test that simply runs the generated GTest. This is intended for Java based CI environments.

GeneratorOptions feature

The GeneratorOptions feature allows to change the behavior of the C++ generator. With the statemachineRefsAsPlainPointer option the generated code in multi-statemachine scenarios will use plain pointers instead of smart pointers for referencing statemachines, just like in the C++ statemachine code generator.

SCTUnit Java Code Generator

The SCTUnit Java code generator transforms SCTUnit tests into JUnit tests. The generated JUnit tests make use of the Mockito framework when operation mocking is required.

The test code generator can be controlled by the following features:

Outlet

The Outlet feature is required and determines where the generated artifacts will be placed. Meaning is same as for the common outlet feature.

Naming

The Naming feature allows to specify a different package for the generated test class than the statechart class belongs to. If this feature is used, it is required to specify the package of the statechart class in the StatechartNaming feature as described in the next chapter.

feature Naming {
    basePackage = "light.tests"
}

StatechartNaming

For Java, the generated state machine code on one hand and the SCUnit test classes on the other hand may well belong to different packages. However, in order to be able to execute the state machine, the SCTUnit tests need to know what the state machine’s package is.

This is what the StatechartNaming feature is good for. It contains all the properties you can set in the state machine’s Java code generator Naming feature.

feature StatechartNaming {
    basePackage = "bar.light_switch"
    typeName = "CustomLightSwitch"
    libraryPackage = "bar.light_switch.lib"
    libraryTargetFolder = "src-lib"
}

LicenseHeader

The LicenseHeader features adds a license header to the generated test files. See also the common LicenseHeader feature.

OutEventAPI

The OutEventAPI feature is not yet used. It is expected that the statechart uses getters for its out event API. The usage of observers is not yet supported. See also the common OutEventAPI feature.

SCTUnit Python Code Generator

The SCTUnit Python code generator transforms SCTUnit tests into Python unittests.

The test code generator can be controlled by the following features:

Outlet

The Outlet feature is required and determines where the generated artifacts will be placed. Meaning is same as for the common outlet feature.

Naming

The Naming feature allows to specify a different package for the generated test class than the statechart class belongs to. If this feature is used, it is required to specify the package of the statechart class in the StatechartNaming feature as described in the next chapter.

feature Naming {
    basePackage = "light.tests"
}

StatechartNaming

For Python, the generated state machine code on one hand and the SCUnit test classes on the other hand may well belong to different packages. However, in order to be able to execute the state machine, the SCTUnit tests need to know what the state machine’s package is.

This is what the StatechartNaming feature is good for. It contains all the properties you can set in the state machine’s Python code generator Naming feature.

feature StatechartNaming {
    basePackage = "bar.light_switch"
    libraryPackage = "bar.light_switch.lib"
    libraryTargetFolder = "src-lib"
}

LicenseHeader

The LicenseHeader features adds a license header to the generated test files. See also the common LicenseHeader feature.

OutEventAPI

The OutEventAPI feature is not yet used. It is expected that the statechart uses getters for its out event API. The usage of observers is not yet supported. See also the common OutEventAPI feature.

JUnitWrapper

The JUnitWrapper creates an additional JUnit test that simply runs the generated GTest. This is intended for Java based CI environments.