Automated Scenario Test

Purpose This document provides guidance and information that is necessary to be able to effectively implementing the automations of the test-cases.

Audience The audience of this documentation is the software SDETs1 responsible for the implementation of the automations of the test-cases. Implementing the automations involves translating the test-cases into XML documents where the steps in the test case are spelled out, and implementing custom actions (in the java programming language) that actually carry out the tasks to in each step. Therefore, knowledge of XML and programming in java are required on the side of the SDET.

Scope Each XML mentioned above captures a test-case (the intention and the steps) in a program-readable format. Throughout this documentation we will refer to that XML as scenario xml. This documentation explains all the elements and attributes expected / allowed in the scenario XML.

1 Software Development Engineer in Test

Scenario XML The scenario XML will be explained using several examples, starting with the one below. <scenario max-thread="5" repetition="1"> ➊ <desc> <wait timeout="2">eq(mediator("isStarted()"), true)

➋

<args>

"lee" <steps>

➌

➍

<step name="step 1" mult="2" concurrent="true"> <desc>step 1 desc <args> "fu"

➎

➏

<desc>action 1.1 desc <args> list("a","b") range(7,8,1) 9 <desc>action 1.2 desc

➐

<script> <args> list(10000, null) <step name="step 2" mult="4" concurrent="false"> <wait>2 <desc>action 2.1 desc <args> list(3, true, "test", null) switch(ref(x), {true:concat("It's true! Look:", ref(x)), "zeppelin":"Wow that's a monster.", any:"Well..."})

Insert 1. Scenario XML #1

First pass through the scenario XML The <scenario> node ➊ is the root of a scenario XML. It has an optional attribute repetition, whose value can be any integer equals to or greater than 1 (by default it is set to 1). It is used to specify how many times the scenario will be ran, sequentially. If we set it to 0, the scenario will be repeated over and over, until we kill the process. Another optional attribute of <scenario> is max-thread, that specifies the maximum number of threads the scenario runner can use to run the actions. The number specified here is not a hard limit; there can be a little more threads than the number specified. As a general guideline, in order to avoid resorting into sequential execution of actions, pick the highest multiplicity in the scenario, add a constant to it, and assign the result to the max-thread attribute. In the above example it's 4 (the multiplicity of step 2), and the constant used is 1 (because step 2 is one level below the scenario). Therefore the chosen value for max-thread is 4 + 1 = 5. Inside the <scenario> node we can put a <desc> node, in which we write the (high-level) description of the scenario. We must enclose the description in a CDATA section if the description contains special characters which otherwise would render the scenario XML unprocessable. A scenario, as well as step and action, is a configurable node; it can take arguments. Those arguments are contained in the <args> node ➋. Each of them is represented by an <arg> node ➌, whose value can be a string literal, integer literal, bigdecimal literal (e.g.: 1.234), boolean literal (true / false), null, XML representation of a java object, XML tree, or function. The following table contains shows some examples of how to define an argument.

Type

Example

Value received in the java program (action class)

String literal

”foobar”

java.lang.String “foobar”

Int literal

78

java.lang.Integer 78

Boolean literal

false

java.lang.Boolean false

BigDecimal literal

7.5

java.math.BigDecimal 7.5

Null

null

An instance of com.baktun.bstrd.director.Null

Any

any

An instance of com.baktun.bstrd.director.Any

XML tree

<node key=”aVar”> Some beans

An instance of org.w3c.dom.Node

2

XML representation of Some beans

An instance of com.eshop.PurchaseOrder3

Function4

An instance of BigDecimal 6

mult(3,2)

Table 1. Examples of argument The arguments defined in a configurable node are visible to the child configurable-nodes. The usual scoping rules apply (i.e.: an argument defined in a configurable will override the argument with the same key defined in the parent node(s)).

2 Assigning null to an argument is not the same as not defining the argument. 3 The class must contain special (JAXB) annotations that would allow automatic mapping to XML . 4 All the available functions will be explained in separate section.

A scenario is composed of one or more steps, which will be executed one after another. Those steps must be contained within the <steps> node ➍. Each of them is represented by a <step> node ➎ that has the following attributes: • name (mandatory): the name of the step. Each step within the same scope5 must have a unique name. • offset (mandatory): the duration, in seconds, for the execution of the step to be delayed. The value must be an integer equal to or greater than 1. • mult (mandatory): the number of instances of the step to be created and executed. The value must be an integer equal to or greater than 1. • concurrent (optional): specifies whether the instances of the step will be executed sequentially or concurrently. The valid value is true (for concurrent) or false (for sequential). The default value is false. Inside the a step we define the actions that belong to the step. Those actions must be contained within the node ➏. Each of them is represented by an node ➐ that has the following attributes: • name (mandatory): the name that uniquely identifies the action within the step. • class (optional): the name of the java class that actually performs the task intended by the action. If we don't specify this, we must write the code directly inside the script node ➑, as exemplified by action 1.2. • asynch (optional): specifies whether the action is to be executed in the same thread as the containing step or in separate thread.

Running a scenario To see a scenario XML in action, a separate program has to be written. This program, which we will refer to as director throughout this documentation, in general performs the following tasks: Load the scenario XML (into a java object of type ScenarioDefinition). Create an instance of mediator. Create an instance of ScenarioRunner. Start the scenario runner by calling its run(...) method6, feeding in the scenario definition created in step #1. ➎ Start the mediator. ➊ ➋ ➌ ➍

The mediator is a generic java object; it can be anything. It is the entity we want to control, manipulate, or test throughout the execution of scenario, by means of actions. The following diagram depicts the relationship between scenario, mediator, and action.

5 The meaning of it will be clarified in the section where “action-nesting” is explained. 6 It has to run in a thread different from the one that runs the mediator.

Figure 1. The relationship between scenario, mediator, and actions. The following insert contains an example of such director. In this example, the mediator is an instance of com.baktun.bstrd.unittest.Frijol7. We have something quite similar for the spectel bridge simulator; the director is a java application named com.baktun.bridgesimulator.BridgeDirector, and the mediator is an instance of com.baktun.bridgesimulator.Bridge, wrapped inside an instance of com.baktun.bridgesimulator.BridgeServer. package com.baktun.bstrd.unittest; import import import import import

java.io.BufferedReader; java.io.FileReader; org.apache.log4j.Logger; com.baktun.bstrd.director.ScenarioDefinition; com.baktun.bstrd.director.ScenarioRunner;

public class FrijolDirector { private static final Logger logger = Logger.getLogger(FrijolDirector.class); private FrijolDirector() {} public static void main(String[] args) throws Exception { if (args == null || !(args.length == 2) ) { System.out.println("Usage: java FrijolDirector <scenario.xml> [keep-frijol-up]"); System.exit(-1); } boolean keepFrijolUp = true; if (args.length == 2) { keepFrijolUp = Boolean.parseBoolean(args[1]); } String scenarioDefXml = null; { BufferedReader br = new BufferedReader(new FileReader(args[0])); StringBuffer sb = new StringBuffer(); try { String line = br.readLine(); while (line != null) { sb.append(line).append("\n"); line = br.readLine();

7 Spanish word for bean.

} } finally { if (br != null) { br.close(); } } scenarioDefXml = sb.toString(); } final ScenarioDefinition scenarioDef = ScenarioRunner

➊ final Frijol frijol = new Frijol(); ➋ .loadScenarioDef(scenarioDefXml);

final ScenarioRunner scenarioRunner = new ScenarioRunner();

➌

Thread scenarioRunningThread = new Thread(new Runnable() { public void run() { try { scenarioRunner.run(scenarioDef, frijol); } catch (Exception e) { e.printStackTrace(); } } }); scenarioRunningThread.setPriority(Thread.MIN_PRIORITY); scenarioRunningThread.start(); frijol.run();

➍

➎

} }

Insert 2. An example director The output of the execution of the director above, in the following insert, will help clarifying a couple of important points about the execution model of the scenario runner. 2009-09-23 15:42:24,564 [INFO] Thread-2 com.baktun.bstrd.unittest.Frijol - Frijol is going to sleep for 5 seconds.... 2009-09-23 15:42:24,583 [INFO] Thread-1 ➋ com.baktun.bstrd.director.ScenarioRunner Running scenario for the 1 times 2009-09-23 15:42:30,640 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner - About to execute step step 1

➊,5,main]:{jet=lee, kung=fu, a=b ➌, c=9, b=8 ➍} Thread[pool-1-thread-1 ➊,5,main]:{jet=lee, kung=fu, a=a ➌, c=9, b=7 ➍} Thread[pool-1-thread-2

----2009-09-23 15:42:30,661 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner - About to execute step step 2

➎,5,main]: Done! : 1913434059 Thread[pool-1-thread-4 ➎,5,main]: Done! : -498702880 Thread[Thread-1 ➋,1,main]:{jet=lee, y=Well..., x=3} Thread[pool-1-thread-3

--Thread[Thread-1,1,main]:{jet=lee, y=It's true! Look:true, x=true} --Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=test} --Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=null} ---

Insert 3. Output of the execution of example director

Threading in the scenario Since step 1 is declared as concurrent, there might be new thread(s) allocated for the execution of the instance(s) of step 1. In the above case, we also declared step 1 to have multiplicity 2, which might cause two instances of step 1 to be created, each of them will be running in separate threads8 ➊. In contrast, when a step is not declared as concurrent, as is the case of step 2, it will be executed in the same thread as the one in which the containing node is running ➋. In the above case, the containing node is the scenario, which is running in the main thread. The scenario runner will go through all the steps sequentially, in the document order. That means, step 2 will be executed only after all the instances of step 1 have finished running. Action will be running in the same thread as the one in which the containing step is running ➊, except when it is declared as asynchronous (the case of action 1.2). Asynchronous action will be running in yet separate thread ➎9. Implementions of action are not expected to either create or manage any threads (although currently there's nothing that prevents it from doing so). If an action is intended to run in the background (e.g.: as a listener), we can simply declare it as asynchronous in the scenario XML. The declarative style of doing things here is aimed at improving the communication among the teammembers, by making the intention spelled out clearly in the scenario XML. Above all, this test framework is a communication tool that fills the gap between the test-case document, and the executable that realizes the test-case. The test-framework takes common, repeated concerns (such as threading) away from us, while still making it accessible in another way (declarations in the scenario XML). This allows us to have a lean and compact implementation of actions, which are expected to consist of only a few lines of code that basically take the arguments passed-in to it, process them a little bit, and use them during the interaction with the mediator.

8 There is no guarantee that instances of a step declared as concurrent will be running concurrently. The scenario-runner makes its best effort to fulfill it, though, but if the maximum number of threads is already reached, the scenario-runner will resort to executing those steps in the same thread as the one in which the parent node is executing, resulting in sequential execution of those instances of step. 9 Care has been taken about creation of new threads. The engine utilizes Java's built-in thread-pooling mechanism. This has consequences such as: action that uses ThreadLocal as storage will likely break.

Argument's-value resolution Initially there was a simple mechanism: you declare the argument by simply specifying the key and the value, what you see (in the XML) is what you get (in the action). Things changed when we introduced dynamism into the scenario XML, by means of a domain-specific expression language. Some of the functions available in the expression language allows the value of an argument to be based on the value of other argument, for example: concat(ref(segment_id), "-", ref(participant_id)). Now we can make the value of an argument to vary according to the index in the iteration: list("white", "red", "black"). We can also make use of a construct similar to the switch-case-default statement in programming language like java: switch(ref(hobby), {"music":"give him a guitar", "literature":"buy him a kindle", null:"give him some cash"}). By combining that dynamic argument-value and multiplicity of steps, a whole set of combinations of inputs for the test-case can be packed into a single scenario XML. There are two functions that particularly deserves a couple of paragraphs (of clarification) here: • The list function, with syntax: list(element_1, element_2, ..., element_n) • The range function, with syntax: range(start_val, end_val, step) Assigning a list to an argument in the scenario XML does not mean that an instance of java.util.List will be passed-in to the action. Instead, the action will only see a single element bound to that argument, taken from the list. Example: <scenario repetition="1"> <steps> <step name="step 1" offset="0" mult="2" concurrent="true"> <args> list("green", "orange") <args> list(1980, 1990)

Insert 3. An example scenario definition with list(...) In the example presented above, there will be two threads. The value of the arguments accessible from the action running in the first thread: {var_a: green, var_b: 1980} . The second thread, on the other hand, will see the following values: {var_a: orange, var_b: 1990}.

The function range behaves in similar fashion. In fact, we can see a range as just another list, only that we don't have to specify all the elements manually. That is to say: • range(10, 15, 1) is equivalent to list(10, 11, 12, 13, 14, 15) • range(10, 15, 2)10 is equivalent to list(10, 12, 14) The number of elements in the list must be at least equal to multiplicity of the step to which it directly belongs. In the above example, list var_a and var_b both belong to the step step 1 whose multiplicity is set to 2. Therefore the length of those list be at least equal to 2. Extra elements at the end will simply be ignored / unused. The following insert contains another example that uses list, this time involves nesting of actions. In that case, the length of list var_c and var_d must be at least 1, which is the multiplicity of step 2 to which those lists directly belong. <scenario repetition="1"> <steps> <step name="step 1" offset="0" mult="2" concurrent="true"> <args> list("green", "orange") <args> list(1980, 1990) <step name="step 2" offset="0" mult="1" concurrent="true"> <args> list(9) <args> list("bar")

Insert 4. Another example scenario definition with list

10 Probably mod-check is necessary to eliminate misunderstanding / mis-expectations.

Available functions •

•

•

• •

•

• • •

•

sum: performs addition of all its arguments, returns an instance of java.math.BigDecimal. Examples: ◦ sum(4, 5) returns BigDecimal(9) ◦ sum(4, 7.15) returns BigDecimal(11.15) ◦ sum(4, 7, ref(x)) returns BigDecimal(16.5), assuming x resolves to 5.5 mult: multiplies all of its (two) arguments, returns an instance of java.math.BigDecimal. Examples: ◦ mult(4, 2.5) returns BigDecimal(10) ◦ mult(ref(x), ref(y)) returns BigDecimal(16), assuming x resolves to 5 and y resolves to 3.2. div: divides its first arguments by its second argument, returns an instance of java.math.BigDecimal. Examples: ◦ div(1, 2) returns BigDecimal(0.5) ◦ div(sum(ref(x), 4), mult(ref(y), 5)) returns BigDecimal(0.5), assuming x resolves to 1 and y resolves to 2. int: cast its argument to an integer, returns an instance of java.lang.Integer. Example: ◦ int(ref(x)) returns Integer(1), assuming x resolves to 1.2 mod: returns the remainder of division of its first argument by its second argument, as an instance of java.lang.Integer. Example: ◦ mod(ref(x), 5) returns Integer(2), assuming x resolves to 7. concat: concatenate all of its arguments, returns an instance of java.lang.String. Example: ◦ concat(ref(x), "_", mult(4, ref(y)), "_", 5) returns the string 7_12_5, assuming x resolves to 7 and y resolves to 3. list: returns a single element, as an instance of java.lang.Object, taken from the list, successively at each iteration. wheel: similar to the list function. However, instead of giving you an error, it quietly returns the first element in the list whenever the end of the list is passed. range: returns a single element, as an instance of java.lang.Object, taken from the range, successively at each iteration. The first input is the beginning of the range, the second input is the stop of the range, and the third input is the step (the difference between two adjacent elements in the range). The first and second input can be any integer (positive, zero, or negative). The first and the second input can't have the same value (otherwise an error will be thrown). If the first input is smaller than the second input, the range will be an “ascending” one. Otherwise it will be a “descending” one. The step can not be less than 1. Examples: ◦ range(2, 4, 1) is equivalent to list(2, 3, 4) ◦ range(4, 2, 1) is equivalent to list(4, 3, 2) ◦ range(4, 2, -1) throws an error. ◦ range(2, 2, 1) throws an error. ◦ range(2, ref(x), 1) is equivalent to list(2, 3, 4), assuming x resolves to 4. ref: takes the name of another argument as input, returns the value that is bound to that other argument. Examples: ◦ ref(x) returns Integer(5), assuming x is set to 5. ◦ ref(y) returns BigDecimal(3.2), assuming y resolves to 3.2 (y might be defined as

mult(1.6, 2), for example).

•

•

•

•

•

switch: takes the form switch(selector, map_of_cases), returns the value of the key-value pair whose key matches the value of the selector. Examples: ◦ switch(ref(x), {"boy": "Rama", "girl": "Shinta", any: ref(generic_name)}) returns: ▪ the string "Rama" if x resolves to "boy". ▪ the string "Shinta" if x resolves to "girl". ▪ the value the variable generic_name resolves to if x resolves to any value other than "boy" or "girl". neg: accepts either integer / bigdecimal / boolean, and returns the negated value of its input. Examples: ◦ neg(1) returns Integer(-1) ◦ neg(-2.5) returns BigDecimal(2.5) ◦ neg(ref(x)) returns Boolean(true), assuming x resolves to false. eq: compares its two inputs, returns true if they're equal. Except for some special cases, the equality is based on the value of the two inputs (i.e.: the result of calling equals method on one of the input, passing in the other input as the argument). Examples: ◦ eq(1, 1) returns Boolean(true). ◦ eq(1, “1”) returns Boolean(false). ◦ eq(1, ref(x)) returns Boolean(true), assuming x resolves to Integer(1). ◦ eq(ref(x), any) returns Boolean(true), assuming x resolves to any non-null value. ◦ eq(mediator(“getWatcher()”), null) returns Boolean(true), assuming the invocation of method getWatcher() on the mediator returns null. ◦ eq(mediator(“getWatcher()”), any) returns Boolean(true), assuming the invocation of method getWatcher() returns any non-null value. gt: compares its two inputs, returns Boolean(true) if the first input is greater than the second input. Will return error if any of the input is neither an instance of Integer nor BigDecimal. Examples: ◦ gt(2, 1) returns Boolean(true). ◦ gt(2.1, 1) returns Boolean(true). ◦ gt(1, 2) returns Boolean(false). ◦ gt(ref(x), 2) returns Boolean(true), assuming x resolves to any Integer or BigDecimal greater than 2. ◦ gt(“2”, 1) throws an error. lt: compares its two inputs, returns Boolean(true) if the first input is smaller than the second input. Will return error if any of the input is neither an instance of Integer nor BigDecimal. Examples: ◦ lt(1, 2) returns Boolean(true). ◦ lt(1, 2.1) returns Boolean(true). ◦ lt(2, 1) returns Boolean(false). ◦ lt(2, ref(x)) returns Boolean(true), assuming x resolves to any Integer or BigDecimal greater than 2. ◦ gt(1, “2”) throws an error.

•

and: performs logical AND operation on all11 of its inputs. All of the inputs must return an instance of Boolean. Examples: ◦ and(true, true) returns Boolean(true). ◦ and(true, false) returns Boolean(false). ◦ and(true, true, ref(x)) returns Boolean(false), assuming x resolves to Boolean(false). ◦ and(mediator(“isStarted()”), mediator(“isOkToTest()”)) returns Boolean(true) assuming the invocation of both isStarted() and isOkToTest()

method on the mediator returns true.

• •

•

◦ and(true, mediator(“getNumberOfLines()”)) throws an error, assuming the invocation of getNumberOfLines() method on the mediator returns anything other than Boolean (object / primitive). or: similar to the and function, only that the logical operation performed on the inputs is OR. rand: takes two inputs, both of them must resolve to integers; will generate a random integer between the range specified by those two inputs. Examples: ◦ rand(3, 7) can return any of the following values: 3, 4, 5, 6, or 7. ◦ rand(2, ref(x)) can return either 2, 3, 4 assuming x resolves to 4. mediator: takes the OGNL12 expression that will be applied on the mediator. The simplest way to see it: it allows us to specify the chain of method invocations (that starts on the mediator). This function returns the value returned from the invocation(s). Example: ◦ mediator(“isStarted()”), invokes the isStarted() method on the mediator, returns the value returned from that invocation. ◦ mediator(“getParticipant(#participant_id)”), invokes the getParticipant(...) method on the mediator, passing in the value of the argument whose key is “participant_id”. This requires an argument to be defined in the same node (action / step / scenario) or above as the node that in which the mediator function is used. Example: mediator(“getParticipant(#participant_id)”) ”part_2009_xx_00021”

•

◦ mediator(“getParticipant(#participant_id).getName()”), invokes the getName() method on the object returned by the invocation of getParticipant(...) method on the mediator. ◦ mediator(“launchRocket()”) throws an error if the method launchRocket() is neither defined nor accessible in the mediator object. context: takes the form context(path, context_var_key) . The path specifies the node in the scenario where the object stored under the key context_var_key13 is to be found. The syntax of the path is as follows: ◦ . (single dot): current node. ◦ ... (triple dot): root node (scenario) ◦ .. (double dot): parent ◦ ./action_1

11 There is no short-circuiting here; each and every arguments will be evaluated. 12 Object-Graph Navigation Language: http://en.wikipedia.org/wiki/OGNL 13 Storing object in the scenario-execution context will be explained in separate section.

•

◦ ../.. ◦ ../action_1 ◦ ../../../action_1 error: it is meant to be used only inside the <expectation> node, to specify the error that is expected to occur inside an execution-node. This function has two possible syntaxes: ◦ with one argument: error(exception_class_name), which will return true if any instance of error of the specified type occured. Examples: ▪ error(“java.lang.Exception”) ▪ error(“java.lang.IOException”) ◦ with two arguments: error(exception_class_name, ognl_expression), which

goes further than the first version, by doing a chain of method invocation starting at the instance of exception (using the specified ognl_expression). This functions returns the value returned from the method invocation(s). Examples: ▪ error(“java.io.FileNotFoundException”, “getMessage()”); returns the message of the instance of the FileNotFoundException. This version of error function is meant to be used together with the eq function. Example: ▪ eq(“c:\\theimportantfile.txt can not be found”, error(“java.io.FileNotFoundException”, “getMessage()”))

•

stepidx: this function returns the index of the instance of the step. It takes as input the path to the execution-node where the calculation of the index is based on14. For example, when we have a step declaring the following, the second instance of the step will see the integer 2 assigned to the argument the_index. ... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepidx(".") ... ...

Caution about the context_path that is used as input for this function: it must resolve to a node that is a step. Otherwise an error will be thrown. Example: ... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepidx(".") <args> ... ... ... ...

14 See the explanation of function context for some examples of path.

The following definition of argument the_index, however, will not throw an error. Instead, the action that is part of the second instance of the step will see the integer 2 assigned to the argument the_index. ... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepidx("..") <args> ... ... ... ...

•

stepmult: just like the stepidx function, it takes as input the path to the execution-node where the calculation of the multiplicity is based on. The same restriction as in the stepidx applies; the path must resolve to a step. It returns an integer, the multiplicity of the step. Examples: ... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepmult(".") ... ... ... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepmult(".") <args> ... ... ... ...

... <step name="step 1" mult="3" concurrent="true"> <desc>step 1 desc <args> stepmult("..") <args> ... ... ... ...

As you can see from the examples above, we can pass a function as argument of another function. This capability allows to build plethora of interesting combinations. The following grammar gives a complete picture of the expression language of this test framework.

stat: expr; expr: STR_LITERAL | INT_LITERAL | BIGDEC_LITERAL | NULL_EXPR | ANY_EXPR | func | bool_expr | mediator_expr | context_expr | rand_func | stepidx_func | stepmult_func | str_func; bool_expr: BOOL_LITERAL | eq_expr | gt_expr | lt_expr | and_expr | or_expr | error_expr; eq_expr: 'eq' '(' expr ',' expr ')' -> ^('eq' expr expr); gt_expr: 'gt' '(' expr ',' expr ')' -> ^('gt' expr expr); lt_expr: 'lt' '(' expr ',' expr ')' -> ^('lt' expr expr); and_expr: 'and' '(' expr ',' expr (',' expr)* ')' -> ^('and' expr expr+); or_expr: 'or' '(' expr ',' expr (',' expr)* ')' -> ^('or' expr expr+); mediator_expr: 'mediator' '(' STR_LITERAL ')' -> ^('mediator' STR_LITERAL); context_expr: 'context' '(' STR_LITERAL ',' STR_LITERAL ')' -> ^('context' STR_LITERAL STR_LITERAL); error_expr: 'error' '(' STR_LITERAL ')' -> ^('error' STR_LITERAL) | 'error' '(' STR_LITERAL ',' STR_LITERAL ')' -> ^('error' STR_LITERAL STR_LITERAL); func: ref_func | list_func | range_func | concat_func | mult_func | sum_func | neg_func | mod_func | switch_func | div_func | int_func | wheel_func | short_func; str_func: 'str' '(' expr ')' -> ^('str' expr); range_func: 'range' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' -> ^('range' arith_func_param arith_func_param INT_LITERAL); ref_func: 'ref' '(' VAR_NAME ')' -> ^('ref' VAR_NAME); mult_func: 'mult' '(' arith_func_param ',' arith_func_param ')' -> ^('mult' arith_func_param arith_func_param); sum_func: 'sum' '(' arith_func_param (',' arith_func_param)+ ')' -> ^('sum' arith_func_param arith_func_param+); div_func: 'div' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' -> ^('div' arith_func_param arith_func_param INT_LITERAL); mod_func: 'mod' '(' arith_func_param ',' arith_func_param ')' -> ^('mod' arith_func_param arith_func_param); neg_func: 'neg' '(' neg_func_param ')' -> ^('neg' neg_func_param); neg_func_param: arith_func_param | bool_expr; int_func: 'int' '(' short_int_func_param ')' -> ^('int' short_int_func_param); short_func: 'short' '(' short_int_func_param ')' -> ^('short' short_int_func_param); short_int_func_param: arith_func_param | bool_expr; rand_func: 'rand' '(' arith_func_param ',' arith_func_param ')' -> ^('rand' arith_func_param arith_func_param); stepidx_func: 'stepidx' '(' STR_LITERAL ')' -> ^('stepidx' STR_LITERAL); stepmult_func: 'stepmult' '(' STR_LITERAL ')' -> ^('stepmult' STR_LITERAL); arith_func_param: INT_LITERAL | BIGDEC_LITERAL | ref_func | mult_func | sum_func | neg_func | list_func | range_func | div_func | int_func | mod_func | switch_func | wheel_func | rand_func | stepidx_func | stepmult_func; list_func: 'list' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('list' map_entry_key_value map_entry_key_value+); wheel_func: 'wheel' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('wheel' map_entry_key_value map_entry_key_value+); switch_func: 'switch' '(' switch_func_selector ',' '{' switch_func_case (',' switch_func_case)* '}' ')' -> ^('switch' switch_func_case+ switch_func_selector); switch_func_selector: arith_func_param | concat_func; switch_func_case: map_entry_key_value ':' map_entry_key_value -> ^(SWITCH_CASE map_entry_key_value map_entry_key_value); map_entry_key_value: STR_LITERAL | NULL_EXPR | bool_expr | ANY_EXPR | concat_func | arith_func_param; concat_func: 'concat' '(' expr (',' expr)+ ')' -> ^('concat' expr expr+); STR_LITERAL: '"' ( options {greedy=false;} : . )* '"'; INT_LITERAL: '-'? ('0'..'9')+; BIGDEC_LITERAL: '-'? ('0'..'9')+ '.' ('0'..'9')+; BOOL_LITERAL: 'false' | 'true'; NULL_EXPR: 'null'; ANY_EXPR: 'any'; VAR_NAME: ('A'..'Z' | 'a'..'z' | '_' | '$') ('A'..'Z' | 'a'..'z' | '0'..'9' | '_' | '$')*; WS: (' ' | '\t' | '\n' | '\r')+ {skip();};

Insert 5. The grammar of the expression language of the test framework

Implementing action Once the scenario is completely laid-out and received an approval, we are for the final step that is implementing the actions that will actually carry out the tasks. There two ways of implementing action: (a) create the action class, or (b) script it directly inside the scenario XML. Creating an action class An action class is a (java) class that extends com.baktun.bstrd.director.Action. We only have to implement one method: callExecute(...). The following insert contains the source code of an example action class, PrintArgsAction that simply prints out all the arguments visible from it. package com.baktun.bstrd.unittest; import import import import import

java.util.Map; com.baktun.bstrd.director.Action; com.baktun.bstrd.director.ActionDefinition; com.baktun.bstrd.director.Step; com.baktun.bstrd.director.ValueProvider;

public class PrintArgsAction extends Action