DataDriver for Robot Framework®
DataDriver is a Data-Driven extension for Robot Framework®. This document explains how to use the DataDriver library listener. For information about installation, support, and more, please visit the project page
For more information about Robot Framework®, see //robotframework.org.
DataDriver is used/imported as Library but does not provide keywords which can be used in a test. DataDriver uses the Listener Interface Version 3 to manipulate the test cases and creates new test cases based on a Data-File that contains the data for Data-Driven Testing. These data file may be .csv , .xls or .xlsx files.
Data Driver is also able to cooperate with Microsoft PICT. An Open Source Windows tool for data combination testing. Pict is able to generate data combinations based on textual model definitions. //github.com/Microsoft/pict
It is also possible to implement own DataReaders in Python to read your test data from some other sources, like databases or json files.
Installation
If you already have Python >= 3.6 with pip installed, you can simply run:
pip install --upgrade robotframework-datadriver
Excel Support
For file support of xls or xlsx file you need to install the extra XLS or the dependencies. It contains the dependencies of pandas, numpy and xlrd. Just add [XLS] to your installation. New since version 3.6.
pip install --upgrade robotframework-datadriver[XLS]
Python 2
or if you have Python 2 and 3 installed in parallel you may use
pip3 install --upgrade robotframework-datadriver
DataDriver is compatible with Python 2.7 only in Version 0.2.7.
pip install --upgrade robotframework-datadriver==0.2.7
Because Python 2.7 is deprecated, there are no new feature to python 2.7 compatible version.
Table of contents
- What DataDriver Does
- How DataDriver Works
- Usage
- Structure of Test Suite
- Structure of data file
- Accessing Test Data From Robot Variables
- Data Sources
- File Encoding and CSV Dialect
- Custom DataReader Classes
- Selection of Test Cases to Execute
- Configure DataDriver by Pre-Run Keyword
- Pabot and DataDriver
What DataDriver Does
DataDriver is an alternative approach to create Data-Driven Tests with Robot Framework®. DataDriver creates multiple test cases based on a test template and data content of a csv or Excel file. All created tests share the same test sequence [keywords] and differ in the test data. Because these tests are created on runtime only the template has to be specified within the robot test specification and the used data are specified in an external data file.
RoboCon 2020 Talk
Brief overview what DataDriver is and how it works at the RoboCon 2020 in Helsiki.
Alternative approach
DataDriver gives an alternative to the build in data driven approach like:
This inbuilt approach is fine for a hand full of data and a hand full of test cases. If you have generated or calculated data and specially if you have a variable amount of test case / combinations these robot files become quite a pain. With DataDriver you may write the same test case syntax but only once and deliver the data from en external data file.
One of the rare reasons when Microsoft® Excel or LibreOffice Calc may be used in testing ;-]
See example test suite
See example csv table
How DataDriver Works
When the DataDriver is used in a test suite it will be activated before the test suite starts. It uses the Listener Interface Version 3 of Robot Framework® to read and modify the test specification objects. After activation it searches for the Test Template -Keyword to analyze the [Arguments] it has. As a second step, it loads the data from the specified data source. Based on the Test Template -Keyword, DataDriver creates as much test cases as data sets are in the data source.
In the case that data source is csv [Default] As values for the arguments of the Test Template -Keyword, DataDriver reads values from the column of the CSV file with the matching name of the [Arguments]. For each line of the CSV data table, one test case will be created. It is also possible to specify test case names, tags and documentation for each test case in the specific test suite related CSV file.
Usage
Data Driver is a "Library Listener" but does not provide keywords. Because Data Driver is a listener and a library at the same time it sets itself as a listener when this library is imported into a test suite.
To use it, just use it as Library in your suite. You may use the first argument [option] which may set the file name or path to the data file.
Without any options set, it loads a .csv file which has the same name and path like the test suite .robot .
Example:
Structure of Test Suite
Requirements
In the Moment there are some requirements how a test suite must be structured so that the DataDriver can get all the information it needs.
- only the first test case will be used as a template. All other test cases will be deleted.
- Test cases have to be defined with a Test Template in Settings secion. Reason for this is, that the DataDriver needs to know the names of the test case arguments. Test cases do not have named arguments. Keywords do.
- The keyword which is used as Test Template must be defined within the test suite [in the same *.robot file]. If the keyword which is used as Test Template is defined in a Resource the DataDriver has no access to its arguments names.
Example Test Suite
In this example, the DataDriver is activated by using it as a Library. It is used with default settings. As Test Template the keyword Invalid Login is used. This keyword has two arguments. Argument names are ${username} and ${password}. These names have to be in the CSV file as column header. The test case has two variable names included in its name, which does not have any functionality in Robot Framework®. However, the Data Driver will use the test case name as a template name and replaces the variables with the specific value of the single generated test case. This template test will only be used as a template. The specified data Default and UserData would only be used if no CSV file has been found.
Structure of data file
min. required columns
- *** Test Cases *** column has to be the first one.
- Argument columns: For each argument of the Test Template keyword one column must be existing in the data file as data source. The name of this column must match the variable name and syntax.
optional columns
- [Tags] column may be used to add specific tags to a test case. Tags may be comma separated.
- [Documentation] column may be used to add specific test case documentation.
Example Data file
*** Test Cases *** ${username} ${password} [Tags] [Documentation] Right user empty pass demo ${EMPTY} 1 This is a test case documentation of the first one. Right user wrong pass demo FooBar 2,3,foo This test case has the Tags 2,3 and foo assigned. ${EMPTY} mode 1,2,3,4 This test case has a generated name based on template name. ${EMPTY} ${EMPTY} ${EMPTY} FooBar FooBar mode FooBar ${EMPTY} FooBar FooBarIn this data file, eight test cases are defined. Each line specifies one test case. The first two test cases have specific names. The other six test cases will generate names based on template test cases name with the replacement of variables in this name. The order of columns is irrelevant except the first column, *** Test Cases ***
Supported Data Types
In general DataDriver supports any Object that is handed over from the DataReader. However the text based readers for csv, excel and so do support different types as well. DataDriver supports Robot Framework® Scalar variables as well as Dictionaries and Lists. It also support python literal evaluations.
Scalar Variables
The Prefix $ defines that the value in the cell is taken as in Robot Framework® Syntax. String is str, ${1} is int and ${None} is NoneType. The Prefix only defines the value typ. It can also be used to assign a scalar to a dictionary key. See example table: ${user}[id]
Dictionary Variables
Dictionaries can be created in different ways.
One option is, to use the prefix &. If a variable is defined that was [i.e. &{dict}] the cell value is interpreted the same way, the BuiltIn keyword Create Dictionary would do. The arguments here are comma [,] separated. See example table: &{dict}
The other option is to define scalar variables in dictionary syntax like ${user}[name] or ${user.name} That can be also nested dictionaries. DataDriver will create Robot Framework® [DotDict] Dictionaries, that can be accessed with ${user.name.first} See example table: ${user}[name][first]
List Variables
Lists can be created with the prefix @ as comma [,] separated list. See example table: @{list}
Be aware that a list with an empty string has to be the cell content ${Empty}.
Python Literals
DataDriver can evaluate Literals. It uses the prefix e for that. [i.e. e{list_eval}] For that it uses BuiltIn Evaluate
See example table: e{user.chk}
*** Test Cases *** ${scalar} @{list} e{list_eval} &{dict} e{dict_eval} e{eval} ${exp_eval} ${user}[id] ${user}[name][first] ${user.name.last} e{user.chk} One Sum List 1,2,3,4 ["1","2","3","4"] key=value {'key': 'value'} [1,2,3,4] 10 1 Pekka Klärck {'id': '1', 'name': {'first': 'Pekka', 'last': 'Klärck'}} Two Should be Equal a,b,c,d ["a","b","c","d"] key,value {'key': 'value'} True ${true} 2 Ed Manlove {'id': '2', 'name': {'first': 'Ed', 'last': 'Manlove'}} Three Whos your Daddy !,",',$ ["!",'"',"'","$"] z,value,a,value2 {'a': 'value2', 'z': 'value'} {'Daddy' : 'René'} René 3 Tatu Aalto {'id': '3', 'name': {'first': 'Tatu', 'last': 'Aalto'}} 4 Should be Equal 1 ["1"] key=value {'key': 'value'} 1 ${1} 4 Jani Mikkonen {'id': '4', 'name': {'first': 'Jani', 'last': 'Mikkonen'}} 5 Should be Equal [] a=${2} {'a':2} "string" string 5 Mikko Korpela {'id': '5', 'name': {'first': 'Mikko', 'last': 'Korpela'}} 6 Should be Equal [1,2] ["[1","2]"] key=value,key2=value2 {'key': 'value', 'key2': 'value2'} None ${none} 6 Ismo Aro {'id': '6', 'name': {'first': 'Ismo', 'last': 'Aro'}}Accessing Test Data From Robot Variables
If neccesary it is possible to access the fetched data tables directly from a Robot Framework® variable. This could be helpfull in Test Setup or in Suite Setup.
There are three variables available within the Data-Driven Suite:
@{DataDriver_DATA_LIST}
A list as suite variable containing a robot dictionary for each test case that is selected for execution.
This can be accessed as usual in Robot Framework®.
${DataDriver_DATA_LIST}[2][arguments][\\${password}] would result in mode .
&{DataDriver_DATA_DICT}
A dictionary as suite variable that contains the same data as the list, with the test names as keys.
&{DataDriver_TEST_DATA}
A dictionary as test variable that contains the test data of the current test case. This dictionary does also contain arguments that are not used in the Test Template keyword. This can be used in Test Setup and within a test case.
Data Sources
CSV / TSV [Character-separated values]
By default DataDriver reads csv files. With the Encoding and CSV Dialect settings you may configure which structure your data source has.
XLS / XLSX Files
To use Excel file types, you have to install DataDriver with the Extra XLS.
If you want to use Excel based data sources, you may just set the file to the extention or you may point to the correct file. If the extention is ".xls" or ".xlsx" DataDriver will interpret it as Excel file. You may select the sheet which will be read by the option sheet_name. By default it is set to 0 which will be the first table sheet. You may use sheet index [0 is first sheet] or sheet name[case sensitive]. XLS interpreter will ignore all other options like encoding, delimiters etc.
or:
MS Excel and typed cells
Microsoft Excel xls or xlsx file have the possibility to type thair data cells. Numbers are typically of the type float. If these data are not explicitly defined as text in Excel, pandas will read it as the type that is has in excel. Because we have to work with strings in Robot Framework® these data are converted to string. This leads to the situation that a European time value like "04.02.2019" [4th January 2019] is handed over to Robot Framework® in Iso time "2019-01-04 00:00:00". This may cause unwanted behavior. To mitigate this risk you should define Excel based files explicitly as text within Excel.
Alternatively you may deactivate that string conversion. To do so, you have to add the option preserve_xls_types to True. In that case, you will get str, float, boolean, int, datetime.time, datetime.datetime and some others.
PICT [Pairwise Independent Combinatorial Testing]
Pict is able to generate data files based on a model file. //github.com/Microsoft/pict
Documentation: //github.com/Microsoft/pict/blob/master/doc/pict.md
Requirements
- Path to pict.exe must be set in the %PATH% environment variable.
- Data model file has the file extention ".pict"
- Pict model file must be encoded in UTF-8
How it works
If the file option is set to a file with the extention pict, DataDriver will hand over this file to pict.exe and let it automatically generates a file with the extention ".pictout". This file will the be used as data source for the test generation. [It is tab seperated and UTF-8 encoded] Except the file option all other options of the library will be ignored.
Glob File Pattern
This module implements a reader class that creates a test case for each file or folder that matches the given glob pattern.
With an optional argument "arg_name" you can modify the argument that will be set. See folder example.
Example with json files:
Example with folders:
File Encoding and CSV Dialect
CSV is far away from well designed and has absolutely no "common" format. Therefore it is possible to define your own dialect or use predefined. The default is Excel-EU which is a semicolon separated file. These Settings are changeable as options of the Data Driver Library.
file=
- None[default]: Data Driver will search in the test suites folder if a *.csv file with the same name than the test suite *.robot file exists
- only file extention: if you just set a file extentions like ".xls" or ".xlsx" DataDriver will search
- absolute path: If an absolute path to a file is set, DataDriver tries to find and open the given data file.
- relative path: If the option does not point to a data file as an absolute path, Data Driver tries to find a data file relative to the folder where the test suite is located.
encoding=
encoding= must be set if it shall not be cp1252.
Examples:
cp1252, ascii, iso-8859-1, latin-1, utf_8, utf_16, utf_16_be, utf_16_le
cp1252 is:
- Code Page 1252
- Windows-1252
- Windows Western European
Most characters are same between ISO-8859-1 [Latin-1] except for the code points 128-159 [0x80-0x9F]. These Characters are available in cp1252 which are not present in Latin-1.
ƒ ˆ Š Œ Ž ˜ š œ ž Ÿ
See Python Standard Encoding for more encodings
dialect=
You may change the CSV Dialect here. The dialect option can be one of the following: - Excel-EU - excel - excel-tab - unix - UserDefined
supported Dialects are:
Usage in Robot Framework®
Example User Defined
User may define the format completely free. If an option is not set, the default values are used. To register a userdefined format user have to set the option dialect to UserDefined
Usage in Robot Framework®