Run Gauge specifications¶

Run a specification¶

You can run a Gauge specification by using the gauge run command. When this command is run, Gauge scans the directories and sub-directories at <project_root> (location at which the Gauge project is created) and picks up valid specification files.

You can use the following command at <project_root> to run a Gauge specification:

gauge run [args] [flags]

• <project_root> - location at which a Gauge project is created

• [args] - directories in which specifications are stored, location of specification files and scenarios

• [flags] - options that can be used with this command such as --tags, -e, -f, and so on

For more information about gauge run command, see the Gauge manual page.

Multiple arguments passed to gauge run¶

You can choose to run specifications at the directory, file, or scenario level or a combination of these by passing appropriate arguments to the gauge run command. The arguments are separated with a space when used with the command.

The arguments can be one of the following:

• Directories and sub-directories that contain specifications

• Path to specification files and scenarios

• Combination of the arguments mentioned in this list

Run specifications by using directory(s) as an argument¶

Specifications can be run from a single directory, sub-directories, or multiple directories. You can specify the directory(s) or path to sub-directory(s) with the gauge run command.

To run all the specifications in the specs directory, use the following command:

gauge run specs

To run specifications at the sub-directory level, use the following command:

gauge run <path_dir>

<path_dir> - location of the sub-directory where specifications are stored

To run specifications stored in multiple directories (including sub-directories), use the following command:

gauge run <dir1> <path_dir2> <path_dir3>

<path_dir2> <path_dir3> - location of the directories where specifications are stored

Example - single directory

In the following example, specifications stored at test_specs are run:

gauge run test_specs

Example - multiple directories

In the following example, specifications stored at specs and test_specs are run:

gauge run specs test_specs

Example - sub-directories

In the following example, specifications stored at sub1_specs and sub2_specs are run:

gauge run specs/sub1_specs specs/sub2_specs

sub1_specs, sub2_specs - directories located in specs

Run specifications by using spec file path as argument¶

You can choose and run only certain specifications by providing the appropriate location of these specifications with the gauge run command.

To run a single specification, use the following command:

gauge run <path_to_spec>

<path_to_spec> - location of the specification

To run multiple specifications, use the following command:

gauge run <path_to_spec1> <path_to_spec2> <path_to_spec3>

Example - run a single specification file

In the following example, Gauge runs the example.spec stored in the specs directory:

gauge run specs/example.spec

Example - run multiple specification files

In the following example, Gauge runs multiple specifications stored in specs and its sub-directory, sub1_specs:

gauge run specs/example.spec specs/sub1_specs/sub1_example.spec

Run specifications by using path to scenario as argument¶

You can choose to run only scenarios of a specification. This can be either a single scenario or multiple scenarios. The argument to the gauge run command can be a specific scenario or a list of scenarios and the line number in the span of that scenario. Any line number which the scenario spans across can be used.

To run a single scenario, use the following command:

gauge run <specification_path>:<scenario_line_number>

To run multiple scenarios, use the following command:

gauge run <specification_path>:<scenario_line_number> <specification_path>:<scenario_line_number>...

Example

Consider the following specification, spec1.spec located at specs directory:

1# Configuration
2
3## Admin Login
4* User must login as "admin"
5* Navigate to the configuration page
6
7## User Login
8* User must login as "user1"
9* Navigation to configuration page is restricted.

gauge run specs/spec1.spec:7

gauge run specs/spec1.spec:9

1# Search specification
2
3## Successful search
4* Log in as "admin"
5* Open the product search page
6* Search for product "Cup Cakes"
7* "Cup Cakes" should show up in the search results

gauge run specs/spec1.spec:3 specs/test1.spec:4

Filter specifications and scenarios by using tags¶

Tags allow filtering specs and scenarios. You can use the --tags flag with the gauge run command to filter specs and scenarios.

Use the following command to filter a Gauge specification or scenario by using tags:

gauge run --tags "Tag_Name" specs

Tag_Name - tag associated with the specs or scenarios

When this command is run, only the scenarios and specifications which are tagged with Tag_Name are executed.

Example

Consider the following spec in the specs directory, which has tags search and admin. The scenario, Successful search, is tagged with successful and the scenario, Unsuccessful search has no tags.

 1# Search Specification
 2
 3The admin user must be able to search for available
 4products on the search page.
 5
 6Tags: search,  admin
 7
 8* User must be logged in as "admin"
 9* Open the product search page
10
11## Successful search
12
13Tags: successful
14
15For an existing product name, the search result will
16contain the product name.
17
18* Search for product "Cup Cakes"
19* "Cup Cakes" should show up in the search results
20
21## Unsuccessful search
22
23On an unknown product name search, the search results
24will be empty
25
26* Search for product "unknown"
27* The search results will be empty

If the tag search is used, then the entire spec is run.

gauge run --tags "search" specs

If the tag successful is used, the scenario, Successful search is run.

gauge run --tags "successful" specs

Execution hooks can also be filtered based on tags. For more information, see Write Gauge specifications.

Tag expressions¶

Tags can be used with expressions. This helps you search and filter specs and scenarios effectively. The following table lists the tags and expressions and their corresponding action while selecting specs and scenarios.

Example

Consider the spec of the previous example - if all the scenarios tagged with search and successful must be run, then use the following command:

gauge run --tags "search & successful" specs

Based on the Tags expressions table, Successful Search scenario is run.

Verbose reporting¶

By default, Gauge generates reports that display whether a spec has passed or failed when executed. To ease debugging, reports can also be generated for every step to display whether a step has been executed successfully. Such reports can be generated by using the --verbose flag. These reports are generated on the console.

Use the following command to generate reports at the step level:

gauge run --verbose specs

Data driven execution¶

A data table is defined in Markdown table format at the beginning of the spec prior to steps. The data table should have a header row and one or more data rows. The header names from the table can be used in the steps within angular brackets < > to refer to a particular column from the data table as a parameter.

When a spec is run, each scenario is executed for every data row of the table. Table parameters are written in Multi-Markdown table formats.

Example

In the following specification hello.spec, the data table is defined at the beginning of the spec. The step uses the name column from the data table as a dynamic parameter. When the spec is run, Scenario and Second Scenario are executed first for the first row values 1, Alice followed by the second and third row values from the table.

 1# Table driven execution
 2
 3     |id| name    |
 4     |--|---------|
 5     |1 |Alice    |
 6     |2 |Bob      |
 7     |3 |Eve      |
 8
 9## Scenario
10* Say "hello" to <name>
11
12## Second Scenario
13* Say "namaste" to <name>

gauge run --table-rows "1" specs/hello.spec

gauge run --table-rows "1,3" specs/hello.spec

gauge run --table-rows "1-3" specs/hello.spec

id,name
1,Alice
2,Bob
3,Eve

1# Table driven execution
2
3table: /system/users.csv
4
5## Scenario
6* Say "hello" to <name>
7
8## Second Scenario
9* Say "namaste" to <name>

├── data
├── env
  └── default
     └── default.properties
  └── qa
     └── qa.properties
  └── uat
     └── uat.properties

gauge_data_dir = data/qa

gauge_data_dir = data/uat

1# Table driven execution
2
3<table:users.csv>
4
5## Scenario
6* Say "hello" to <name>
7
8## Second Scenario
9* Say "namaste" to <name>

Parallel execution¶

Specs can be executed in parallel to run the tests faster. Running tests in parallel creates a number of execution streams depending on the number of CPU cores available on your system and distributes the load among worker processes.

The number of parallel execution streams can be specified by using the -n flag.

Use the following command to run specs in parallel:

gauge run --parallel specs

OR

gauge run -p specs

Example

In the following example, four parallel execution streams are created.

gauge run --parallel -n=4 specs

Run specs before parallel execution¶

Experimental Feature

Specs can be filtered based on tags to run in parallel, so that only those will be run in parallel after executing other specs in serial.

Set allow_filtered_parallel_execution variable to true in /env/default/default.properties to enable this feature.

Execution can be done by the command:

gauge run specs --parallel --only "parallelizable"

This runs all the specs that are not tagged as parallelizable in serial first, and then runs the tagged ones in parallel.

Test suite execution by using the --strategy option¶

The --strategy option allows you to set the strategy for parallel execution of tests. This option has two values: lazy and eager. By default, the option is set to lazy.

gauge run -n=4 --strategy="lazy" specs

gauge run -n=4 specs

gauge run -n=4 --strategy="eager" specs

Executing a group of specification¶

Gauge sorts the specifications by alphabetical order and then distributes these specifications into groups. You can use the --group | -g flag to execute a specific group of specs.

Executing specs with the -n and --g flags ensures that Gauge executes the same group of specifications in the same order regardless of the number of times the gauge run command is executed.

Use the following command to execute a group of specifications:

gauge run -n=<number_of_groups> -g=<group_number> specs

• -n - number of groups

• -g - group number

Example

In the following example, Gauge creates four groups of specification and selects the second group for execution.

gauge run -n=4 -g=2 specs

Even if the command is run multiple times, Gauge still executes the same group of specifications in the same order.

Rerun failed scenarios¶

Gauge provides the ability to rerun only the scenarios which failed in the previous execution. You can use the --failed flag with the gauge run command to rerun failed scenarios.

Use the following command to rerun failed scenarios:

gauge run --failed

For example, if three scenarios failed during gauge run specs, the failed scenarios can be rerun instead of executing all scenarios.

When the --failed flag is used with the gauge run command, the flags that were set during the previous execution is once again set.

Example

Consider an example, where specs are run with a --env and --verbose flags. Three scenarios fail during this run.

gauge run --env="chrome" --verbose specs

To rerun only the failed scenarios, use the following command:

gauge run --failed

When this command is run, Gauge internally sets the --env and --verbose flags to corresponding values used in the previous execution. Hence, gauge run --failed is equivalent to the following command:

gauge run --env="chrome" --verbose specs <path_to_failed_scenarios>

gauge run --max-retries-count=<number of retries>

gauge run --max-retries-count=3

gauge run --max-retries-count=<number of retries> --retry-only "<tag_name>"

gauge run --max-retries-count=3 --retry-only="should-retry"

Errors during execution¶

[ParseError] hello_world.spec : line no: 25, Dynamic parameter <product> could not be resolved

[ValidationError] login.spec:33: Step implementation not found. login with "user" and "p@ssword"

[ValidationError] foo.spec:11 Duplicate step implementation => 'Vowels in English language are <table>'