A run is considered failed, if the Appium test produces fails, or the response code isn’t valid, or the expression didn’t match. For page load and get tests also any timeouts are considered fails.
A run is considered errored, if the test could not be run for some reason - either Appium produced errors; the files were corrupted; etc. Check out the error message and logs for further info. It’s also possible that this is an infinitely regrettable infrastructure error. In such a case, please contact firstname.lastname@example.org.
An incident is started, and the check is placed in “warning” state, when one or more of your runs fail. The clock starts, and your action list will start executing. You can stop the execution - for example, before the CTO is awoken via SMS and the team has to incur her wrath - by acknowledging the incident. After an incident is acknowledged, the action list is stopped and no further actions will be taken, but the incident will stay open, and the check in “warning” -state.
The incident is considered resolved, if someone resolves an acknowledged incident, or that all of the runs in the locations of your check start working again. If at least one of them fails - even when it’s a different one that started the incident - the incident is not resolved. You have to get a simultaneous all-clear from all of the locations before the check is considered resolved.
When the check is acknowledged or resolved, the action list is consulted and all the places that were already notified about the incident are sent an all-clear signal. If the acknowledger was fast, and the CTO did not get a warning SMS, then she will not get an “all clear” SMS either, and will happily continue sleeping.