Retrying Selenium Tests

Selenium tests are dreamy: how spectacular that your whole UI can be given a right thrashing on a variety of browsers! But intermittent Selenium test failures… now, they are truly the bane of one’s existence, especially when they hold up progress in a continuous build environment.

So enter: the retry test listener.

In my company’s suite of Selenium tests, there’s a small percentage that always seem to fail intermittently. We mark those tests as retry candidates, and if they do fail, they’ll be given another shot at passing.

To get something similar set up, firstly, implement an ITestListener like so:

public class RetrySeleniumTestListener implements ITestListener {
    private static final Logger LOG = LoggerFactory.getLogger(RetrySeleniumTestListener.class);

    @Override
    public void onTestFailure(final ITestResult result) {
        final IRetryAnalyzer retryAnalyzer = result.getMethod().getRetryAnalyzer();
        if (retryAnalyzer != null && retryAnalyzer instanceof RetrySeleniumTestCounter && ((RetrySeleniumTestCounter) retryAnalyzer).isOnFirstRun()) {
            LOG.info("Test failed on first run and will be marked as skipped: " + result);
            // You might ask: why not set this result as skipped in RetrySeleniumTestCounter? 
            // That's too late and the change in status is ignored. 
            // Have to do it here instead.
            result.setStatus(ITestResult.SKIP);
        }
    }

    /*
        Here would be the the implementations of the other methods of ITestListener
     */
}

Next up, implement an IRetryAnalyzer like so:

public class RetrySeleniumTestCounter implements IRetryAnalyzer {

    private int retryCount;

    @Override
    public boolean retry(final ITestResult result) {
        if (retryCount < 1) {
            retryCount++;
            return true;
        }
        return false;
    }

    public boolean isOnFirstRun() {
        return retryCount == 0;
    }
}

And then annotate your intermittently failing test methods and test classes appropriately like so:

@Listeners({ RetrySeleniumTestListener.class })
public class DataExtractsSeleniumTest extends AdmanagerSeleniumTestCase {

    @Test(retryAnalyzer = RetrySeleniumTestCounter.class)
    public void testCannotAccessFunctionalityForUsers() {
        // Contents of intermittently failing test method
    }
}

Selenium then knows to apply the test listener to the annotated class, and the retry counter is then applied to the annotated methods.

And voilà, those intermittently failing tests now get a second chance at passing.

Git Blame: List a Particular Author’s Files Across a Code Base

My goal: to have git tell me which files across the breadth of our current code base have lines of code in them that are attributed to a particular author.

My googling for a script came up with bubkes, so I decided to write my own and share that on the internet. No doubt there’s a better way of doing this, and no doubt there’s a better bash script possible (my bash scripting skills are rudimentary at best), but what I did does do the job.

Here’s the bash script:

#!/bin/bash

matchGitBlame() {

	if [[ "$1" == *$2* ]]
	then
		local count=`grep -o "$2" <<< "$1" | wc -l`

		if [ $count -gt 0 ]
		then
			echo "$count : $2 : $3"
		fi
	fi
}

files=`find . -type f`
name=$1
name2=$2

for file in $files
do
	blame=`git blame $file`
	matchGitBlame "$blame" "$name" "$file"
	matchGitBlame "$blame" "$name2" "$file"
done

If you execute the script like so (the script works with one or two author names as attributes):
./blame-script.sh authorName1 authorName2 > blame-script-results.txt && sort blame-script-results.txt -rVo blame-script-results.txt

you should find in the file named blame-script-results.txt something like the following:

180 : authorName1 : ./Breadcrumb.java
43 : authorName1 : ./ManagementTrail.java
41 : authorName1 : ./DashboardTrail.java
24 : authorName2 : ./NavigationTabsPanel.java
17 : authorName1 : ./DropDownMenuPanel.java
5 : authorName2 : ./ManageReportDropDownMenuPanel.java
1 : authorName1 : ./CrumbItem.java