Who’s Testing What?

Recently I was asked how we could see who is assigned to execute all the tests for this release in Azure Test Plans.

Lets describe the problem using a couple of user stories:

• As a tester I need to be able to easily see what test cases I have been assigned to execute.
• As a test lead I want to be able to generate a report showing which testers are assigned to the test cases in the test plans we’ve created.

We’ll get back to this in a minute.

This article is broken down into the following sections and based on your experience with Azure Test Plans you may want to skip ahead.

• Test Plans — For those who are new to Azure Test Plans, you might want a little background.
• Assigning Testers to execute Test Cases — Perhaps you have worked with azure test plans and don’t need that background. However you would like to better understand how to assign testers to test cases.
• Solutions — I know Azure Test Plans, I know how to assign testers to test cases. I want to jump to the solution.
• The Full Script — I understand the problem, don’t really care how you solved it, let me get the script so I can use it.

Now you can pick your starting point and carry on.

Test Plans

For those new to Azure Test Plans here are the pieces we are going to talk about.

• Test Plans — A plan to track manual testing for a sprint or release
• Test Suites — A group of test cases, usually organized by requirement
• Test Cases — Tests to be executed
• Configurations — The configuration under which to run a test case
• Tester — The tester assigned to execute a test case

Test Plan with assigned Testers

For more details check out these articles:

• Test Cases
• Test Plans
• Test Execution

Assigning Testers to execute Test Cases

To assign testers to a test case in a test suite you can click on the context menu on the test suite in your test plan. I said test too many times there.

Assign Testers to a Test Suite

This will open a dialog where you can select testers to validate all the test cases in this test suite. From this dialog you can also send them an email letting them know they have been assigned to these test cases.

Assign testers to a suite of test cases

NOTE: In the Select testers field you can add multiple testers. When you switch to the execute view in your test plan you will see the test cases duplicated for each tester selected.

This is the same when you add configurations to the test plan. The test cases are replicated for each configuration. Your users may install or run your app on a wide variety of configurations, such as different operating systems, web browsers, and other variations. You will want to run some of your tests in those different configurations.

Test Cases duplicated by configuration

You can also assign a tester right on the test case instance by clicking on the test case context menu. You need to be in the Execute view to do this. You can also select multiple test cases to assign someone to more than one at a time.

Solutions

Now you have testers assigned to all the test cases in your test plans. How can you see who is testing what.

1. Filtering the Test Plan

In Azure DevOps there is one way to see this per test plan. By simply filtering the test plan. Open one of your test plans, and take a look at the test suites. At the end of the test suite name is a number in brackets. That is the number of test cases assigned to this test suite.

Test Suites and Number of Test Cases

To see just the tests you or someone else is assigned to. Switch to the execute view on the test plan, then filter the test plan by tester.

Filter by Tester

Now take a look at the test suites. It should now reflect how many are assigned to you, or whoever you selected. By adding a second number in the brackets (4 of 8) for example means you are assigned to 4 of the 8 test cases in that test suite. All the tests you are not assigned to will be filtered out.

This is nice, not obvious in my opinion, but it works. Also if there were multiple test plans I would have to look at each test plan to know everything I need to test.

2. Using the Rest API to create a report

I have written a PowerShell script with a built in filter to pull this information and generate a markdown document. The bonus is it will report on all the test plans in your team project.

The Report

First lets take a look at the report being generated.

The report is a markdown document that will contain a heading for each test plan where assigned testers were found.

Within that there will be a sub heading for each test suite where assigned testers were found.

And then a table containing all the test cases, with the configuration and the tester assigned to that test case/configuration.

The next section explains how I wrote the script, in case you are hear to learn how to write such a script. If you just want to copy paste the script and use it, and don’t care how it was written jump to the last section The Full Script.

How I wrote the Script

You will need an Azure DevOps Personal Access Token. Create an environment variable with your token, like this:

$env:ADO_PAT="My PAT"

Now for the script, we need some variables to hold the org and project names. Handy if you have multiple Team Projects or even multiple Orgs.

lastPlan and lastSuite are used in the script to check if we are still on the same test suite and test plan as we iterate over test cases. You will see down further.

We need a name for the report we are going to generate, and lets do an initial set-context so that every time we run this it generates a new copy of the report.

$org = "myorg"
$project = "myProject"
$lastPlan = ""
$lastsuite = ""

$report = "$org-$project-AssignedTesters.md"
set-content -path $report -value "# Testers Assigned to Test Cases"

I have also included an array called testers. If you include an element ‘all’ it will return everyone. If you add 1 or more testers names you can have the report only show those testers.

# Just show results for two testers
$testers = @('John Cleese', 'Eric Idle')

# Show all results
$testers = @('all')

The script makes 4 calls to the Azure DevOps Rest API:

1. List Test Plans — Get a list of all the Test Plans in the Team Project

This script gets all the test plans in your Team Project.

$params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/testPlan/plans?api-version=7.1-preview.1' -f $org, $project)
            'Headers'     = @{'Authorization' = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($env:ADO_PAT)"))}
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
$plans = Invoke-RestMethod @params 

2. Get Test Suites for Plan — Get a list of all the Test Suites in a Test Plan

Now we iterate over each test plan. Get the plan name for the report and the plan id to be used in the API call to get all the test suites in that plan.

I also added an if here to check the test plan state and ignore inactive test plans.

foreach ($plan in $plans.value) 
{
    $testPlanName = $plan.name
    $testPlanId = $plan.id
    $testPlanState = $plan.state 
    if ($testPlanState -ne 'Active') {
        write-host "Skipping $testPlanName as it is not active" -ForegroundColor Yellow
        continue
    }
    
    write-host "Plan - $testPlanName" -ForegroundColor Green

    $params = @{'Uri'     = ('https://dev.azure.com/{0}/{1}/_apis/testplan/Plans/{2}/suites?api-version=7.1-preview.1' -f  $org, $project, $testPlanId)
            'Headers'     = @{'Authorization' = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($env:ADO_PAT)"))}
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
    $testsuites = Invoke-RestMethod @params

3. Get Test Cases — Get a list of all the Test Cases in a Test Suite

Now we iterate over the test suites. Get the test suite name for the report and the test suite id can be used in the API call to get all the Test Cases assigned to that test suite.

foreach($testsuite in $testsuites.value) 
    {
        $testsuiteName = $testsuite.name
        $testsuiteID = $testsuite.id

        $params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/test/Plans/{2}/suites/{3}/testcases?api-version=7.1-preview.3' -f $org, $project, $testPlanId, $testsuiteID)
                'Headers'     = @{'Authorization' = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($env:ADO_PAT)"))}
                'Method'      = 'GET'
                'ContentType' = 'application/json'}
        $testcases = Invoke-RestMethod @params

4. Get Workitem — Get the Test Case

This last call is just a work item API call using the test case id to get the test case name for the report.

foreach ($testcase in $testcases.value) {

            $testcaseId = $testcase.testCase.id
            $params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/wit/workitems/{2}?api-version=7.2-preview.3' -f $org, $project, $testcaseId)
            'Headers'     = @{'Authorization' = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($env:ADO_PAT)"))}
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
            $wi = Invoke-RestMethod @params
            $testcaseName = $wi.fields.'System.Title' 

Now for the meat of the script and the actual report generation.

The test case returned from our previous API call will contain the test case id plus a list of point assignments each of which contain a configuration and the assigned tester.

foreach ($pointAssignment in $testcase.pointAssignments) 
            {
                $tester = $pointAssignment.tester.displayName
                $configuration = $pointAssignment.configuration.name

Next we’ll check our testers array to see if this is one of the testers we want to report on

if (($Testers -contains $tester) -or ($Testers -contains 'all')) {

If there are no test cases in this test suite, log a message to that effect and move on the next test case.

if ($testcases.count -eq 0) {
    write-host "No test cases found in test suite $testsuiteName" -ForegroundColor Yellow
}

If there are test cases under this test suite lets fill out our report.

• Log how many test cases were found in this test suite.
• Using lastPlan, check if this is a new test plan. If it is:
  - Add the test plan name to the report
  - set lastPlan = testPlanName
• Using lastSuite, check if this is a new test suite. If it is:
  - Add the test suite name to the report
  - Add the header for a table containing all the test cases
  - Set lastSuite = testSuiteName
• Add a row to the table with test case name, Configuration and tester.

else
{
    write-host "$($testcases.count) test cases found in suite $testsuiteName" -ForegroundColor Green
    if ($lastPlan -ne $testPlanName) 
    {
        $lastPlan = $testPlanName
        add-content -path $report -value "# Test Plan: $testPlanName"
    }

    if ( $lastsuite -ne $testsuiteName) 
    {
        $lastsuite = $testsuiteName
        add-content -path $report -value "## Test Suite: $testsuiteName"
        add-content -path $report -value "|Test Case|configuration|Tester|"
        add-content -path $report -value "|---|---|---|"
    }
}
write-host "$testcaseName - $tester" -ForegroundColor Cyan
Add-Content -Path $report -Value "|$testcaseName|$configuration|$tester|"

As always, I hope someone finds it useful.

The Full Script

I added the script in it’s entirety below, so you can just copy paste and try it out.

$org = "myOrg"
$project = "myProject"

$testers = @('all')
$lastPlan = ""
$lastsuite = ""
$token = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($env:ADO_PAT)"))

$report = "$org-$project-AssignedTesters.md"

set-content -path $report -value "# Assigned Testers"

$params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/testPlan/plans?api-version=7.1-preview.1' -f $org, $project)
            'Headers'     =  @{'Authorization' = ('Basic {0}' -f $token) }
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
$plans = Invoke-RestMethod @params

foreach ($plan in $plans.value) 
{
    $testPlanName = $plan.name
    $testPlanId = $plan.id
    $testPlanState = $plan.state 
    if ($testPlanState -ne 'Active') {
        write-host "Skipping $testPlanName as it is not active" -ForegroundColor Yellow
        continue
    }
    
    write-host "Plan - $testPlanName" -ForegroundColor Green

    $params = @{'Uri'     = ('https://dev.azure.com/{0}/{1}/_apis/testplan/Plans/{2}/suites?api-version=7.1-preview.1' -f  $org, $project, $testPlanId)
            'Headers'     = @{'Authorization' = ('Basic {0}' -f $token) }
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
    $testsuites = Invoke-RestMethod @params
    foreach($testsuite in $testsuites.value) 
    {
        $testsuiteName = $testsuite.name
        $testsuiteID = $testsuite.id

        $params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/test/Plans/{2}/suites/{3}/testcases?api-version=7.1-preview.3' -f $org, $project, $testPlanId, $testsuiteID)
                'Headers'     =  @{'Authorization' = ('Basic {0}' -f $token) }
                'Method'      = 'GET'
                'ContentType' = 'application/json'}
        $testcases = Invoke-RestMethod @params
    
        foreach ($testcase in $testcases.value) {

            $testcaseId = $testcase.testCase.id
            $params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/wit/workitems/{2}?api-version=7.2-preview.3' -f $org, $project, $testcaseId)
            'Headers'     =  @{'Authorization' = ('Basic {0}' -f $token) }
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
            $wi = Invoke-RestMethod @params
            $testcaseName = $wi.fields.'System.Title' 
            foreach ($pointAssignment in $testcase.pointAssignments) 
            {
                $tester = $pointAssignment.tester.displayName
                $configuration = $pointAssignment.configuration.name

                if (($Testers -contains $tester) -or ($Testers -contains 'all')) {

                    if ($testcases.count -eq 0) {
                        write-host "No test cases found in test suite $testsuiteName" -ForegroundColor Yellow
                    }
                    else
                    {
                        write-host "$($testcases.count) test cases found in suite $testsuiteName" -ForegroundColor Green
                        if ($lastPlan -ne $testPlanName) 
                        {
                            $lastPlan = $testPlanName
                            add-content -path $report -value "# Test Plan: $testPlanName"
                        }
                        if ( $lastsuite -ne $testsuiteName) 
                        {
                            $lastsuite = $testsuiteName
                            add-content -path $report -value "## Test Suite: $testsuiteName"
                            add-content -path $report -value "|Test Case|configuration|Tester|"
                            add-content -path $report -value "|---|---|---|"
                        }
                    }
                    write-host "$testcaseName - $tester" -ForegroundColor Cyan
                    Add-Content -Path $report -Value "|$testcaseName|$configuration|$tester|"
                }
            }
        }
    }
}

Publish to Wiki

Here is a little bonus script for you. If you would like to take the report generated in the script above and publish it to your wiki, just add this to the bottom of the script and change the $wikiName and $wikiPagePath to suite your environment.

$wikiName = "myProject.wiki"
$wikiPagePath = 'Test Plan Assignments'
$Headers = $null

$params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/wiki/wikis/{2}/pages?version=wikiMaster&path={3}&api-version=7.1-preview.1' -f $org, $project, $wikiName, $wikiPagePath)
            'Headers'     = @{'Authorization' = ('Basic ' + $token)}
            'Method'      = 'GET'
            'ContentType' = 'application/json'}
$page = Invoke-RestMethod @params -ResponseHeadersVariable ‘Headers’
$etag = $Headers.'ETag'

$content = get-content -path $report -Raw
$body = @{content="$content"}

$params = @{'Uri'         = ('https://dev.azure.com/{0}/{1}/_apis/wiki/wikis/{2}/pages?path={3}&api-version=7.1-preview.1' -f $org, $project, $wikiName, $wikiPagePath)
            'Headers'     =  @{'Authorization' = ('Basic {0}' -f $token);”If-Match”=”$eTag”}
            'Method'      = 'PUT'
            'ContentType' = 'application/json'
            'Body'        = ($body | ConvertTo-Json)}
Invoke-RestMethod @params

The result: