GitHub/EMU/Terraform Update

About a year and half ago, I wrote an article about using Terraform to manage teams in a GitHub Enterprise with Enterprise Managed Users enabled. Here’s the link GitHub/EMU/Terraform.

Background

A bit about Enterprise Managed Users from the documentation.

“You can centrally manage identity and access for your enterprise members on GitHub from your identity provider.

With Enterprise Managed Users, you can control the user accounts of your enterprise members through your identity provider (IdP). Users assigned to the GitHub Enterprise Managed User application in your IdP are provisioned as new user accounts on GitHub and added to your enterprise. You control usernames, profile data, team membership, and repository access for the user accounts from your IdP.”

Full Documentation is here About Enterprise Managed Users — GitHub Enterprise Cloud Docs

In my case the Idp was Azure Active Directory.

This is an awesome addition to GitHub. In a nutshell once this is set up you can add a team to the GitHub Enterprise Managed User application in AAD. After it’s provisioned as an external group in GitHub Enterprise, you can link it to a GitHub team. Then you just add people to a group in AAD and they are automatically added to the corresponding GitHub team.

Automation with Terraform

You can use Terraform to create the AAD Group and the GitHub Team. However when I wrote the previous article, there was no way to use Terraform to map the AAD group and GitHub Teams that were created. Therefore I supplied a script that you could use to do this after the fact.

Since that time the GitHub provider has had some updates and there is now a resource called github_emu_group_mapping that can be used to link the AAD Group to the GitHub Team. However there is still one issue.

When you create a group in AAD, it will get added to your GitHub enterprise as an External Idp Group. However this takes time. Approximately 20-40 minutes. The official Azure Active Directory messaging says you need to wait up to 40 minutes before the team will show up as an external team in GitHub that can be linked to a GitHub team. In my experience it’s often around 20 minutes. But still how do you create a Terraform module for this.

The Solution

We solved this by creating a terraform module that does the following:

1. Create the AAD Group (azuread_group)
2. Create a GitHub Team (github_team)
3. Query the External Teams in GitHub until your new AAD group has been provisioned.
4. Map the External Group to the GitHub Team

I won’t bother showing items 1 and 2. They are well documented here AAD Group and here GitHub Team. I am just going to show you the tricky parts 3 and 4.

Querying for the External Team

The GitHub Provider has a Data source that will return a list of external Idp Groups in GitHub. The Data Source is github_external_groups.

NOTE: Be careful here, you need to use version 5.25.1 or higher of the integrations/terraform-provider-github. Prior to this there was a bug in the github_external_groups data source, it did not do pagination and only returned 30 groups.

Before we can call the Data Source we need to wait for the provisioning to happen, which as I mentioned could take up to 40 minutes.

To ensure the external group is there when we call github_external_groups we can add a null resource with a local-exec provisioner that runs a script to wait for the external group to be added before we proceed. In this instance we used a bash script.

resource "null_resource" "wait_for_idp" {
  provisioner "local-exec" {
    command = <<EOC
      found=false
      count=0
      echo "Waiting for external group ${var.team_name} to be provisioned into {org}"
      until [ "$found" = "true" ]; do
        # Query the external group with the team name that was added
        externalgroup=$(curl -H "Authorization: Bearer $GITHUB_TOKEN" -sSf https://api.github.com/orgs/{org}/external-groups?display_name=${var.team_name} | jq '.groups[0].group_name')
        if [ "$externalgroup" = "null" ]
        then
          # We don't want an endless loop in our Terraforom, therefore
          # if something goes wrong with provisioning lets exit after 1 hour.
          if [ $count -gt 3600 ]
          then
            echo "Waited 60 minutes for external group ${var.team_name} to be provisioned. Exiting..."
            exit 1
          fi
          echo "Still waiting..."
          sleep 30
          count=$((count+30))
        else
          # The group is in the list so we can continue
          echo "External Group ${var.team_name} Found. Begin mapping to GitHub Team ${var.team_name}"
          found=true
        fi
      done
    EOC
    interpreter = ["/bin/bash", "-c"]
  }
  # Before we can run this script which waits for the external group to be created, we need to ensure that the AAD Groups have been created
  depends_on = [ module.aad_security_groups ]
}

Now that we know our AAD group is in the list of external groups in GitHub, we can use the Data Source to get the list of external groups.

Adding a depends_on the null_resource will force Terraform to wait.

data "github_external_groups" "idp_external_groups" { 
  # Before we get the list of external groups we need to wait for the external group to be added
  depends_on = [null_resource.wait_for_idp]
}

Finally we can call the github_external_groups resource to do the mapping.

resource "github_emu_group_mapping" "emu_group_mapping" {
  team_slug = lower("${var.team_name}")
  group_id = data.github_external_groups.idp_external_groups.external_groups[index(data.github_external_groups.idp_external_groups.external_groups[*].group_name, "${var.team_name}")].group_id
}