Configure OIDC Provider

In order to avoid having to create long-term credentials for Terraform Cloud to use so that it can make changes to your infrastructure we can configure Terraform Cloud to assume an IAM role instead.

Similarly to the previous chapter, we'll need to create some resources from our local machine until we've configured enough infrastructure to allow Terraform Cloud to take over management duties.

Clone the repository that was created in the last section:

git clone https://github.com/your-github-org/tfcloud-pipeline
cd tfcloud-pipeline

Firstly, we'll create a new module that will create the roles that Terraform Cloud will assume during its plan and apply runs.

Copy and paste the following into a new file, modules/tfcloud_role/variables.tf:

variable "apply_role_arns" {
  description = "List of role ARNs to the workspace's apply policy"
  type        = list(string)
}

variable "environment" {
  description = "Environment"
  type        = string
}

variable "oidc_provider_url" {
  description = "Terraform Cloud OIDC Provider URL"
  type        = string
  default     = "https://app.terraform.io"
}

variable "plan_role_arns" {
  description = "List of role ARNs to the workspace's plan policy"
  type        = list(string)
}

variable "tfcloud_org" {
  description = "Terraform Cloud Organization name"
  type        = string
}

variable "tfcloud_project" {
  description = "Terraform Cloud project name"
  type        = string
}

variable "workspace_name" {
  description = "Terraform Cloud workspace name"
  type        = string
}

Copy and paste the following into a new file, modules/tfcloud_role/main.tf


locals {
  oidc_provider_site_address = replace(data.aws_iam_openid_connect_provider.tfcloud.url, "https://", "")
}

data "aws_iam_openid_connect_provider" "tfcloud" {
  url = var.oidc_provider_url
}

data "aws_iam_policy_document" "tfcloud_pipeline_plan_assume_role" {
  statement {
    effect = "Allow"

    principals {
      type = "Federated"

      identifiers = [
        data.aws_iam_openid_connect_provider.tfcloud.arn,
      ]
    }

    actions = [
      "sts:AssumeRoleWithWebIdentity",
    ]

    condition {
      test     = "StringEquals"
      variable = "${local.oidc_provider_site_address}:aud"
      values   = data.aws_iam_openid_connect_provider.tfcloud.client_id_list
    }

    condition {
      test     = "StringLike"
      variable = "${local.oidc_provider_site_address}:sub"

      values = [
        "organization:${var.tfcloud_org}:project:${var.tfcloud_project}:workspace:${var.workspace_name}:run_phase:plan",
      ]
    }
  }
}

resource "aws_iam_role" "tfcloud_pipeline_plan" {
  name               = "${var.workspace_name}-plan"
  assume_role_policy = data.aws_iam_policy_document.tfcloud_pipeline_plan_assume_role.json
}

resource "aws_iam_role_policy_attachment" "tfcloud_pipeline_plan" {
  count      = length(var.plan_role_arns)
  role       = aws_iam_role.tfcloud_pipeline_plan.name
  policy_arn = var.plan_role_arns[count.index]
}

data "aws_iam_policy_document" "tfcloud_pipeline_apply_assume_role" {
  statement {
    effect = "Allow"

    principals {
      type = "Federated"

      identifiers = [
        data.aws_iam_openid_connect_provider.tfcloud.arn,
      ]
    }

    actions = [
      "sts:AssumeRoleWithWebIdentity",
    ]

    condition {
      test     = "StringEquals"
      variable = "${local.oidc_provider_site_address}:aud"
      values   = data.aws_iam_openid_connect_provider.tfcloud.client_id_list
    }

    condition {
      test     = "StringLike"
      variable = "${local.oidc_provider_site_address}:sub"

      values = [
        "organization:${var.tfcloud_org}:project:${var.tfcloud_project}:workspace:${var.workspace_name}:run_phase:apply",
      ]
    }
  }
}

resource "aws_iam_role" "tfcloud_pipeline_apply" {
  name               = "${var.workspace_name}-apply"
  assume_role_policy = data.aws_iam_policy_document.tfcloud_pipeline_apply_assume_role.json
}

resource "aws_iam_role_policy_attachment" "tfcloud_pipeline_apply" {
  count      = length(var.apply_role_arns)
  role       = aws_iam_role.tfcloud_pipeline_apply.name
  policy_arn = var.apply_role_arns[count.index]
}

With the module in place, we can set up the root module, which will create the OIDC provider and associated IAM roles.

Copy and paste the following into a new file, variables.tf:

variable "environment" {
  type        = string
  description = "Environment"
}

variable "oidc_provider" {
  description = "Terraform Cloud OIDC Provider details"
  type = object({
    url             = string
    site_address    = string
    client_id_list  = list(string)
    thumbprint_list = list(string)
  })
}

variable "region" {
  type        = string
  description = "AWS region"
}

variable "tfcloud_org" {
  type        = string
  description = "Terraform Cloud Organization name"
}

variable "tfcloud_project" {
  type        = string
  description = "Terraform Cloud Project names"
}

variable "workspace_name" {
  description = "Name of workspace"
  type        = string
}

Copy and paste the following into a new file, terraform.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.20.0"
    }
  }

  required_version = "~> 1.6.0"
}

provider "aws" {
  region = var.region
}

Next, copy and paste the following into a new file, main.tf:

locals {
  configure_oidc = var.environment == "pre-dev" && var.region == "eu-west-2"
}

resource "aws_iam_openid_connect_provider" "terraform_cloud" {
  count           = local.configure_oidc ? 1 : 0
  url             = var.oidc_provider.url
  client_id_list  = var.oidc_provider.client_id_list
  thumbprint_list = var.oidc_provider.thumbprint_list
}

module "tfcloud_pipeline_roles" {
  source          = "./modules/tfcloud_role"
  environment     = var.environment
  tfcloud_project = var.tfcloud_project
  tfcloud_org     = var.tfcloud_org
  workspace_name  = var.workspace_name

  plan_role_arns = [
    "arn:aws:iam::aws:policy/IAMReadOnlyAccess",
    "arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess",
  ]

  apply_role_arns = [
    "arn:aws:iam::aws:policy/IAMFullAccess",
    "arn:aws:iam::aws:policy/AmazonVPCFullAccess",
  ]
}

Next, copy and paste the following into a new file, tfcloud_pipeline.auto.tfvars:

environment = "pre-dev"

oidc_provider = {
  client_id_list = [
    "aws.workload.identity",
  ]

  site_address = "app.terraform.io"

  thumbprint_list = [
    "9e99a48a9960b14926bb7f3b02e22da2b0ab7280",
  ]

  url = "https://app.terraform.io"
}

region          = "eu-west-2"
tfcloud_org     = "your-tfcloud-org"
tfcloud_project = "tfcloud-pipeline"

Run terraform init:

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.20.0"...
- Installing hashicorp/aws v5.20.0...
- Installed hashicorp/aws v5.20.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

We'll need to create the roles for every workspace so that Terraform Cloud can assume them for their plan and apply runs. We'll start off in the pre-dev eu-west-2 workspace so that the OIDC provider is created:

$ terraform workspace new tfcloud-pipeline-pre-dev-eu-west-2
Created and switched to workspace "tfcloud-pipeline-pre-dev-eu-west-2"!

You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.

Now we'll create the OIDC provider separately from the rest of the resources. This is required because there's an aws_openid_connect_provider data source in the tfcloud_role; running a terraform apply without a matching data source in AWS will error.

Run terraform apply -target aws_iam_openid_connect_provider.tfcloud to create the OIDC provider in your AWS account:

$ terraform apply -target aws_iam_openid_connect_provider.tfcloud

...
Terraform will perform the following actions:

  # aws_iam_openid_connect_provider.tfcloud[0] will be created
  + resource "aws_iam_openid_connect_provider" "tfcloud" {
      + arn             = (known after apply)
      + client_id_list  = [
          + "aws.workload.identity",
        ]
      + id              = (known after apply)
      + tags_all        = (known after apply)
      + thumbprint_list = [
          + "9e99a48a9960b14926bb7f3b02e22da2b0ab7280",
        ]
      + url             = "https://app.terraform.io"
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Warning: Resource targeting is in effect
│ 
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by
│ the current configuration.
│ 
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵

Do you want to perform these actions in workspace "tfcloud-pipeline-pre-dev-eu-west-2"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_iam_openid_connect_provider.tfcloud[0]: Creating...
aws_iam_openid_connect_provider.tfcloud[0]: Creation complete after 1s [id=arn:aws:iam::012345678901:oidc-provider/app.terraform.io]
╷
│ Warning: Applied changes may be incomplete
│ 
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output
│ values may not be fully updated. Run the following command to verify that no other changes are pending:
│     terraform plan
│  
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors
│ or mistakes, or when Terraform specifically suggests to use it as part of an error message.
╵

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Warning

You may encounter the following error if you haven't configured your AWS credentials correctly from the pre-requisites section:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: No valid credential sources found
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on terraform.tf line 10, in provider "aws":
│   10: provider "aws" {
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error
│ ec2imds: GetMetadata, request canceled, context deadline exceeded

If you haven't saved credentials in your default profile, you may need to export AWS_PROFILE=your-profile-name prior to running terraform apply

With the OIDC provider now in place, we can run a full terraform apply to create the IAM roles and attach the required policies to them:

$ terraform apply
module.tfcloud_pipeline_roles.data.aws_iam_openid_connect_provider.tfcloud: Reading...
aws_iam_openid_connect_provider.tfcloud[0]: Refreshing state... [id=arn:aws:iam::320797911953:oidc-provider/app.terraform.io]
module.tfcloud_pipeline_roles.data.aws_iam_openid_connect_provider.tfcloud: Read complete after 0s [id=arn:aws:iam::320797911953:oidc-provider/app.terraform.io]
module.tfcloud_pipeline_roles.data.aws_iam_policy_document.tfcloud_pipeline_plan_assume_role: Reading...
module.tfcloud_pipeline_roles.data.aws_iam_policy_document.tfcloud_pipeline_apply_assume_role: Reading...
module.tfcloud_pipeline_roles.data.aws_iam_policy_document.tfcloud_pipeline_apply_assume_role: Read complete after 0s [id=595414985]
module.tfcloud_pipeline_roles.data.aws_iam_policy_document.tfcloud_pipeline_plan_assume_role: Read complete after 0s [id=4136116602]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:
...
module.tfcloud_pipeline_roles.aws_iam_role.tfcloud_pipeline_apply: Creating...
module.tfcloud_pipeline_roles.aws_iam_role.tfcloud_pipeline_plan: Creating...
module.tfcloud_pipeline_roles.aws_iam_role.tfcloud_pipeline_apply: Creation complete after 1s [id=tfcloud-pipeline-pre-dev-eu-west-2-apply]
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_apply[0]: Creating...
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_apply[1]: Creating...
module.tfcloud_pipeline_roles.aws_iam_role.tfcloud_pipeline_plan: Creation complete after 1s [id=tfcloud-pipeline-pre-dev-eu-west-2-plan]
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_plan[1]: Creating...
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_plan[0]: Creating...
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_apply[0]: Creation complete after 0s [id=tfcloud-pipeline-pre-dev-eu-west-2-apply-20231008194953744100000001]
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_apply[1]: Creation complete after 0s [id=tfcloud-pipeline-pre-dev-eu-west-2-apply-20231008194953786500000002]
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_plan[1]: Creation complete after 0s [id=tfcloud-pipeline-pre-dev-eu-west-2-plan-20231008194953905700000003]
module.tfcloud_pipeline_roles.aws_iam_role_policy_attachment.tfcloud_pipeline_plan[0]: Creation complete after 0s [id=tfcloud-pipeline-pre-dev-eu-west-2-plan-20231008194954014200000004]

Create Roles for all Workspaces

We'll now need to go around the remaining workspaces and create the roles necessary for Terraform Cloud to be able to run correctly.

$ terraform workspace new tfcloud-pipeline-dev-eu-west-2
$ sed -i 's/pre-dev/dev/' tfcloud_pipeline.auto.tfvars
$ terraform apply
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

$ terraform workspace new tfcloud-pipeline-test-eu-west-2
$ sed -i 's/dev/test/' tfcloud_pipeline.auto.tfvars
$ terraform apply
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

$ terraform workspace new tfcloud-pipeline-prod-eu-west-2
$ sed -i 's/test/prod/' tfcloud_pipeline.auto.tfvars
$ terraform apply
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Migrate Local State To Terraform Cloud

Just as we had to in the last chapter, we'll need to let Terraform Cloud take ownership of the state information for the above resources. We can let Terraform migrate all of our local workspaces in one go by using tags instead of explicitly naming the target workspace.

Edit terraform.tf, adding the following inside the existing terraform {} block:

terraform {
  ...
  cloud {
    organization = "your_tfcloud_org_name"

    workspaces {
      tags = [
        "tfcloud-pipeline",
      ]
    }
  }
}

Initialize Terraform again, and confirm that you want the current (local) state to be migrated to the Terraform Cloud Workspace:

$ terraform init

Initializing Terraform Cloud...
Would you like to rename your workspaces?
  Unlike typical Terraform workspaces representing an environment associated with a particular
  configuration (e.g. production, staging, development), Terraform Cloud workspaces are named uniquely
  across all configurations used within an organization. A typical strategy to start with is
  <COMPONENT>-<ENVIRONMENT>-<REGION> (e.g. networking-prod-us-east, networking-staging-us-east).
  
  For more information on workspace naming, see https://www.terraform.io/docs/cloud/workspaces/naming.html
  
  When migrating existing workspaces from the backend "local" to Terraform Cloud, would you like to
  rename your workspaces? Enter 1 or 2.
  
  1. Yes, I'd like to rename all workspaces according to a pattern I will provide.
  2. No, I would not like to rename my workspaces. Migrate them as currently named.

  Enter a value: 2

Migration complete! Your workspaces are as follows:
  tfcloud-pipeline-dev-eu-west-2
  tfcloud-pipeline-pre-dev-eu-west-2
* tfcloud-pipeline-prod-eu-west-2
  tfcloud-pipeline-test-eu-west-2

Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v5.20.0

Terraform Cloud has been successfully initialized!

You may now begin working with Terraform Cloud. Try running "terraform plan" to
see any changes that are required for your infrastructure.

If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.

With the state migration complete, you can safely remove the cloud{} block that was temporarily added above; this is ignored by Terraform Cloud when the workspace is being managed by a VCS integration like GitHub.

Commit Code

As with the tfcloud-mgmt repository, the main branch in the tfcloud-pipeline repository is protected from being pushed to directly. Commit your code to a branch:

git checkout -b oidc-provider
git add .
git commit -m "Add OIDC provider and roles"
git push

This time, when you open a PR in GitHub, you should see 4 Terraform Cloud pipelines run, and all should report no changes being required. Go ahead and approve and merge the PR.

Terraform Cloud Path to Production Guide

Configure OIDC Provider

Create Roles for all Workspaces

Migrate Local State To Terraform Cloud

Commit Code