Helm provider is Unable to Upgrade Resource State to v3

[batagy]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: OpenTofu v1.10.6
Terragrunt version: v0.86.2
Provider version: 3.0.2
Kubernetes version: 1.31

Affected Resource(s)

• helm_release

Terraform Configuration Files

resource "helm_release" "redis" {
    atomic                     = false
    chart                      = "redis"
    cleanup_on_fail            = false
    create_namespace           = false
    dependency_update          = false
    disable_crd_hooks          = false
    disable_openapi_validation = false
    disable_webhooks           = false
    force_update               = false
    id                         = "oauth2proxy-redis"
    lint                       = false
    max_history                = 0
    metadata                   = [
        {
            app_version = "6.2.7"
            chart       = "redis"
            name        = "oauth2proxy-redis"
            namespace   = "oauth2proxy"
            revision    = 1
            values      = jsonencode(
                {
                    architecture   = "standalone"
                    global         = {
                        redis        = {
                            password = "password"
                        }
                        storageClass = "encrypted-gp3-csi"
                    }
                    image          = {
                        registry = "company.com/dockerhub-remote"
                    }
                    master         = {}
                    serviceAccount = {
                        create = true
                        name   = "oauth2proxy-redis-master"
                    }
                }
            )
            version     = "16.13.2"
        },
    ]
    name                       = "oauth2proxy-redis"
    namespace                  = "oauth2proxy"
    recreate_pods              = false
    render_subchart_notes      = true
    replace                    = false
    repository                 = "https://company.com/artifactory/proj-connx-helm"
    repository_password        = (sensitive value)
    repository_username        = "username"
    reset_values               = false
    reuse_values               = false
    skip_crds                  = false
    status                     = "deployed"
    timeout                    = 60
    values                     = [
        (sensitive value),
    ]
    verify                     = false
    version                    = "16.13.2"
    wait                       = true
    wait_for_jobs              = false
}

Debug Output

Please let me know if debug log cannot be avoided.

Steps to Reproduce

1. Change from helm provider from 2.5.1 to 3.0.2
2. Update kubernetes configuration object (from kubernetes { to kubernetes = { )
3. terragrunt init --upgrade
4. terragrunt plan

Expected Behavior

terragrunt plan and terragrunt apply (tofu plan) should upgrade the helm_release Resource State to newer schema.

Actual Behavior

Terraform is unable to Upgrade Resource State. Below error message is received:

16:12:25.262 STDERR tofu: │ This resource was implemented with an UpgradeState() method, however
16:12:25.262 STDERR tofu: │ Terraform was expecting an implementation for version 0 upgrade.
16:12:25.262 STDERR tofu: │
16:12:25.262 STDERR tofu: │ This is always an issue with the Terraform Provider and should be reported
16:12:25.263 STDERR tofu: │ to the provider developer.

The complete log of action:

username@LocalPC:/mnt/c/GIT/CI CD/AWS Deployments/nddp-iac/live/nddp-infradev2/5-oauth2proxy$ terragrunt plan
16:12:03.438 INFO   Terragrunt Cache server is listening on 127.0.0.1:42501
16:12:03.439 INFO   Start Terragrunt Cache server
16:12:23.649 STDOUT tofu: module.oauth2proxy.random_password.redis_password: Refreshing state... [id=none]
16:12:23.650 STDOUT tofu: module.oauth2proxy.random_password.cookie_secret: Refreshing state... [id=none]
16:12:23.940 STDOUT tofu: data.aws_eks_cluster_auth.cluster: Reading...
16:12:23.940 STDOUT tofu: data.aws_eks_cluster.cluster: Reading...
16:12:23.942 STDOUT tofu: data.aws_eks_cluster_auth.cluster: Read complete after 0s [id=nddp-infradev2-eks]
16:12:24.254 STDOUT tofu: data.aws_eks_cluster.cluster: Read complete after 0s [id=nddp-infradev2-eks]
16:12:24.372 STDOUT tofu: module.oauth2proxy.kubernetes_namespace.oauth2proxy: Refreshing state... [id=oauth2proxy]
16:12:24.844 STDOUT tofu: module.oauth2proxy.kubernetes_secret.oauth2proxy-secret: Refreshing state... [id=oauth2proxy/oauth2proxy-secret]
16:12:24.845 STDOUT tofu: module.oauth2proxy.kubernetes_secret.ingress_tls_secret[0]: Refreshing state... [id=oauth2proxy/oauth2proxy-tls-secret]
16:12:25.261 STDOUT tofu: OpenTofu used the selected providers to generate the following execution
16:12:25.261 STDOUT tofu: plan. Resource actions are indicated with the following symbols:
16:12:25.261 STDOUT tofu:   ~ update in-place
16:12:25.261 STDOUT tofu: OpenTofu planned the following actions, but then encountered a problem:
16:12:25.261 STDOUT tofu:   # module.oauth2proxy.kubernetes_secret.ingress_tls_secret[0] will be updated in-place
16:12:25.261 STDOUT tofu:   ~ resource "kubernetes_secret" "ingress_tls_secret" {
16:12:25.261 STDOUT tofu:         id                             = "oauth2proxy/oauth2proxy-tls-secret"
16:12:25.261 STDOUT tofu:       + wait_for_service_account_token = true
16:12:25.261 STDOUT tofu:         # (3 unchanged attributes hidden)
16:12:25.261 STDOUT tofu:         # (1 unchanged block hidden)
16:12:25.261 STDOUT tofu:     }
16:12:25.261 STDOUT tofu:   # module.oauth2proxy.kubernetes_secret.oauth2proxy-secret will be updated in-place
16:12:25.261 STDOUT tofu:   ~ resource "kubernetes_secret" "oauth2proxy-secret" {
16:12:25.262 STDOUT tofu:         id                             = "oauth2proxy/oauth2proxy-secret"
16:12:25.262 STDOUT tofu:       + wait_for_service_account_token = true
16:12:25.262 STDOUT tofu:         # (3 unchanged attributes hidden)
16:12:25.262 STDOUT tofu:         # (1 unchanged block hidden)
16:12:25.262 STDOUT tofu:     }
16:12:25.262 STDOUT tofu: Plan: 0 to add, 2 to change, 0 to destroy.
16:12:25.262 STDOUT tofu:
16:12:25.262 STDERR tofu: ╷
16:12:25.262 STDERR tofu: │ Error: Unable to Upgrade Resource State
16:12:25.262 STDERR tofu: │
16:12:25.262 STDERR tofu: │   with module.oauth2proxy.helm_release.redis,
16:12:25.262 STDERR tofu: │   on .terraform/modules/oauth2proxy/generic/kubernetes/k8s-oauth2proxy/redis.tf line 7, in resource "helm_release" "redis":
16:12:25.262 STDERR tofu: │    7: resource "helm_release" "redis" {
16:12:25.262 STDERR tofu: │
16:12:25.262 STDERR tofu: │ This resource was implemented with an UpgradeState() method, however
16:12:25.262 STDERR tofu: │ Terraform was expecting an implementation for version 0 upgrade.
16:12:25.262 STDERR tofu: │
16:12:25.262 STDERR tofu: │ This is always an issue with the Terraform Provider and should be reported
16:12:25.263 STDERR tofu: │ to the provider developer.
16:12:25.263 STDERR tofu: ╵
16:12:25.361 ERROR  tofu invocation failed in /home/username/terragrunt-cache/xUxlVSYh7KSdDryyapxo-Y_yYLg/PVGIMcb89Mf4K_ZwTzpdbQP6qS4/aws/stages/oauth2proxy
16:12:25.361 INFO   Shutting down Terragrunt Cache server...
16:12:25.361 INFO   Terragrunt Cache server stopped
16:12:25.362 ERROR  error occurred:

* Failed to execute "tofu plan" in /home/username/terragrunt-cache/xUxlVSYh7KSdDryyapxo-Y_yYLg/PVGIMcb89Mf4K_ZwTzpdbQP6qS4/aws/stages/oauth2proxy
  ╷
  │ Error: Unable to Upgrade Resource State
  │
  │   with module.oauth2proxy.helm_release.redis,
  │   on .terraform/modules/oauth2proxy/generic/kubernetes/k8s-oauth2proxy/redis.tf line 7, in resource "helm_release" "redis":
  │    7: resource "helm_release" "redis" {
  │
  │ This resource was implemented with an UpgradeState() method, however
  │ Terraform was expecting an implementation for version 0 upgrade.
  │
  │ This is always an issue with the Terraform Provider and should be reported
  │ to the provider developer.
  ╵

  exit status 1

Important Factoids

We use terragrunt and opentofu. Versions listed above.

References

I'm aware of below existing issue, but no sure if that is exactly the same issue. Because I get a different error message. But this maybe related:

• provider/v3 unable to read prior state #1574

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[giacomolanciano]

I managed to unblock the provider upgrade by first removing and importing again the helm_release in the state

[RazCrimson]

Based on the error message we can conclude that the tf state is using version 0 of the state schema for the helm provider and UpgradeState is not defined for upgrading from version 0. (It is defined for version 1 to version 2)

So solution I got working is, instead of a full nuke and import is to upgrade in steps:

• Upgrade helm provider to 2.17.0 (Probably any release where they added version 1 of state schema will work)
• Refresh the helm_release resource state (State schema updated to version 1)
• Upgrade helm provider to 3.0.x (3.0.x releases upgrade state schema to version 2)
• Refresh the helm_release resource state (State schema updated to version 2)

Note: Make sure to change the provider blocks for kubernetes ({ -> = {) and vice versa when switching major versions of the helm provider.

We have a very large number of resources, so trying to remove and import is quite tedious compared to just refreshing.

[huziahmetovsv]

• schema

work for me. But i neet to run "terrafrunt apply" on 2.17.0 version to update state. And only after that error is gone