What better way to commemorate the one-year anniversary of setting up a website?

$ curl https://0x1b.me
<html>
<head><title>526 Origin SSL Certificate Error</title></head>
<body bgcolor="white">
<center><h1>526 Origin SSL Certificate Error</h1></center>
<hr><center>cloudflare-nginx</center>
</body>
</html>

Cloudflare Origins

When you put a site behind Cloudflare, there are a few options for configuring TLS. You can pick how clients connect to the Cloudflare’s edge and how Cloudflare’s edge connects to your origin. The easiest way to do the latter is turn TLS off, meaning the traffic is unencrypted in transit. I’m only running a static site, but I’d prefer not making it that easy to modify my content before it reaches the client. Fortunately, Cloudflare can also make a TLS connection to the origin, which is called strict mode. But rather than having to go get a cert from a globally-trusted CA, Cloudflare will generate one from their own CA that is specific to this use case. I find it quite convenient, since it prevents me from having to integrate with another service provider.

And naturally, this setting can be controlled via terraform.

resource "cloudflare_zone_settings_override" "apex" {
  zone_id = var.cloudflare_zone_id

  settings {
    always_use_https = "on"
    ssl              = "strict"
  }
}

That 526 status code means that Cloudflare was unable to validate the cert that my origin was serving - the one that I provisioned using their API. So at least I know where to start looking.

The fix

I won’t deny there is some luck involved here, but I’m simultaneously a bit proud that the tooling I have set up for managing the infrastructure makes this so easy. In my main.tf file, I have one resource defined to have Cloudflare create the cert and another for it’s deployment to the filesystem:

resource "cloudflare_origin_ca_certificate" "tls" {
  csr                = tls_cert_request.tls.cert_request_pem
  hostnames          = [local.domain, "*.${local.domain}"]
  request_type       = "origin-rsa"
  requested_validity = 365
}

resource "linux_file" "origin_cert" {
  src  = format("data:;base64,%s", base64encode(cloudflare_origin_ca_certificate.tls.certificate))
  dest = "/etc/pki/tls/certs/cf-origin.pem"

  provider = linux.main
}

Well, the 365 there mostly confirms my suspicion, and running openssl x509 makes me absolutely certain. The cert is generated by Cloudflare via an API call, but that request is made by terraform via the provider. I’m only stuck thinking for a minute until I remember that command I haven’t had a use for yet:

$ terraform taint -h
Usage: terraform taint [options] <address>

  ...

  This will not modify your infrastructure directly, but subsequent
  Terraform plans will include actions to destroy the remote object
  and create a new object to replace it.

Yep, that’ll do. But I know to expect this again, so I might as well write it down.

$ git show HEAD
commit b711e5a39d35b4186eac9b2579bd1ec48ec091c2
Author: Jordan Griege <>
Date:   Mon Jan 11 17:03:51 2021 -0600

    steps for checking and rotating origin tls cert

diff --git a/Makefile b/Makefile
index b4dc8f1..560f002 100644
--- a/Makefile
+++ b/Makefile
@@ -27,3 +27,12 @@ ssh: id_rsa
 .PHONY: repo-init
 repo-init:
 	ssh -i id_rsa root@$(shell $(MAKE) main_ip) "cd /srv/git/$(REPO)/ && git init --bare && chown -R git:git * && echo '$(DESC)' > description"
+
+.PHONY: tls-expiry
+tls-expiry:
+	terraform state show cloudflare_origin_ca_certificate.tls | grep expires_on | awk -F\" '{print $$2}'
+
+.PHONY: tls-rotate
+tls-rotate:
+	terraform taint cloudflare_origin_ca_certificate.tls
+	$(MAKE) apply

Sure enough, the plan shows that the cert has to be recreated, and the only other step I need is to restart nginx. Ideally that dependency would also be expressed in the terraform graph, but I’ll leave it for another time.

And that’s how you rotate a TLS certificate using terraform ^_^ (/r/BrandNewSentence ?)