Terraform custom logs in Oracle Cloud

Thanks to boliviainteligente for making this photo available freely on unsplash 🎁

Have you ever reverse-engineered a state file before? A better question might be: why would you ever need to do so? Boy, do I have the use case for you.

Yesterday, I was terraforming logging integrations with one of our company's flagship products. Specifically, I was getting the product and its ancillary networking components bundled up as an Oracle Stack. It's pretty cool tech that I haven't seen on Azure or AWS - you can zip up terraform code and upload it, then plan and apply and all the other things you can do with terraform.

Anywho, we wanted to ship the logs from our product, running on a compute instance, to the portal's log viewer.

After a few minutes on the OCI provider docs, I could not find any resource with names like "custom|log" (it was indeed there - more on why I couldn't find it later). I knew the functionality existed, as I had POC'd this in the portal before trying to write the terraform for it.

Azure has terrafy for this exact purpose; turns out Oracle has something even better.

You can "stack-ify" (my term) existing resources in any OCI compartment, natively within the portal! Removes the unwieldiness of a tool like terrafy. You can do it like this:

.

So I created the custom log setup in a separate compartment. It required three resources - an a log group, a log, and a custom log. The latter is similar to what you'd expect - you pass a file path on the compute instance that matches the logs you want to see in the portal.

After the stack was provisioned, I looked at the resources created, to finally discover the name of the resource I was trying to terraform:

.

Oh, what kinda name is oci_logging_unified_agent_configuration?

Well, whatever. Now we can finally terraform it, since we know what it's named. I'm going to submit a github issue to rename that resource.

Extra points if you reverse engineer the state file - you can view the state (which Oracle lets you do), and do some json-to-hcl regexing on the required properties that are actually set. There are a few thousand properties for this resource, which is understandable, but is quite tedious to reproduce by hand from the docs. The resource's state only contains the props you need to set, if you don't mind wading through the state file:

...
  "sources": [
    {
      "channels": [],
      "name": "pathfinder logs",
      "parser": [
        {
          "delimiter": "",
          "expression": "",
          "field_time_key": "",
          "format": [],
          "format_firstline": "",
          "grok_failure_key": "",
          "grok_name_key": "",
          "is_estimate_current_event": true,
          "is_keep_time_key": false,
          "is_merge_cri_fields": false,
          "is_null_empty_string": false,
          "is_support_colonless_ident": false,
          "is_with_priority": false,
          "keys": [],
          "message_format": "",
          "message_key": "",
          "multi_line_start_regexp": "",
          "nested_parser": [],
          "null_value_pattern": "",
          "parser_type": "NONE",
          "patterns": [],
          "rfc5424time_format": "",
          "syslog_parser_type": "",
          "time_format": "",
          "time_type": "",
          "timeout_in_milliseconds": 0,
          "types": {}
        }
      ],
      "paths": [
        "/var/log/pathfinder/*"
      ],
      "source_type": "LOG_TAIL"
    }
...

  "mode": "managed",
  "name": "export_test_2",
  "provider": "provider[\"registry.terraform.io/hashicorp/oci\"]",
  "type": "oci_logging_unified_agent_configuration"
}

Oracle has a cool button that allows you to apparently view and edit the terraform code that produced the stack:

.

I say 'apparently' because the button does not work, but produces a nice devtools barf. Maybe it'll work sometime soon?