Unlock your Infrastructure-as-Code with the power of Configuration Languages

Why do you need a configuration language?

You will benefit greatly from a configuration language if you regularly write JSON or YAML files and have found a need to adopt:

• Text templating tools like Mustache, Go Template, or Jinja;
• General purpose programming languages to manage infrastructure with tools like CDK or Pulumi;
• Or data-aware template tooling such as Jsonnet or Carvel ytt.

If you are a Platform or Cloud engineer, then this definitely applies to you. The reason is simple: your daily interactions with code often involve Infrastructure-as-Code (IaC). Modern IaC tooling takes a declarative approach to defining infrastructure. It focuses on the outcomes we want irrespective of the steps taken to get there. IaC is a configuration of state, or put simply data. Data which can be produced by a configuration language.

Now you know if you need a configuration language, the question is what exactly is a configuration language?

What is a configuration language?

A configuration language is a language which is used to define data structures. Unlike programming languages, configuration languages do not need to be Turing-complete to be useful. In fact, those who aren’t glean power from their simplicity and shallow learning curve. Another distinction is the relative purity of a configuration language. On the one hand, a general-purpose programming language is most often used to write programs which act upon the world. On the other hand, the goal of a configuration language is to transform given input to output data. Side effects such as file IO or network calls are either not directly supported or play only a supporting role in defining data.

There are many examples of configuration languages out there, such as Vimscript, Jsonnet, Carvel, and Nix. Some of these you may already be using, while others may be the more esoteric option if used outside of their intended domain. However, all of the options that existed before the year 2020 are limited. They are either domain-specific, bound to a data format, or tightly scoped in their solution.

The New Wave

There is now a new wave of configuration languages which boast a more holistic approach to configuration. This distinguishes them from the swathes of domain-specific cousins. So why exactly do you need a configuration language? Allow me to highlight two projects which offer complete configuration solutions worthy of defining even the most complex infrastructure.

Born out of use cases dealing with large-scale configuration and policy management, CUE and KCL offer complete solutions. Both are configuration languages that offer ways to enforce policy and package and distribute code. These latter requirements define the new wave of configuration tooling. They set it apart from the previous generation of tooling, such as Jsonnet and Carvel.

This is not a comparison of CUE and KCL. It is rather an introduction to the tooling void they are designed to fill. After reading this section, I want you to feel excited about the possibilities of modern configuration languages!

The examples shown above are only a very small part of the power of these languages. Although simple, they are exciting to YAML wranglers like myself. As you can gather from the blurbs of CUE and KCL, they both talk about schema, validation, policy and configuration. These are high level concepts, and they need to be to fit on the home page of their websites. So what can you actually do with this?

Characteristics of configuration languages

Use your imagination and reinvent anything that revolves around data. There are so many valuable uses for configuration languages, and they’re all made possible by their defining characteristics. So let’s talk about them.

1. A Simplified Language

While inspired by general-purpose languages, both CUE and KCL limit the number of language features on offer to keep the toolbox fit for purpose. This minimal approach reduces the amount of foot guns present in many other general purpose languages. It also simplifies the writing and reading of language syntax. A key in how both these languages operate is an effort in the reduction of indirection. In large, complex configurations it could be difficult to pinpoint the line of code responsible for a configuration outcome if complex inheritance structures or overlays and mutability were allowed. As such, both CUE and KCL prioritise immutability, albeit in different ways.

2. First-Class Policies

CUE and KCL are different from the configuration authoring tools of the past. They include ways to write and enforce policy directly in the language spec. CUE does this through a unique approach. This approach narrows data types using a value lattice. Meanwhile, KCL achieves the result through a schema check syntax and rules. Making policy a first class citizen with powerful expressiveness is an important part of forming the identity of these projects as configuration languages and sets them apart from the tools that came before.

3. Modules

CUE and KCL have both embraced the OCI standard. This can be used to package and distribute modules which can be imported into other modules to form your configuration. The implementation of these is particularly useful for distributing schemas or policies to users. Authors of APIs can provide schema modules such as those for Kubernetes. Security, cloud and platform engineers in the organisation can add policy and abstraction before redistributing it internally for developers to consume by supplying the final layer of data. This ability to import, enhance, and distribute configuration using OCI images is an incredible way to speed up developers. All while ensuring correctness and compliance.

4. Data Agnostic

There are many data models out there from JSON and YAML to Nix and Lisp. While many tools focus on just one, CUE and KCL have their own intermediate data formats and provide integrations which map data between it and the formats we use. While formats such as Nix and Lisp are not yet supported, many of the common ones are with the list growing over time.

5. SDKs

When a piece of the puzzle is missing for your use case, such as a certain import or export option, all is not lost. CUE and KCL both offer extensive SDKs to allow you to embed the usage of these configuration languages into your own programs. CUE provides a very comprehensive Go API to enable anything from using CUE as your application’s primary configuration format to implementing a full test suite leveraging the power of CUE’s unification engine. KCL also offers SDKs across an extensive list of languages ranging from Go to Lua.

6. Automation

Like tools that have come before, both CUE and KCL output data by invocation, either to stdout or to a provided output file. Though unlike many others, both CUE and KCL implement APIs to enable automation right in the language (this is where Execute comes into play for the E in CUE). The benefit of this is that you can implement your integration actions close to the data that drives them instead of finding another tool to parse and execute like you’d commonly find a Python script doing. CUE provides a tooling layer for enhancing the CLI with custom workflow commands which can print various data to stdout, write to file, or even make HTTP requests. KCL also offers system packages for writing to stdout and file IO with the opportunity to extend capabilities through custom plugins. Though it is important to note that as of the time of publishing, custom plugins require the usage of the KCL SDK rather than the CLI. Something I hope to see made possible via modules in the future.

---

Should you pick the CUE Playground or the KCL Playground?

I strongly urge you to take a closer look at either CUE or KCL. Pick any and start having a play with them. I find it difficult to pick my favourite and keep going back and forth, they both have so much to offer. I would characterise CUE as the dogmatic choice and KCL as pragmatic. Given my love for Haskell, the poster child of dogmatic programming, CUE satisfies me in just the right way. However, KCL’s roadmap has allowed it to tick a lot of the boxes early in development enabling it to already be a productive configuration language in many scenarios. At the time of writing, both are in initial development with major version zero and both projects have plans to tick boxes in all the same categories. I would absolutely recommend keeping your finger on this pulse.

Delving deeper into how configuration languages facilitate Infrastructure as Code will empower you to simplify your cloud management practices significantly. Learn more in our article “Infrastructure as Code: Simplify Your Cloud Management Today!“