Terraform is the tool for managing the lifecycle of your infrastructure. It allows you to build, change, and version infrastructure efficiently and safely. Terraform can help you create databases, roles, security tokens, and access lists. Read on to see how you can get started.
DataStax has created an open-source Terraform provider for DataStax Astra DB. A provider that makes it easy to script the creation of an Astra DB database, roles, security tokens, and access lists for your database.
A provider is the translation layer provided by a vendor that allows the provider’s platform to execute Terraform engine commands. For example, Amazon AWS has a Terraform provider that will translate the Terraform scripts into real actions (i.e. creating compute instances or any other AWS asset). Similarly, there are providers for Google, Azure, Oracle, and over a thousand other service providers. You can find all current providers on the registry.terraform.io page.
It’s a bit misleading to say that the provider translates the Terraform commands. It’s not like the good folks at Hashicorp (the creators of Terraform) defined a single way of expressing every idea in cloud computing! That’s impossible (there is no spoon). But, they provide a framework to give you provider-specific commands from Terraform to the provider of your choice. You’ll see precisely how this works when we create the Astra database.
Terraform is very easy to use once you get your head wrapped around it. It’s not a procedural language or an object-oriented one. Instead, you create files that describe the desired end-state of your database, and Terraform will then create or destroy it, depending on your specific commands.
Getting started is as simple as installing Terraform and then telling Terraform to implement the Astra “provider.”
To install Terraform:
- Click on this link and press the “Download CLI” button.
- Follow the installation instructions from Terraform.
- Add the terraform directory to your system’s PATH environment variable so you can start it from anywhere.
Once Terraform is fully installed, create a folder on your local file system for your project, then change it into that new folder.
In your new project folder, create a file with the name main.tf. Technically you could name it anything you want, but main.tf is the traditional name for the file that contains the terraform{} declaration.
At this point, you technically have a working Terraform project. However, it doesn’t do much. All it does is install the Astra provider. To run it, use the terraform init command, which will produce the following output:
So far, our main.tf file has provided two instructions:
- Ensure we are running Terraform version 1.0.0 or greater
- Load the Astra provider for Terraform version 1.0.0 or higher
If you examine your project directory you’ll see a new hidden directory, .terraform.
Pro Tip: When specifying versions, it’s best to use the
>=notation to ensure that you are getting the latest version (with the latest security patches, features, etc.).
Let’s modify our project to create an Astra DB database. Before you can do that, you will need the following:
- An Astra DB Database account. You can register for free at astra.datastax.com/register.
- You’ll need to create an Astra DB API for the Organization Administrator or Database Administrator roles. The security token will allow Terraform to connect to the Astra API layer to execute its commands. If you’re new to Astra and unsure how to create your security token, check out this page.
Pro Tip (Security): When you create your organization security token, use an export statement to put it in a
setenv.shfile along with the other Astra-required information and then set a.gitignorerule.
I’m very sensitive to accidentally sharing my security tokens, and I hope you feel the same way. When I created my organization security token, I put it in a setenv.sh file using an export statement, along with the other Astra information that will be needed:
Then, from the command line, I set that environment variable using a source setenv.sh command (Mac or Linux). Then set a .gitignore rule for that file. That way, it’s highly unlikely that my security token will even end up on GitHub!
Where can I find my Organization ID?
Logging into your main dashboard is the easiest way to find your organization ID. In the URL in your browser, your organization ID is the UUID listed after the astra.datastax.com/ in your browser’s address bar.
Define variables
You can pass information into the Terraform process using variables. In our case, we need to pass in ASTRA_API_TOKEN and ASTRA_ORGANIZATION_ID values from our environment variables. To do this, create a file called variables.tf and give it the following contents:
That tells the Terraform process that a variable exists, but it doesn’t define the contents of those variables. It’ll define the contents of these variables via the command line. Local variables are different, you’ll read about them in the next section.
Define resources
Terraform describes a resource as: “Each resource block describes one or more infrastructure objects, such as virtual networks, compute instances, or higher-level components such as DNS records.”
In simpler terms, a resource is something that you create. In Astra, that will be things like databases, roles, tokens, and access lists.
Get started by creating a resources.tf file in your project directory. Set the content of that file to the following:
Save the file. As you can probably guess, we’re instructing Astra to create a database named hello_astra with a keyspace of helloastra (the value of the local.keyspace variable) on the Google Cloud Platform (gcp) in the us_west1 region.
What isn’t so easy to guess is that this creates a variable named astra_database.hello_astra_db in the Terraform process. We have provided the required fields to create the database, which creates additional information fields like .data_endpoint_url, the base URL for REST APIs, and the .grafana_url, which is the URL to view the Grafana health charts for the database.
Now run the command:
When the command finishes successfully, you can then execute the terraform apply helloastra command to see the database being created:
Now open up the Astra DB web console and you’ll see your database has been created:
Pretty cool, but minimally useful in practice. There’s little reason to use Terraform only to create databases with key spaces. However, the database is just the top-level “container” for most projects. Real-world projects also contain roles, each of which needs a security token. You can also script the IP safe-listing using Terraform, which is where Terraform starts to shine for Astra DB.
Define a role
An Astra DB role is simply another resource type. So open up your resources.tf file and append the following text:
You can see there is a lot of thought required for creating a role. A solid understanding of the policies and the resources is required. That’s too much information to cover in this article, but you can find much more on the main Astra DevOps page. And you can find more information if you visit the Manage Roles with DevOps API page.
Note: I commented that many of the policies for the role we’re creating weren’t needed. However, I left them in the file as a quick and easy reference if the role you define needs different policies. Comment and uncomment at your whim!
There are two main sections in the role, the resources and the policy arrays. They are closely related. The resources section is where you select the resources for which you want to set policies. So in our resources section:
You can see, we are telling Terraform we want to create a role with privileges that are specific to:
- Our organization
- Database we are creating
- Keyspace in that database
- Tables in that keyspace
If you look at the Manage Roles with DevOps API page and scroll down, you’ll see a table that maps policies to specific resource types. So policies only apply to specific resources, which is a mistake you can make. Suppose you add policies to access the tables but don’t include the last resource for all tables (:table:*). You can end up table policies with no resource to which they could attach.
Define a security token
Now that we have a role, we need to define a security token that users can use to authenticate the right to use that role (and access the database and its data). So, still editing the resources.tf file, append the following lines of code:
You can see there can be any number of roles in the roles array. As a result, you should think about tokens aggregating roles, not working with a single role (as we are doing in this simple example).
Define access lists
An access list defines the IP addresses that are allowed to connect to the database. If you don’t define an access list, then ALL IP addresses are allowed to connect, assuming they have the correct security token. If you know of specific IP addresses or classless inter-domain routing (CIDR) blocks that you want to allow to connect (still requiring the security token though), then you can define your rules using the template code below.
The addresses{} object can contain multiple request{} objects, so your IP safe-listing can get to be very fine-grained. The address{} within each request{} is formatted as a CIDR block.
Pro Tip (Security): The
terraform.tfstatefile, created by the terraform plan command, contains sensitive information and should never be stored in a public repository!
Our script generates a lot of information that we’ll want to use elsewhere, namely the Astra DB security token for accessing our database using our role. To have Terraform show us the values we are interested in, open up the main.tf file and append the following lines at the bottom (outside of the terraform{} ) declaration:
We’ve done a lot of editing. Now let’s run our project to see the fruits of our labor. Execute the commands:
Terraform also lets you delete/destroy what you create. After successfully performing an apply action in Terraform, you can then execute the delete action and destroy everything created by the apply action.
Destroying everything you created is especially handy when you are first developing your scripts and learning Terraform. The command to destroy/delete your Astra database with all of its roles and security tokens takes the following form:
The file .terraform.lock.hcl “locks” in the provider versions last used. If you want to update all of your providers to the current version, delete the .terraform.lock.hcl file and run the terraform init command again.
The Astra provider is great at what it does: creating databases, roles, security tokens, and access lists. In its current form, that’s all it does. It does not create tables or populate data. You still need to do that using traditional methods.
Hopefully, though, the veil of mystery that surrounds Terraform, and the Astra provider more specifically, has now been removed for you. The Astra provider is under active development, so be sure to check for updates often! You can find the main Astra DB provider page in the Terraform Registry, which provides complete instructions to guide you through all of its capabilities.
If you want to go deeper or contribute to the open-source project, you can do that on the Astra Provider GitHub page. You can also join the DataStax Developer’s Discord channel to learn more there.
Follow the DataStax Tech Blog for more developer stories. Check out our YouTube channel for tutorials and here for DataStax Developers on Twitter for the latest news about our developer community.
