AWS Timestream CDK

This article provides some ready made AWS Timestream configuration as code examples using Typescript CDK code snippets. Over the past months I got the chance to use one of AWS latest additions to their already impressive database offering called Timestream. Being a big fan of infrastructure automation I needed to configure this database using code. I found out first-hand that there are not many examples out there on scripting Timestream automation. Even though it’s not the hardest AWS services to automate deployment for, it still took some experimentation to get to get it to acceptable level. At the end of the article there is a link to a related Github repo.

AWS Timestream basics

Timestream is as defined by AWS as a:

“fast, scalable, serverless time series database”.

This one-line basically sums up most you need to know about its offering. Its a database designed for the particular use-case where you are storing data with at its heart a timestamp and properties describing what you want to record on that particular moment, commonly called dimensions. Related offerings that might sound more familiar are InfluxDB, Grafana, and Apache Druid but there are many more.

Timestream CDK

Even though the CDK for Timestream is not that complex. Let’s start with a simple working example, discuss it, and add to it to make it complete.

import {Construct} from "@aws-cdk/core";
import {CfnDatabase, CfnTable} from "@aws-cdk/aws-timestream";

export interface SimpleTimestreamConstructProps {
    databaseName: string;
}

export class SimpleTimestreamConstruct extends Construct {
    public readonly database: CfnDatabase;
    constructor(scope: Construct, id: string, props: SimpleTimestreamConstructProps) {
        super(scope, id);
        this.database = new CfnDatabase(this, 'SimpleTimeStreamDatabase', {
            databaseName: props.databaseName,
        });
    }
}

At time of writing, there are not ready-made This construct shows how to do a basic bootstrapping of a database in Timestream. Once added to the stack it can be deployed using the `cdk deploy` command.

Once deployed you can view your freshly created database in the console. As you can see there is nothing much happening after you created a Timestream database. Unlike with relational databases in (like Postgres), there is not much you can manage on the database level. Without a table, there is nothing you can do with this database. So let’s expand the example to add a table.

import { Construct} from "@aws-cdk/core";
import { CfnDatabase, CfnTable } from "@aws-cdk/aws-timestream";

export interface SimpleTimestreamConstructProps {
    databaseName: string;
    tableName:string;
}

export class SimpleTimestreamConstruct extends Construct {
  public readonly database: CfnDatabase;
  public readonly table: CfnTable;

  constructor(scope: Construct, id: string, props: SimpleTimestreamConstructProps) {
    super(scope, id);
    this.database = new CfnDatabase(this, 'SimpleTimeStreamDatabase', {
      databaseName: props.databaseName,
    });

    this.table = new CfnTable(this, 'SimpleTimeStreamTable', {
      tableName: props.tableName,
      databaseName: props.databaseName,
    });
    this.table.node.addDependency(this.database);
  }
}

This example can be deployed so our database actually contains a usable table. Please note that this example adds a dependency from table on the database resource. Without it, your script will introduce a race condition which often fails. The database needs to be present before the table is added.

Timestream CDK with retention properties

Timestream consists of an integrated data life-cycle management solution. These life-cycle tiers can be configured with poorly documented retention properties in the following maner.

this.table = new CfnTable(this, 'TimeStreamTable', {
  tableName: props.tableName,
  databaseName: props.databaseName,
  retentionProperties: {
    memoryStoreRetentionPeriodInHours: (24*7).toString(10),
    magneticStoreRetentionPeriodInDays: (365*2).toString(10)
  }
});

This example represents an in-memory tier of a week and afterwards data will be moved to read-optimised tier for two years.

Timestream CDK removal policy

After deploying and changing the setup a couple of times, you might notice that the database and table resource have very very very limited support for infrastructure migration (like renaming). As a consequence your data will be forever lost. So in order to mature this setup retention options are necessary.

this.database = new CfnDatabase(this, 'TimeStreamDatabase', {
    databaseName: props.databaseName,
});
this.database.applyRemovalPolicy(RemovalPolicy.RETAIN);

this.table = new CfnTable(this, 'TimeStreamTable', {
    tableName: props.tableName,
    databaseName: props.databaseName,
    retentionProperties: {
        memoryStoreRetentionPeriodInHours: (24 * 60).toString(10),
        magneticStoreRetentionPeriodInDays: (365 * 10).toString(10)
    }
});
this.table.node.addDependency(this.database);
this.table.applyRemovalPolicy(RemovalPolicy.RETAIN);

This removal policies are commonly used in Cloudformation scripts. The retail policy flags to disassociate the resource when a resource is altered in a way that would otherwise be destructive. This way your data remains safely stored into your table.

A sample repo can be fout on Github.

Conclusion

Based on the examples showed Timestream is perfectly manageable by using CDK. The configuration options are still limited, however the product is only just released. We can expect first class CDK support and more configurable options in the time to come.