AWS Timestream — lessons learned

AWS Timestream’s offering

Photo by Aron Visuals on Unsplash

Why Timestream?

  • Increasingly, projects seem to have an event-driven and streaming nature. Storing these events based on time and having a feature-rich api to query by time is very useful in our toolbox.
  • Command Query Responsibility Segregation (CQRS) is getting more and more widely adopted. This architecture approach makes it easier to pick specific database types for specific tasks. With documents, relations, and graphs covered in the AWS Services stack its only a healthy follow-up to add a timeseries database management system.
  • Serverless databases allow us to go to market quickly and prove to allow for more transparant billing in terms of the relation between usage and costs. Storage is for many non-serverless projects still a source of high fixed costs and upfront investments (a.o. licenses, skills, infrastructure).

Positive experiences with AWS Timestream

  • Databases and tables are easy to set up. There is little configuration or documentation required.
  • The SQL API is feature-rich and will feel comfortable for everybody with extensive query experience. Within days, you will write elaborate queries with all kinds of concatenations to get the most response for your money.
  • The data structure of time and dimensions is well-known to timeseries. If you have worked with products like InfluxDB but also Cloudwatch logs you will quickly adjust to this kind of datastructure.
  • I didn’t make an extensive study comparing Timestream with other databases, but in general the time spent on performing multiple complicated queries in parallel seems okay. On the other hand it’s certainly not the fastest database money can buy. For simple queries it feels a bit slower than what I would expect. On queries spanning thousands of records and concatenating multiple queries to one results it feels more up to standard.
  • Its a good serverless companion. When using Timestream from your Lambda’s its easy to implement the query api, and also result fetching and processing is a breeze. Results are type-save and the rich sql query api ensures that you have little processing to do in your Lambda’s.
  • IoT Rules has support for directly ingesting IoT data into Timestream.

AWS Timestream cons

  1. Let’s start with the most upsetting ‘feature’. Timestream does not allow you to insert data with a timestamp older than the database retention window! Currently, the product page shouts ‘historical’ four times, however, the options to ingest data from the past is very limited. To counter this, one must never destroy the database, and table, AND setup a liberal retention window. However, personally, I could not believe that a company that promotes resilience patterns (like rebuilding your infrastructure, which is not possible with Timestream), and scripted infrastructure (which you can barely use if you are not allowed to change your database) creates a new service with these characteristics. Competitive offerings like that of InfluxDB do not have this disadvantage.
    Edit: in November 2022 AWS released an update statement saying its possible to insert historical data.
  2. There is no record deletion API! This means that if you store for example a person’s weight over time in Timestream, and you found out that one of the values is corrupt, that there is NO way to delete. There is in fact an upsert api that allows you to enter a version. I am happy with this feature, however if there is really no sensible value for you to update a record to, you are stuck with it!
  3. There is no way to change the schema datatype. Timestream extrapolates your datatype from the first insert, after which it checks for type safety. However, without the option to remove data, combined with the option to never delete your table (because of bullet 1) it is impossible to change your value type.
  4. There is no backup / restore functionality. Combined with the three points above AWS also ruled out to create an ‘easy’ ETL solution for this database.

Conclusion

Timestream automation

--

--

--

Software delivery and cloud enthousiast

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

10 Lines on Computer in English

My journey into Software Engineering

This Week in Atomist — 17th January, 2017

Modernizing the Data Analytics Platform at Financial Engines

Behind the Scene of Java

Automating the Process of Web Scraping a Table and Pushing it to MongoDB Using Python

Top frameworks to learn in 2019!

Explore Realtime Database In Flutter

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Kevin van Ingen

Kevin van Ingen

Software delivery and cloud enthousiast

More from Medium

Custom Lenses for the AWS Well-Architected Tool

Cedar: Your intuitive AWS Lambda metrics and logging visualizer

API & Serverless Architecture Conference Berlin 2021: An opinionated review

Redlink Team watching the Serverless conference

AWS Re:invent Day 4 Highlights