Managing AWS IoT Rules through CLI scripts

If you are doing a lot of IoT messaging in AWS, chances are you are using AWS’s IOT Core. At this point, the support of IoT Rules is missing on topics of Cloudwatch logs and Timestream rules. Timestream is a really new offering so it makes sense the API has not caught up with integrating with this product.

However, it’s awful that something as fundamental to AWS as Cloudwatch has such limited support. You can define an action to write metrics to Cloudwatch using the API though. It’s writing logs that are missing from CDK’s API. Which for me is the goto action as error action if your regular action fails.

Use aws-cli-js to use Typescript to use AWS cli commands to manage your rules

Scripting IOT Rules with Typescript and aws-cli-js library

There are numerous options to choose from when you want to script these IoT rules. A CDK custom-resource would be one option. Bash with AWS CLI would be another. This time I needed a solution using the comfort of a higher-level language, without Cloudformation execution so I chose the aws-cli-js Javascript library which is sort of a CLI wrapper. It offers Javascript (and therefore usable with Typescript) to execute AWS CLI commands. So it’s much like using a bash script, only with a language that has — in my opinion — a better development experience.

However, it turned out to be a bit of a struggle to get the proper json based messages into the API of aws-cli-js. Initialization of the library is easy. You can use environment variables in which point to a valid session. For example, using aws-vault.

const options = new Options(
process.env.AWS_ACCESS_KEY_ID,
process.env.AWS_SECRET_ACCESS_KEY,
process.env.AWS_SESSION_TOKEN
);
const aws = new Aws(options);

Now the AWS object is ready for your CLI commands. Just like in normal CLI calls that involve pushing some complicated configuration, you often have two options. You either chain a lot of variables together. Or you use a JSON-based document describing your configuration. The latter is a more likely pick, given you are already in a language that supports JSON natively.

The tricky thing in this situation was to actually create a JSON document that got passed to the CLI in the proper format. This was tedious to debug, and after a lot of attempts, I found out that obvious attempts like using JSON.stringify() were not working because the library is passing the resulting string with unescaped quotes which leads to early termination of your command. Eventually, I ended up manipulating strings and putting escape marks in my object.

const payload = `{ \
\\\”sql\\\”: \\\”SELECT value FROM ‘\\\$aws/sitewise/#’\\\”, \
\\\”awsIotSqlVersion\\\”: \\\”2016–03–23\\\”, \
\\\”description\\\”:\\\”${ruleName}\\\”,\
\\\”actions\\\”:[\
{\
\\\”timestream\\\”:{\
\\\”roleArn\\\”:\\\”${process.env.TIMESTREAM_INSERT_ROLE}\\\”,\
\\\”databaseName\\\”:\\\”${process.env.TIMESTREAM_DB_NAME}\\\”,\
\\\”tableName\\\”:\\\”${process.env.TIMESTREAM_TABLE_NAME}\\\”,\
\\\”dimensions\\\”:[ \
{\\\”name\\\”:\\\”property\\\”,\\\”value\\\”:\\\”dimensional-value\\\”}, \
]\
}\
}\
], \
\\\”errorAction\\\”:{\
\\\”cloudwatchLogs\\\”:{ \
\\\”logGroupName\\\”:\\\”${process.env.TIMESTREAM_ERROR_LOG_GROUP}\\\”, \
\\\”roleArn\\\”:\\\”${process.env.TIMESTREAM_ERROR_LOG_ROLE}\\\” \
}\
}\
}`;

This configuration object describes two different IoT Rule actions that are not supported in CDK. The first one is a Timestream action, the second action is triggered in case of error when executing the first action. It writes a log to Cloudwatch.

To actually perform this command u can use the following script.

await aws.command(“iot create-topic-rule — rule-name “ + ruleName + “ — topic-rule-payload \”” + payload + “\””).then(function (data: any) {
    console.log(`Successful added rule for asset ${ruleName}`);
    dataReturn = data;
}).catch(function (error: any) {
     console.error(error);
     throw Error(error);
});

So that’s it! Using this step you can manage your IoT Rules using the power of Typescript. I used it to manage over a hundred rules. The only downside it that the execution of these CLI rules is not really fast. A hundred rules can take minutes, depending on the amount of processing you need to do. This can be improved by a bit of parallelism. However, be aware that the quota of IoT Rules does not allow for massive parallel reconfiguration. I believe it currently can handle 10 changes per second by default.

If you get lost, I have a Github repo with an example project. https://github.com/kevinvaningen/aws-iot-rule-script

Have fun automating!