Spotlight
AWS Lambda Functions: Return Response and Continue Executing
A how-to guide using the Node.js Lambda runtime.
Joel Haubold | Dec 07 2023
5 min read
5 min read
Trek10's work below got spacklepunched by AWS when they introduced one-minute resolution for CloudWatch Events. See here. This wasn't the first and won't be the last time, which is why we love working in such a fast-paced ecosystem.
The original post is below, but we highly recommend you ignore it and use normal schedule expressions for Cloudwatch Events!
Thu, 10 Mar 2016
Currently, CloudWatch Event rules can be scheduled at minimum intervals of 5 minutes. At Trek10, we saw the need for a smaller interval, namely 1 minute. This need arose specifically in the context of handling Auto Scaling for ECS. Since scaling ECS can require two steps to scale out (increase services desired count, then increase cluster instances), the resulting 10 minute delay (5 minutes for each action) for scaling was just too long.
We’ve found this to be extremely valuable for customers opting for smaller and faster release cycles. Faster deployments let you have less scaling jitter, and controlled scaling reduces costs.
There are five CloudWatch Event Rules each of which uses a cron expression for scheduling:
0/5 * ? * MON-SUN * // LambdaClock_01/5 * ? * MON-SUN * // LambdaClock_12/5 * ? * MON-SUN * // LambdaClock_2......Each one of these rules has a target of the LambdaClock function. This leads us to what we feel is an obvious question, ”If an event triggers each minute, why not have the event publish to an SNS topic and be done?”
However, we saw inconsistent delays between the CloudWatch Event trigger time and publishing to an SNS topic or invoking a Lambda function. The problem might have ended there if the delays were consistent, but after reviewing over 4,700 CloudWatch Events, we recorded an average delay of 25 seconds. That said, some rules consistently invoked Lambda functions within 10 seconds, while others were consistently recorded at over 40 seconds. Thankfully, all rules tended to stay below 60 seconds.
A Lambda function (LambdaClock) is the target of each CloudWatch Event rule. LambdaClock calculates the time it must wait until the next minute by comparing the current time and the time the event was triggered. LambdaClock then uses setTimeout to wait until the 0th second of the next minute, at which point it publishes to an SNS topic, which is also named LambdaClock. A second Lambda function is used to analyze the time it took for this SNS topic to invoke a subscribed Lambda function. Some preliminary results are shown below:
The ultimate goal was to achieve a scheduling period of one, and thus far the LambdaClock system seems to be holding up nicely.
Visit our GitHub repo to download the LambdaClock.json file, use it to create a stack through CloudFormation, and you’ll have your very own LambdaClock SNS topic. All we ask is that you treat it well, and don’t feed it after midnight … well, it’ll probably work fine after midnight.
Check out our other blog posts and learn more about our work at trek10.com. And, don’t be afraid to tweeter at us trek10inc. Let us know what you think and we’ll be back with more soon.
A how-to guide using the Node.js Lambda runtime.
