“Pre:Invent” feels like a main course this year as AWS developers salivate over the enticing new features rolled out in the runup to next week’s big show in Las Vegas.
A few patterns have emerged in the dozens of announcements, but none more important than this: AWS finally seems to be getting serious about making their internal monitoring and observability tools competitive with existing third-party options. Moreover, they’ve rolled back the curtains to give developers more control over information that was previously trapped in the black box of managed services. To the roundup we go…
Let’s start with the biggest announcement: CloudWatch ServiceLens. If you’ve tried to monitor or trace serverless applications before, you know that the existing AWS-native solutions have left quite a bit to be desired. Most egregiously, X-Ray and CloudWatch don’t always play nicely together, so I often found myself hopping back and forth between the two services to check a trace or drill down into a log stream.
ServiceLens attempts to bridge that gap by providing a “single pane of glass” unifying X-Ray traces with CloudWatch features such as metrics and alarms. The resulting experience definitely feels X-Ray-esque, built around the concept of a “service map” you can drill down into to get more information about a poorly-performing request. My favorite thing about the new console experience is how easy it is to slice and dice requests by dimensions, such as client IP or URL.
Say what you will about CloudWatch and X-Ray, but I’ve never heard a developer express that they were particularly delightful to use. ServiceLens doesn’t really provide that much new functionality, but it has the pretty dashboards and intuitive design that have been missing from so many of AWS’s user experiences.
Ultimately, though, the usefulness and adoption of ServiceLens will depend on the quality of data that can be collected going forward. We’ll be looking for more AWS services to support X-Ray integration for the true serverless observability experience.
Think of CloudWatch Synthetics as a “managed canary service” that’s continually pinging critical aspects of your application to make sure they’re holding up as needed.
Unless the provided blueprint options meet your needs, you’ll still be writing custom Lambda code to perform the heartbeat checks themselves, so the main advantage here over an ad-hoc solution is the managed scheduling and alerting via CloudWatch rules and alarms.
The big challenge is around price: at $0.0012 per canary run, you’ll be paying something like $50 per month to run per-minute checks on a single endpoint. Is that worth the added simplicity over running your canary checks in regular Lambda functions essentially for free? Right now, that seems like a hard sell.
Note: Synthetics is still in open preview in just a few regions, which means it could change before it’s generally released.
CloudWatch Contributor Insights is another slice-and-dice feature that lets you create rules to analyze custom CloudWatch Events and create time series. I believe this feature has some analogues to what Honeycomb has been trying to do: surface high-cardinality events and provide powerful tools for analyis across dimensions.
To get a feel for how this can be powerful, check out the built-in rules provided for DynamoDB analysis. This thing can surface heatmaps of frequently-accessed keys, which is something you used to have to plead with internal AWS support for. That alone is worth the price of admission (fifty cents per rule/month, plus $.02 per million monthly rule matches.)
If you’re not getting the picture by now, the value of all these features is directly proportional to your reliance on CloudWatch - a service many AWS customers have historically been reluctant to double down on, finding it scattered and feature-poor compared to existing monitoring tools. What AWS has done pre-re:Invent this year, in my opinion, has changed that calculus a little bit.
Let’s wrap up with a couple more features that don’t directly relate to app observability, but still surface important information that wasn’t easily accessible until now.
The CodeSuite (CodeCommit, CodeBuild, and CodePipeline) continues to lag behind the CI/CD tooling pack in terms of DX, but the will to catch up may be evident with the release of test reports for CodeBuild. Instead of squinting at raw job output, now you can export your JUNIT XML or Cucumber JSON test results into a nice-looking GUI, and even analyze trends to some extent with aggregates. Cool!
This is what it sounds like: anomaly detection for unexpected CloudTrail events, like an API call that starts happening more frequently than usual. Previously you were writing a bunch of custom code to figure this out, or more likely piping CloudTrail logs to an external tool. Pricing is based on the number of events analyzed — $.10 per 100K — so that could get spendy in huge environments, but might be worth it to help find unexpected S3 bucket reads like the one that smacked CapitalOne.
If there’s one consistent theme in the pre-re:Invent announcements this year, it’s that AWS is finally giving developers the tools they need to understand and operate complex service-based workloads. As always, AWS tends to release minimal versions of their services on Day 1, so don’t be surprised if these services are defined at first more by what they can’t do than by what they can. But as the very existence of these updates testifies, AWS remains committed to improving its platform, and we developers continue to benefit from the expanding ecosystem.
Now, what kinds of services will you be monitoring with all these new features? To find out, you’ll have to tune into the big show! See you in Las Vegas…
Trek10 obsesses over the firehose of AWS re:Invent feature releases so you don’t have to. Let’s work together.
