Optimise AWS Costs: 5 Focal Areas of Cost Optimisation

As you’re already very much aware, AWS Cost Optimisation isn’t a one-off checklist item that one can ‘fire-and-forget’. It’s an ongoing, repeated process. And that can be a chore, especially if you’re the only one driving it.

But what about a more proactive approach to Cost Optimisation? How can we apply an attitude of continual improvement to managing our overall cloud investment, its returns, and other benefits to business outcomes? And how can we leverage the huge number of AWS services to realise this goal?

In this blog, we will explore AWS Cost Optimisation as not just a cost-centric, resource-focused exercise but also a result of adopting a positive and forward-thinking technology and business culture—one where cost savings are the responsibility of the many, not the few.

1. Practice Cloud Financial Management

According to AWS, Cloud Financial Management (CFM), “…enables organisations to realise business value and financial success as they optimise their cost and usage and scale on AWS…”

This is broken down further, into the following items:

Functional Ownership

Create a business function – either an individual or a team – that will take responsibility for driving improved cost awareness in the business. This function should focus its efforts in two ways: centralised and decentralised.

Centralised Approach

This focuses mainly around economies-of-scale incentives, such as Reserved Instances and Savings Plans.

These cost models provide a good degree of flexibility, allowing you to make all-upfront commitments to reserve data-centre capacity (CAPEX cost model) or a pay-monthly commitment (OPEX model), or even a 50/50 hybrid of the two.

Reservation commitments can be made for either 12 or 36 months, either when reserving instances (such as EC2 or RDS), or taking out compute plans (useful for adopters of Fargate and Lambda, for example).

In cases where you know that you are guaranteed to require a certain number of always-on EC2s, pretty much forever, you can leverage the highest cost-saving by choosing an all-upfront 3-year reservation. Typically, you can save in the region of ~62% by doing this. If you cannot commit to that much capital expenditure, you can go halves and pay the rest in monthly instalments across the 3 years. This brings the saving down to ~61%. No capex budget? Then go with the no-upfront, pay-monthly deal, and save ~60%. Dropping down to the 1-year deal with see these discounts drop to around the ~40% mark.

By contrast, if you’re planning migrations from EC2 to Fargate or Lambda, then you might choose to take out one of the more flexible Savings Plans – such as the Compute Savings Plan – which will provide cover across all three technologies whilst you migrate. Despite this, you still make a single up-front reservation of capacity, and are still able to pay all-upfront, monthly or both. Discounts vary in the range of ~30-40% in the 12-month bracket, and move up to ~50% in the 3-year deals.

Decentralised Approach

This is where your new AWS Cost Optimisation function will come into its own, as it begins to drive an improved cultural and behavioural attitude towards cost awareness and avoidance. Constant trade-offs between short-term cost avoidance vs long-term investment in improved business efficiencies (often driven by technology choices) will be of particular focus here.

Finance and Technology Partnership

Ensure that finance and technical teams invest a good proportion of their time to understand each other’s requirements. This is often overlooked, and usually with frictional results.

As your technology teams migrate further away from on-premise and traditional data-centre or fixed infrastructure, so too must your finance team move away from fixed-cost forecasts and budgets (CAPEX) to a more flexible, variable-cost model (OPEX).

Use AWS services such as AWS Budgets & Alarms, AWS Cost Explorer and AWS Cost & Usage Reports to help finance teams become more agile in their own processes and culture. Having better real-time awareness of variable costs and related data will help them to align their management practices with your technology teams’ deliveries and requirements.

Cloud Budgets and Forecasts

As you make better use of cloud technologies, your costs are likely to become more variable. This is a good thing, as it means you are likely to enjoy vastly reduced costs during quiet periods. But you need to protect yourself against sudden spikes in variables costs when demand is high.

One of the ways to do this is to set up AWS Budgets and Forecasts.

You can easily set fixed usage or monetary budgets for a given period, and have AWS send you alerts when your forecast usage is expected to exceed them. These are highly customisable, easy to configure and also offer a sliding-scale model (which is useful for product launches or re-platforming projects).

Also consider using AWS Cost Explorer, which can forecast daily (up to 3 months) or monthly (up to 12 months) cloud costs, based on machine-learning algorithms applied to your historical costs (trend based forecasting).

Cost-Aware Processes

Encourage everyone in your business to think about the cost (investment) and any return on that cost (Return On Investment), before undertaking new projects in particular.

For example, Business Analysts should seek to understand how cost factors have driven existing processes and purchasing decisions, Product Owners should seek to understand how product improvements can reduce demand on digital services and Technical Leaders should support both roles by suggesting innovative use of technology to deliver short term objectives with longer-lasting cost benefits.

Engineers, meanwhile, should enjoy a high level of autonomy in their role, so they can iterate and deliver at high speed. But they should also be constantly educated about the cost implications of their own activities. Developer sandbox accounts should have their own budgets and alarms for example, including escalations to line managers and budget controllers.

You can use AWS Organizations and AWS SSO to provision sandbox accounts to individuals; leverage the organisation-wide billing capabilities of AWS Cost Explorer and AWS Cost & Usage Report to understand and monitor these costs.

Quantify Business Value Delivered Through Cost Optimisation

Whether you are migrating services from one compute or database model to another, applying Savings Plans or Reserved Instances, terminating old EC2 instances, or decommissioning other resources, it’s important to keep a good track record of before-and-after costs.

This can be achieved using AWS Cost & Usage Reports before and after a given cost-reduction exercise. Cost savings can be monitored mid-flight using the sliding-scale model in AWS Budgets.

This is all very useful, but immediate cost savings are not the sole benefit of Cost Optimisation. Longer-term investment in future efficiencies is.

AWS offer the following examples:

Executing AWS cost optimisation best practices

For example, resource lifecycle management reduces infrastructure and operational costs and creates time and unexpected budget for experimentation. This increases organisational agility and uncovers new opportunities for revenue generation.

Implementing automation

For example, Auto Scaling, which ensures elasticity at minimal effort, and increases staff productivity by eliminating manual capacity planning work.

Forecasting future AWS costs

Forecasting enables finance stakeholders to set expectations with other internal and external organisation stakeholders, and helps improve your organisation’s financial predictability.

2. Expenditure and Usage Awareness

Every product team or revenue-generating area of your business should have its own cost-monitoring, budgeting and forecasting capability, as well as the business as a whole having a centralised one.

This decentralised model ensures that those closest to the coal face can remain agile in their delivery, whilst also having ownership, accountability and control of their own costs (and savings).

The three following items should be considered:

Governance

Refine Policies

By leveraging AWS services, such as IAM, AWS Organisations & Service Control Policies and AWS Config, your business can gradually introduce measures to ensure that use or modification of certain services or resources is restricted.

Basic examples might include applying a region restriction to your AWS Organisation, or disallowing the use of expensive, overpowered EC2s instance families.

With time, you can refine these restrictions to target specific AWS accounts or environments, IAM roles or user groups. Your developers might have ample freedom to explore different services in their sandbox accounts, but only on free-tier. A staging environment might allow larger instances but avoid multi-regional deployments, for example. In production, we constantly re-evaluate restrictions to best balance demand and performance.

Once again, lean on AWS Budgets to provide real-time notifications of off-policy activities that perhaps have not been explicitly enforced.

Use Multiple Accounts

AWS has a one-parent-to-many-children account structure that is commonly known as a management account (the parent) account-member (the child) account.

A best practice is to always have at least one management account with one member account, regardless of your organisation’s size or usage. All workload resources should reside only within member accounts.

Use consolidated billing within AWS Organisations to aggregate all costs into the parent account. This is useful for finance teams and bill-payers, for obvious reasons. However, each child account can still contain its own budgets and alerts, which is useful for implementing the distributed model of AWS Cost Optimisation accountability and autonomy in your wider teams.

Aggregating costs and usage in the management account lets you maximise service volume discounts and the use of your commitment discounts (Savings Plans and Reserved Instances) for the highest savings.

AWS Control Tower quickly sets up and configures multiple AWS accounts, aligning governance with your organisation’s requirements.

Track & Decommission Resources

Ensure that you track the entire lifecycle of a workload. This ensures you can decommission or modify workloads or workload components when they’re no longer needed.

You can use AWS Config or AWS Systems Manager to quickly collate inventories of existing resources and configurations, regardless of how many accounts you operate. You can then begin to terminate resources that are no longer required.

Monitor Cost and Usage

Your teams must be able to track and understand their own costs before they can be expected to control them.

There are several things you should consider here.

Configure Detailed Data Sources

“Enable hourly granularity in Cost Explorer and create a Cost and Usage Report (CUR). These data sources provide the most accurate view of cost and usage across your entire organisation. The CUR provides daily or hourly usage granularity, rates, costs, and usage attributes for all chargeable AWS services. All possible dimensions are in the CUR including: tagging, location, resource attributes, and account IDs.”

Consider using AWS QuickSight to visualise and explore your CUR. Other services, such as Amazon Athena can be used to perform custom data analysis, should your organisational scale require it.

Identify Cost Attribution Categories

Your efforts to categorise costs can be greatly supported using AWS Tag Editor, for example, which can be used to manage resource tagging at scale. For example, you might want to tag all currently running EC2s in your development account, or certain DB instances for some other cost-tracking purpose. This can easily be done here.

Use AWS Tag Policies in your Organisation, in order to prevent misused of tagging across your accounts.

Establish Workload Metrics

Using services such as AWS CloudWatch Metrics, begin to align workload success (or failure) metrics with associated cost metrics or budgets in AWS Cost Explorer or AWS Budgets. Use these tools to understand how investments in new technology may have improved or worsened your workload’s output. This in information can be used to help determine overall business value and ROI.

Use Cost Categories

“AWS Cost Categories allows you to assign organisation meaning to your costs, without requiring tags on resources. You can map your cost and usage information to unique internal organisation structures. You define category rules to map and categorise costs using billing dimensions, such as accounts and tags. This provides another level of management capability in addition to tagging. You can also map specific accounts and tags to multiple projects.”

Decommission Resources

The basic steps to decommissioning resources are as follows:

Track Resources Over Their Lifetime

Tag, tag and tag some more, especially in pre-production environments where resources are often short-lived but regularly left running. If you deploy resources automatically, in a CICD pipeline, make the CICD pipeline responsible for tearing them down again. But tag everything. Any resources that remain can then be tracked using the AWS Cost Explorer.

For longer-lived resources, consider tagging them with future dates for decommissioning, if these dates are known. Have a process for filtering those tags and reviewing their accuracy.

Implement a Decommissioning Process

“Implement a standardised process across your organisation to identify and remove unused resources. The process should define the frequency searches are performed, and the processes to remove the resource to ensure that all organisation requirements are met.”

Decommission Resources

“The frequency and effort to search for unused resources should reflect the potential savings, so an account with a small cost should be analysed less frequently than an account with larger costs. Searches and decommission events can be triggered by state changes in the workload, such as a product going end of life or being replaced. Searches and decommission events may also be triggered by external events, such as changes in market conditions or product termination.”

Decommission Resources Automatically

“Use automation to reduce or remove the associated costs of the decommissioning process. Designing your workload to perform automated decommissioning will reduce the overall workload costs during its lifetime. You can use AWS Auto Scaling to perform the decommissioning process. You can also implement custom code using the API or SDK to decommission workload resources automatically.”

3. Manage Demand and Supply Resources

The Cloud is billed on-demand by default. This can be a blessing and a curse, as we have discussed in other sections. Cloud resources are also scalable – some vertically and others horizontally. You should work to understand how these two scaling models compare.

For example, a static EC2 instance with a large amount of computational headroom will meet a temporary spike in demand, but it is a sledge-hammer to crack a walnut in cost terms. And if you require redundancy, then your already-high wastage will double or triple. You could introduce the AWS Auto-Scaling feature of EC2 in order to scale up when demand is high, but scale back down when demand is low.

But you also need to consider the speed at which auto-scaling works. Migrating your workloads to AWS ECS+Fargate or Lambda Functions will increase their scaling speed, whilst also reducing the size of each node; so you save twice.

Consider this slicker horizontal scaling model to be ‘just-in-time’ provisioning of resources.

You can also plan to preemptively upscale for known high-traffic events, such an upcoming marketing campaign. AWS Auto-Scaling can be triggered by AWS CloudWatch Events (now EventBridge) either based on real-time events or on a schedule. You can also trigger auto-scaling events via the CLI or SDKs.

You can also use AWS Cost Explorer or Amazon QuickSight with your Cost & Usage Report (CUR) or your application logs, to perform a visual analysis of workload demand. This is useful for future capacity planning and rightsizing requirements.

4. Manage Demand

Throttling

If you’re simply receiving more traffic than you can afford to service, then you can implement throttling on selected services, such as API Gateway.

Buffering

Decouple boundary-layer transactions (such as API calls) from underlying back-end processing. This will allow transactions to execute faster by minimising the scope of synchronous tasks. You can push asynchronous messages to a queue system like AWS SQS and process them later or at a slower rate. This improves the end-user experience and reduces costs.

Designing your workload with a buffer-based approach? Make sure it services requests within the required time and handles duplicate work requests.

5. Optimise Over Time

One of the great benefits of the cloud is that it’s always improving. That’s good for your business because you will be able to benefit from these improvements, all the while avoiding the direct ownership costs of operating bare metal resources in the data centres.

However, it also means that you must stay abreast of new services or improvements to existing ones, to maintain your AWS Cost Optimisation benefits.

Regular audits are your friend, but so is your new culture of shared ownership of costs, savings, and other related optimisations. Continue to run internal workshops, promote ongoing improvements to cost visibility, and learn, educate, and share ideas.

Deliberately set aside time to review and evaluate new services, and prove or disprove their Cost Optimisation potential through POC and MVP deliverables.

AWS Cost Optimisation: What’s next?

AWS Cost Optimisation isn’t just about saving money; it’s about building a more efficient, resilient, and ultimately more valuable cloud infrastructure. By embracing the principles of the AWS Well-Architected Framework’s Cost Optimisation pillar, organisations can gain better visibility into their spending, eliminate waste, and make informed decisions that drive long-term value.

Implementing these best practices empowers you to innovate faster and scale confidently, knowing your cloud investments are strategically aligned with your business goals. The journey of cost optimisation is ongoing, requiring continuous monitoring and adaptation, but the rewards, in terms of both financial efficiency and architectural excellence, are well worth the effort.

Contact our experts and start your optimisation journey now!