How to Reduce Microsoft Fabric Capacity Costs with Azure Runbooks

Did you know that inefficient capacity management can inflate your cloud costs by up to 80%? For businesses leveraging Microsoft Fabric, over-provisioned resources and idle capacities often lead to unnecessary expenses. If you’re looking to optimise your cloud spending without compromising performance, capacity automation is the key. This guide demonstrates how to automate Microsoft Fabric capacity management with Azure Runbooks, effectively reducing costs.

Understanding Microsoft Fabric Capacity Costs

First, let us revisit how Fabric is billed. When using Microsoft Fabric, your costs will originate from two sources: computing and storage.

Microsoft Fabric offers various capacity SKUs like F2, F4, F8, F16, F32, and F64, each representing different levels of computational power. The higher the capacity tier, the greater the compute power—and the higher the cost. Storage, on the other hand, is billed for the amount of GB you have in various types of Storage (Standard, BCDR, and Cache).

Microsoft Fabric Capacity Pricing Structure

To better understand how Microsoft Fabric is billed, let’s break down the monthly cost of running an F64 capacity in the North EU region, assuming it’s running 24/7 for a 30-day month and using 1,000 GB of standard storage. If you operate in a different region, you can check your regional pricing for Microsoft Fabric here: Microsoft Fabric – Pricing | Microsoft Azure.

Compute Costs for F64 Capacity

SKU	Capacity unit (CU)	Pay-as-you-go (monthly)	Reservation (monthly)
F64	64	€8,220.401	€4,889.568~40.5% saving over Pay-as-you-go

Storage Costs

OneLake is billed at €0.0213/GB/month for normal storage, €0.0395/GB/month for BCDR (Business Continuity and Disaster Recovery – can be optionally enabled or disabled), and €0.2223/GB/month for cache (for KQL cache and retained Data Activator data). To simplify our calculation, we will assume that we have no streaming workloads and no BCDR activated for our OneLake data.

• Data Stored: 1,000 GB
• Monthly Storage Cost: 1,000 GB * €0.0213/GB/month = €21.30

Total Monthly Cost

• Total Cost (Pay-as-you-go): €8,220 + €21.30 = €8241,30
• Total Cost (Reservation): €4,890 + €21.30 = €4911,30

As seen in the example, compute costs vastly outweigh storage costs, accounting for over 99% of the total bill. Likewise, we can see that a reservation makes sense for workloads that need to run continuously. What should we do, however, if we do not require our capacity to run 24/7? 

The primary benefit of the Pay-as-you-go option is that it only incurs compute costs when the capacity is active. Hence, when a capacity is deactivated (suspended), only OneLake storage is billed. This opens the door to cost optimisation through efficient capacity management.

Let’s say we want our capacity to run only on workdays for twelve hours a day, from 7 am to 7 pm. In this scenario, our total costs would be:

• Total Cost (Pay-as-you-go): €8,220 * 12/24 * 5/7 + €21.30  = €2957,01

Adjusting our capacity uptime reduced our compute bill by approximately 64,3%, allowing us to save even more than if we had chosen the reservation option. While this is promising, managing Fabric capacities manually can be time-consuming and prone to errors. This is why we chose to automate our capacity operations with Azure Runbooks.

Step-by-Step Guide to Implement Capacity Automation

Step 1: Set Up an Azure Automation Account

To get started with this automation method, you’ll need an Azure Automation account.

1. Navigate to the Azure Portal: Log in to your Azure account and go to the Azure Portal.
2. Create a New Automation Account:
   • Click on “Create a resource”.
   • Search for “Automation Account” and select it.
   • Click “Create” and fill in the necessary details like Subscription, Resource Group, Name, and Region.
   • Click “Create” to set up the account.

Step 2: Authorize Azure Automation on Fabric Capacity

Add your automation account’s managed identity as “contributor” to your fabric capacity under “Access control (IAM)”.

Step 3: Create a Runbook

An Azure Runbook is a collection of tasks that perform automated processes.

1. Access Your Automation Account: Go to your newly created Automation Account.
2. Add a Runbook:
   • Click on “Runbooks” in the left-hand menu.
   • Click “Create a runbook”.
   • Provide a Name (e.g. “suspend_or_resume_fabric_capacity”), select “PowerShell” as the Runbook type, and click “Create”.

Step 4: Insert the Automation Script

Copy and paste the following PowerShell script into your runbook. This script automates the suspension or resumption of your Microsoft Fabric capacities. Alternatively, you can import a similar script from the Azure Runbook’s gallery by searching for “fabric”.

powershell

Param(
    [string]$ResourceID, # e.g. "/subscriptions/12345678-1234-1234-1234-123a12b12d1c/resourceGroups/fabric-rg/providers/Microsoft.Fabric/capacities/myf2capacity"
    [string]$operation   # "suspend" or "resume"
)

Connect-AzAccount -Identity

# Retrieve the access token as a SecureString
$tokenObject = (Get-AzAccessToken -ResourceUrl "https://management.azure.com/" -AsSecureString).token 

# Convert the SecureString to plain text
$token = ConvertFrom-SecureString $tokenObject -AsPlainText

$azContext = Get-AzContext
$azProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azProfile)
$tokenObject = $profileClient.AcquireAccessToken($azContext.Subscription.TenantId)
$token = $tokenObject.AccessToken

$url = "https://management.azure.com$ResourceID/$operation" + "?api-version=2022-07-01-preview"
Write-Output $url

$headers = @{
    'Content-Type'  = 'application/json'
    'Authorization' = "Bearer $token"
}

$response = Invoke-RestMethod -Uri $url -Method Post -Headers $headers

$response

Explanation of the Script:

• Parameters:
  • $ResourceID: The unique identifier for your Microsoft Fabric capacity. You can find this information under the “properties” tab of your fabric capacity.
  • $operation: The action to perform—either “suspend” or “resume“.
• Authentication: Connects to your Azure account using managed identity.
• API Call: Sends a POST request to the Azure Management API to suspend or resume the specified capacity.

Once we’ve inserted the code, we can test our Runbook to verify that it works as intended:

1. Publish the Runbook:
   • In the runbook editor, click “Save” and then “Publish”.
2. Test the Runbook:
   • In your Runbook’s “overview” tab, click on “Start” and insert your Parameters $ResourceID and $operation (either “resume” or “suspend”) for your run. 
   • Click on “OK” to start the job.
   • You will be redirected to your job’s logging page. If everything is configured correctly, your job should complete without errors and exceptions after a couple of minutes.

Step 5: Schedule the Runbook

Now, let’s automate our runbook to run at specific times.

1. Create two schedules (one for “resume” and one for “suspend”):
   • In your Automation account, under “Shared Resources”, click “Schedules” and then “Add a schedule”.
   • Click “Create a new schedule”.
   • Provide a Name, Start Time, and Recurrence pattern.
   • Click “Create”.
2. Link the Schedule to the Runbook:
   • In your Runbook settings, go to “Schedules” under “Resources”
   • Click “add a schedule”
   • Link your Runbook to one of your recently created schedules
   • Set the Parameters $ResourceID and $operation for each schedule (either “resume” or “suspend”).
   • Click “OK” to finalise.

Step 6: Monitor and Adjust as Needed

Regularly check the runbook’s performance.

• View Job History: In the runbook overview, click “Jobs” to see past executions.
• Adjust Schedules: Modify the schedule if your capacity usage patterns change.
• Update Script: Edit the runbook script for any changes in your environment.

Limitations

Although capacity automation offers the potential for significant cost savings, it’s important to be aware of its limitations. Pausing a capacity will prevent all workloads within its associated workspaces from executing. This means that dashboards will no longer refresh, warehouses will no longer execute queries, and new jobs will be rejected until the capacity is resumed. If you have workloads that require 24/7 running, like streaming analytics or a global workspace across time zones, this kind of optimisation will not be possible with the associated capacity. In those cases, it could be beneficial to split your capacities between continuous and intermittent (or regional) workloads to automate the tasks that can be scheduled and downsize the computing needed for your continuous workloads. An example of this kind of optimisation is shown in the figure below:

Further Considerations

Azure Automation Costs: Azure Automation comes with 500 free runtime minutes per subscription, which should be more than sufficient to automate your fabric capacity for free. Once you exceed 500 minutes, each additional minute is billed at €0.002/minute.

Architectural alternatives: While Azure Automation is a particularly convenient and cost-effective method for automating your capacity, there are also other strategies. Using Logic Apps to send requests to the Fabric REST API is another elegant way of solving this problem.

Conclusion

In this guide, we explored how to optimise our Fabric Capacity cost structure with Azure Runbooks. We’ve seen that by implementing capacity automation, Fabric Administrators can significantly reduce their compute costs while optimising resource utilisation. 

Automating capacity utilisation is a practical first step towards higher ROI on your tech investments. However, there is much more to consider when managing and administering Fabric capacities. Here at Devoteam, we are committed to helping our customers become data-driven and AI-empowered enterprises through comprehensive and fully automated solutions. If you’re interested in how to optimise your cloud data platform operations further, don’t hesitate to contact us. Start automating today and take control of your cloud expenses.

Resources

• Azure Automation Documentation: Learn more about Azure Automation features, including Azure Runbooks.
• Microsoft Fabric Capacity Management: Official Microsoft guide on managing Fabric capacities.
• Optimise Costs with Azure Advisor: Use Azure Advisor to get personalised recommendations for further improvements.