About Aspect Capital
Aspect Capital is a global systematic investment manager, founded in 1997 by pioneers in quantitative investment techniques. Aspect has since grown into a team of over 130 professionals, managing over $7.5bn. Research and technology are at the core of what Aspect does, ensuring the innovative evolution of its quantitative models.
Aspect has traditionally maintained most of its Microsoft stack applications and databases on-premises. Devoteam is assisting Aspect Capital to migrate several key workloads to AWS.
The Challenge
Aspect Capital previously relied on workload automation solutions to perform routine tasks, such as Extract Transform Load (ETL), data analysis and forecasting, reporting, and signalling to other workloads.
The large number of tasks performed by these workflows was highly compute-intensive and ran on expensive static machines; often overprovisioned and underutilised between workflow executions. Scaling out when workloads spiked unexpectedly was difficult, given the inelastic nature of the static hosts.
The environments were all managed by in-house staff, which carried a large amount of unwanted overhead. This was a concern for business continuity and disaster recovery as well.
The challenge for Devoteam was to migrate a workload designed to make use of a more elastic compute solution, include orchestrated infrastructure deployments, and provide more control and flexibility of costs.
The Solution
As part of the AWS Migration Acceleration Program (MAP), Devoteam worked closely with members of Aspect Capital’s infrastructure and development teams to perform several exploratory proof of concept (PoC) exercises before augmenting both teams on a day-to-day basis to deliver a working end-to-end solution.
Amazon Managed Workflows for Apache Airflow (MWAA) was wrapped in a reusable and highly configurable bespoke Terraform module. This module not only conforms to Aspect’s own high-security requirements but also to AWS Well-Architected Framework best practices.
AWS Elastic Container Service (ECS) was adopted to host containerised tasks and sub-tasks from the wider workflows. ECS containers are currently hosted on autoscaling AWS Elastic Compute Cloud (EC2) clusters due to the client’s high floating-point precision requirements having ruled out the immediate use of Fargate (AWS serverless compute for containers).
Further migration to Fargate containers can now be easily achieved when the Fargate service can provide the precision that Aspect requires. Baseline EC2 capacity can be purchased upfront, using Reserved Instances, to realise a significant cost discount.
Logging and monitoring of the entire solution is now provided by use of:
- AWS CloudWatch Logs (covering MWAA, ECS, EC2, and app logs)
- AWS CloudWatch Alarms
- AWS CloudWatch Log Metrics
- AWS CloudWatch Metrics
- AWS Simple Notification Service (SNS)
- OpsGenie integrations
- Searchable audit logs – written by the MWAA service
Finally, Devoteam and Aspect team members collaborated on testing of Direct Acyclic Graphs (DAG) within Airflow, to help prepare Aspect Capital for future workload migrations and automated Continuous Integration (CI) pipelines. These pipelines are used for building and deploying DAGs to different MWAA environments and were constructed on top of Aspect’s existing Jenkins CI solution.
Your Success Starts Here
Partner with Devoteam to leverage award-winning tech expertise, agile execution, and a culture of continuous learning.