33x Faster Data Processing
Time reduced from 4 minutes to 7 seconds.
Asda
Data Engineering and Agile Transformation.
Asda Accelerates Business Growth with 97% Faster Data Processing and Agile Transformation.
24x Faster Database
From 12 hours to 28 minutes runtime for customer database processing.
97% Faster Basket Tables
Runtime down from 2 days to 1.5 hours.
Empowering Asda with Agile Transformation and Advanced Data Engineering
Since the change of ownership from Walmart in 2021, Asda‘s strategy has included a strong focus on delivering value from data, with a particular focus on the new Asda Rewards program.
A new Decision Science and Data function was set up to enhance customer-centric data activities, focused on customer insights, Asda Rewards, and retail media. A key part of Asda’s growth strategy, this team wanted to partner with specialists to embed Agile methodologies and to lead their growing data engineering function.
Nimble partnered with Asda’s Decision Science and Data department to turn data into a business asset that would deliver commercial value and drive improved and personalised customer experience across Asda Rewards and retail media.
”Nimble’s hands-on focus on engineering best practices alongside the product development itself is exactly the support I needed.
Jonty HaberfieldHead of Data Product and Strategy at Asda
Asda’s Objective
Deliver opportunities for business growth and improved customer experience with enhanced data capabilities and Agile teams.
The new Asda DS&D team quickly identified the potential to refine how they work by adopting Agile principles to boost efficiency and productivity. At the same time, the DS&D team began to build a set of key data products to enhance their capabilities related to customer insights, segmentation, and audience building, but faced some challenges. The goal was to create a suite of data products, from a single customer view to audience-based decision engines, generating actionable insights for informed decision-making and understanding customer preferences to give them a better experience.
Opportunities Identified for Asda
- To disrupt the market, but a need to stay ahead on innovation and agility
- To improve speed to market and product resilience through technical and process improvements
- To upskill the relatively junior data product engineering squads, especially in Agile working practices and CI/CD development patterns
- To improve data and tooling reliability, to combat issues stemming from teething problems with a new tech stack
- To standardise best practice engineering patterns
Nimble’s Solution
Nimble offered a dual-faceted approach combining Agile Coaching with robust Data Engineering to propel the Asda DS&D team forward together. Our experts were able to refine the tech stack, push for best practices, and modernisation to not only improve but future-proof DS&D’s work.
Data Engineering:
- Spearheaded significant advancements to DS&D’s data management, processing, and business intelligence capabilities
- Helped implement significant cost-savings by hugely reducing cloud data size and processing requirements
- Supported the integration of Machine Learning (ML) and Artificial Intelligence (AI) into Data products. Allowing for automatic prediction and intelligence to influence business decisions and predict outcomes
- Accelerated deployment cycles by suggesting and implementing a Site Reliability Engineering (SRE) model to reduce more traditional deployment lifecycle practices
- Establish smart data models that continue to improve in a future-proof and intelligent way without the need for manual intervention
- Empowered teams to better prioritise and unlock their potential with extensive upskilling in new technologies and methodologies such as table sampling techniques, engineering best practices and prioritisation estimation
- Presented an ML-Ops Operating Model and Framework to roll out enhanced ML-Ops adoption in the DS&D Department
- Designed and implemented engineering best practices, ways of working, and technical frameworks, including source control, testing and CI/CD
- Thorough documentation and transfer of knowledge so the Asda team can continue to improve and maintain the solutions
- Allowed DS&D to take long discussed concepts and ideas and get them live, providing immediate value to the teams and the consumers of those Data Products
- Worked closely across all team engagements and interactions to upskill and mentor junior team members and take them on a journey for growth and self-development.
Agile Coaching:
- Conducted a comprehensive review of current practices to identify opportunities for growth and development
- Facilitated the adoption of an Agile mindset and culture within the Asda DS&D team
- Embedded Agile tools and best practices, increasing the team’s ability to prioritise and deliver at pace E.g. Using and optimising Jira tooling to meet quarterly objectives
- Improved the team’s workflows, increasing team productivity and morale
- Partnering with the engineering and data science squads to design tailor-made agile approaches, running initial ceremonies while training the squads to become self-sufficient.
The Technology Stack
In collaboration with DS&D, Nimble carefully evaluated the available technologies in the current tech stack, architecture, and in-house capabilities to select the best technology for sustainable growth.
- Python – for automation and rigorous testing
- Databricks – for data transformation, machine learning, and governance
- Pyspark – to handle large-scale big data processing
- Azure – as the foundational cloud environment
- Azure Data Factory – for efficient and low-code data operations and powerful data transformation
- Azure DevOps – for end-to-end software development capabilities
- SQL – for robust database management and data analytics
- AI/ML – to build predictive models and enhance product categorisation using LLMs
- GitHub – for cloud-based code repository management
Nimble are also continuing to input into the evolution of the DS&D tech stack, including through their ML Ops Operating Model work.
Discover what's possible for you
Speak to one of our projects team to see how Nimble could help your organisation.
