The big idea
Meta’s next-generation Andromeda brings a more capable model to the first stage of ad selection by designing the model, execution pipeline, and GPU use together. The goal is to evaluate a much larger pool of ads without spending proportionally more compute or exceeding the existing latency budget.
Why it matters
Retrieval determines which ads get a closer look from a slower, more expensive ranking stage. Improvements at this early filter can affect the candidates available downstream. Meta reports that, since 2025, Andromeda has increased Facebook conversions by 2.4%.
Key insights
- 01
Retrieval is approaching ranking-level model performance at a much larger scale.
Meta says its production system scores every eligible ad and surfaces leading candidates at more than one billion ads per second—100 times the throughput of its 2024 system. Separately, it reports model inference throughput above 10 billion ads per second on one GH200 GPU. Those figures describe different measurements and should not be treated as interchangeable.
- 02
Specialized model components replace a single approach to representing users.
The new model uses multiple specialized user models, or experts, to capture changing and contextual interests. An attention-based routing mechanism directs candidate ads toward relevant experts. The ad representation also incorporates real-time serving context. Meta reports near-ranking-level performance with approximately 300 times fewer inference-time floating-point operations than its 2024 retrieval model.
- 03
At billion-scale throughput, moving data becomes a central constraint.
Meta identifies memory bandwidth as a bottleneck when ad data moves through the memory hierarchy. Intermediate data between GPU operations creates terabytes per second of memory traffic, while transfers between CPU hosts and remote accelerators can saturate network links. The proposed system co-design brings these distributed workloads into a single GPU pipeline.
Practical takeaways
- 01
Assess early-stage candidate quality alongside the latency budget.
Meta attributes advertiser benefits to matching against a larger candidate pool within the existing latency budget. For a retrieval-and-ranking pipeline, the first-stage filter is therefore an important place to evaluate both quality and speed.
- 02
Evaluate specialized experts and candidate routing when redesigning a retrieval model.
Andromeda combines specialized user models with a mechanism that selects relevant experts for each candidate. The source presents this as part of its model redesign, rather than as a standalone guarantee of better results.
- 03
Examine pipeline components against the limits of the hardware in use.
Meta frames hardware headroom as a way to understand how much capacity remains for model growth. Its approach examines the retrieval pipeline component by component rather than considering model quality in isolation.