F3 Lightning: How Meta Accelerates Ads ML Features to Production at Scale

What happened

Production machine learning systems rarely consume raw data directly. Before a model can make a prediction, its feature-serving system may need to read stored signals, filter events to a time window, count or sum values, normalize them, and combine intermediate results.

Where the sources line up

The simplest design to complete this work treats every feature as an independent function. That is easy to understand, but it becomes wasteful at scale, because features are not computationally independent. A seven-day click count and a seven-day impression count may read the same event source and apply the same time-window filter. A click-through rate feature may then reuse both results. Running each feature separately would repeat that filter, and every other shared step, on every request.

Practical impact for readers

F3 approached this differently, by representing feature extraction as a directed acyclic graph (DAG). Each node performs one unit of work, such as reading an input, applying a filter, or calculating an aggregation. An edge means one computation depends on the result of another. The graph’s outputs are the feature values a model consumes.

Who should pay attention now

Engineers author each feature independently in a high-level language. The F3 compiler turns each definition into a small graph, then merges the graphs for every feature a service uses into one merged graph (Figure 1). Where features share an operation, such as the seven-day filter above, the compiler collapses the duplicates into a single node. That shared work then runs once per request, not once for every feature that needs it. At runtime, feature extraction is a traversal of this merged graph.

What is still unclear

F3’s merged graph shared computation efficiently, but it came with a major catch: The graph was also the release, and one malformed definition could hold back every other feature in the merged graph.

Source notes

Meta Engineering