A recommendation widget can look impressive and still have little commercial impact.
The opposite can also be true: a subtle recommendation may participate in many purchases without standing out during a visual review of the storefront.
Evaluating recommendation performance therefore requires tracking what happens between the moment a recommendation is displayed and the actions that follow.
There is also an important distinction to establish from the beginning:
attributing a sale to a recommendation is not the same as proving that the recommendation caused the sale.
Understanding that difference leads to more useful metrics and more realistic conclusions.
Start with impressions
The first metric is the number of times recommendations were actually displayed.
Without an impression count, absolute numbers of clicks or purchases have limited meaning.
Consider two recommendation placements:
- Placement A: 500 clicks;
- Placement B: 200 clicks.
A appears stronger.
But suppose A was displayed 100,000 times while B was displayed only 10,000 times.
The click-through rates become:
- A: 0.5%;
- B: 2%.
Absolute volume still matters, but the rate provides a better view of how frequently each placement generates interaction.
A click is an intermediate metric
Click-through rate is useful for measuring whether recommended products attract attention.
It should not be treated as the final business objective.
A recommendation may generate many clicks and very few purchases.
That is why the journey after the click matters:
Impression → click → add to cart → purchase
Each stage answers a different question.
Impression
Was the recommendation actually shown?
Click
Did one of the recommended items attract the shopper’s attention?
Add to cart
Did that interest turn into a stronger purchase intention?
Purchase
Did the recommended product eventually become part of an order?
This funnel helps identify both successful stages and drop-off points.
Track add-to-cart behavior
Add-to-cart events are especially useful because they are closer to purchase intent than clicks.
If a recommendation receives many clicks but very few products are added to the cart, several explanations are possible:
- the products attract attention but are not relevant enough;
- price or availability discourages the shopper;
- the product page does not match the expectation created by the recommendation;
- the recommendation is being displayed in the wrong context.
The metric does not identify the cause by itself.
It tells you where further investigation should begin.
What is an attributed conversion?
An attribution rule needs a clear definition.
Consider a shopper who sees a recommendation, clicks a product, adds it to the cart and later completes the order.
There is an observable path connecting the recommendation with the purchase.
Under a defined attribution methodology, that purchase can be classified as attributed to the recommendation.
The important issue is interpretation.
The shopper may already have intended to purchase the product.
They might have found it later through search.
Or the recommendation may indeed have been the mechanism that introduced the product.
Attribution describes an observed relationship in the recorded journey.
It does not automatically establish causality.
Attributed revenue is not necessarily incremental revenue
This distinction is critical.
Suppose products associated with recommendation interactions generate $10,000 in sales during a given period.
According to the chosen methodology, it may be accurate to report $10,000 in attributed revenue.
It is a much stronger claim to say that recommendations generated an additional $10,000 in revenue.
The second statement requires estimating what would have happened without the recommendation.
That alternative outcome cannot be directly observed in the same shopping session.
Controlled experiments help answer that question.
Measuring incremental impact
A stronger approach is to compare equivalent groups.
For example:
- Group A receives product recommendations;
- Group B does not receive the same intervention.
With enough observations, teams can compare metrics such as conversion rate, revenue per session or average order value.
If users are properly allocated and other conditions remain comparable, the difference provides stronger evidence of incremental impact.
This is the basic idea behind A/B testing and holdout experiments.
Without a controlled test, teams can still compare periods, cohorts or segments.
Those comparisons can be useful, but they should be interpreted cautiously because other factors may have changed at the same time.
Reading a recommendation funnel
Consider this illustrative dataset:
10,000 impressions
600 clicks
120 add-to-cart events
60 purchases
The resulting metrics are:
CTR: 600 / 10,000 = 6%
Click to cart: 120 / 600 = 20%
Click to purchase: 60 / 600 = 10%
These values describe the behavior recorded after recommendations were displayed.
Now suppose the 60 purchases represent $6,000 in products attributed to recommendation interactions.
That figure is commercially useful.
It still does not mean all $6,000 would have disappeared without recommendations.
That is the difference between attribution and incremental impact.
Analyze placements separately
A recommendation on a product page operates in a different context from one displayed in the cart.
They should not necessarily be evaluated as a single unit.
Useful breakdowns may include:
- product page;
- cart;
- homepage;
- category page;
- other store locations.
Performance can also be segmented by:
- device;
- product category;
- price range;
- new or returning shopper;
- recommendation strategy.
A single overall average may hide one highly effective placement and another that needs improvement.
Consider cancellations and returns
An order recorded as a conversion is not always final revenue.
Depending on the purpose of the report, teams may need to account for canceled orders, failed payments and returns.
The rule should remain consistent.
If reports count orders when they are created, state that clearly.
If they count only paid or completed orders, document that methodology instead.
Changing the definition between reporting periods makes comparisons unreliable.
Measurement in SmartVitrines
SmartVitrines works with recommendation-journey events such as views, clicks, add-to-cart actions and conversions.
These events make it possible to follow the path between recommendation exposure and subsequent commercial activity.
The purpose of measurement should not be to produce one impressive number.
It should answer practical questions:
- are recommendations being seen?
- are shoppers interacting with them?
- do recommended products reach the cart?
- do they appear in orders?
- which placements perform better?
Those answers are much more useful for continuously improving a recommendation strategy.
Better metrics support better decisions
Evaluating product recommendations only through attributed sales can produce incomplete conclusions.
A stronger framework follows the complete funnel:
- impressions;
- clicks;
- add-to-cart events;
- purchases;
- attributed revenue;
- when possible, incremental impact.
With those layers in place, teams can compare placements, strategies and periods without treating correlation as causality.
Recommendations become more than a visual storefront feature: they become a measurable part of the shopping experience.
SmartVitrines was developed by AGTI to integrate product recommendations into e-commerce journeys and provide data for monitoring those interactions.