Counters
Event count is a nice and simple signal to affect ranking. But implementation-wise it is quite tricky:
- counters are constantly changing in time, so to do proper model training and backtesting, you need to maintain a historical view on counter values. For example, if you're counting clicks over products, you need to know the resulting number on each point in time when each click happened.
- global and always incrementing counters may work well for user/session scoped things, but counting clicks over products requires time windowing, as having 100 clicks over the whole lifetime is completely different from having the same 100 clicks, but for yesterday.
In Metarank, there are two different types of counters implemented:
- interaction_counter: a simple global always incrementing counter on interactions. Good for counting number of clicks within session, cart size and so on.
- window_counter: when you need to time-frame all the events. Good for item popularities.
Interaction counter is configured in a following way:
- name: click_count
type: interaction_count
scope: item // you can also count actions by a user/session
interaction: click
refresh: 60s // optional, how frequently we should update the value, 0s by default
ttl: 90d // optional, how long should we store this field if there were no updates
Refresh field can be useful for counters when you don't want to update the value used for inference frequently and want to limit the write throughput to feature store.
Windowed counter has the same semantics as the interaction one, but is configured in a different way:
- name: clicks
type: window_count
interaction: click
scope: item
count: 10 // optional, take only the last 10 clicks performed by the user
bucket_size: 24h // make a counter for each 24h rolling window
windows: [7, 14, 30, 60] // on each refresh, aggregate to 1-2-4-8 week counts
refresh: 60s // optional, how frequently we should update the value, 0s by default
ttl: 90d // optional, how long should we store this field if there were no updates
So this feature extractor will emit a group of following features:
- clicks_7: 12
- clicks_14: 34
- clicks_30: 70
- clicks_60: 124
These feature values will be updated at least every 60 seconds.
Window counters are implemented as a circular buffer of counters. There is an each separate counter for each time bucket, and there are as many time buckets as max window size. As counters are updated frequently, it can be computationally expensive to refresh them on each interaction, so it's usually worth it to limit the refresh rate to something reasonable like 10 minutes.
There is also a way to combine multiple windowed counters into a rate to make streaming computation of CTR/Conversion rates easier.
rate feature extractor is useful to calculate things like conversion rate, click-through rate and other values where you need to divide one interaction counter by another. You can configure it in the following way: - name: ctr
type: rate
# name of the feature extractor used as a dividend
top: click
# name of the feature extractor that is used as a divider
bottom: impression // it's a special synthetic event generated by metarank.
bucket: 24h
periods: [7,30]
scope: item # optional, default item, options: item, item.<field>
refresh: 1h # optional, how frequently we should update the value, 1h by default
ttl: 90d # optional, how long should we store this field if there were no updates
normalize: # optional, disabled by default
weight: 10
In this example, we use a
bottom: impression type of interaction. It's a special synthetic interaction event generated by metarank over the items which were examined by the visitor. See click models for details.As for current version 0.5.x, rate feature can be only scoped to an item.
Rate value can be quite noisy for cases with low number of events. Consider these two CTRs:
- impressions=100, clicks=50. CTR=50%
- impressions=2, clicks=1. CTR=50%
Both of these CTRs have the same value, but have completely different confidence. Inspired by the talk An approach to modelling implicit user feedback from HaystackUS22 conference, Metarank can normalize per-item rate based on a global prior rate:
rate = (a + item_clicks) / (b + item_impressions)
Where
a and b can be defined as a normalization constants, where a / b is proportional to the global prior CTR rate.As an example, imagine that we have the following click statistics:
- over all items: 100 impressions, 10 clicks.
- over item A: 2 impressions, 1 click.
Non-normalized CTR for item A is 50%, which is way above an average 10% CTR for the whole inventory. But we can mix global CTR with per-item CTR:
item_ctr = (10 total clicks + 1 click on item A) / (100 total impressions + 2 impressions on A)
To maintain a constant scaling factor
w, we can define normalized per-item CTR in the following way:item_ctr = (w + item_clicks) / (w * (global_impressions / global_clicks) + item_impressions)
Then, our example above will look in a more reasonable way:
weight | global impressions | global clicks | item impressions | item clicks | global CTR | item CTR | normalized item CTR |
|---|---|---|---|---|---|---|---|
10 | 100 | 10 | 2 | 1 | 10.00% | 50.00% | 10.78% |
1 | 100 | 10 | 2 | 1 | 10.00% | 50.00% | 16.67% |
10 | 100 | 10 | 10 | 3 | 10.00% | 30.00% | 11.82% |
10 | 20 | 5 | 10 | 3 | 25.00% | 30.00% | 26.00% |
With this normalization approach, per-item CTR will have much fewer outlier values for cases with low number of clicks. Choosing the right value of
w coefficient is dependent on your dataset:- typical range is between 5 and 10
- the more outlier CTRs you have, the higher
wshould be - default value is 10
When the
scope: item.<field_name> parameter is set, rate feature can aggregate counters per a specific item field value.This is useful as a way to solve the cold-start problem, when there's not enough interactions made per item, but you can aggregate it per category/color/brand.
So in a case when your inventory has a field
brand=cocacola, you can compute CTR per each brand, using the following feature extractor: - name: ctr
type: rate
top: click
bottom: impression
bucket: 24h
periods: [7,30]
scope: item.brand
- normalization is also supported
- the target aggregating field should have low cardinality.
Last modified 17d ago