Adaptive Recommendations and the Cold-Start Problem

Most recommendation engines are built once. Collect a few months of clicks and purchases, fit a model, ship it, then retrain on a schedule. Collaborative filtering, content-based filtering and the hybrids between them all follow that shape, and for a catalogue that holds still they do a decent job.

It is worth saying out loud what the shape assumes. A model fitted on history can only rank on history. It encodes the shop as it was during the training window, then serves that picture until the next refresh replaces it.

What goes stale, and how quickly

Retail rarely holds still for a week. Between one retrain and the next:

• items sell out, come back, or get pulled when a supplier changes terms;
• new products land, usually in batches, with nothing recorded against them;
• the promoted range rotates as commercial priorities move;
• a warm spell shifts demand off coats within days;
• one line takes off for reasons nobody in merchandising predicted.

None of this makes the model wrong. It makes it out of date, which from the shopper’s side looks identical. Running the retrain more often narrows the gap a little and costs a lot, and it leaves the harder problem untouched: a model fitted on logged behaviour can only rank things that have logged behaviour. Everything else is a blank column.

A bandit asks a different question

Multi-armed bandit algorithms come out of reinforcement learning, and they change the question. Instead of “what did shoppers click last month”, it becomes “given what I know so far, what should I show in this slot, and how much am I prepared to spend to learn what I do not know yet”.

Two things are being traded. Showing a proven performer monetises what the system already knows. Showing an item whose performance is uncertain buys information: it costs a little expected click-through now and pays back if the item turns out to be good. A bandit manages that balance on every impression and adjusts it as evidence accumulates.

Picture an A/B test that never finishes and reallocates its own traffic while it runs. A classic test splits traffic evenly, waits for significance, and then someone reads the result and acts. A bandit moves traffic toward the winner as the winning becomes apparent, while keeping a measured share on the alternatives in case the answer changes. In a rotating catalogue it changes constantly, and nothing has to be rerun when it does.

There is a second difference underneath that one. A scheduled retrain learns from whatever data the previous ranking left behind, and that ranking was already biased toward its own favourites. A bandit chooses the data it will learn from.

Context makes the trade sharper

A plain bandit finds the best item on average, and averages are blunt in a shop with a million products.

A contextual bandit conditions the decision on what you know at the moment of the impression: the page, the item on screen, the search term that brought the shopper in, the device, the season, the product’s own attributes. “Best item” becomes “best item here, now, alongside this”. Exploration is spent per context rather than globally, so an item that only works on one category page gets discovered there instead of being written off on its overall numbers.

Reinforcement learning and ordinary supervised learning meet at this point. One half estimates performance from features; the other decides how far to trust that estimate.

Cold start stops being a dead end

A new product under a history-only ranking is caught in a loop that closes on itself. No clicks means no rank, no rank means no impressions, no impressions means no clicks. The item is never judged badly, because it is never judged at all, and on a fast-turning catalogue that can quietly cover a large slice of the range.

Adaptive recommendation breaks the loop from both ends. The exploration budget gets the new item in front of shoppers on merit rather than on record: not because it deserves to win, but because the system does not know yet, and not knowing has a cost worth paying.

The context half means it does not start from nothing either. A product arrives with attributes: category, colour, cut, material, price band, the words in its description. Those features carry real signal, learned from the products that do have history, so a new arrival begins with a usable estimate rather than a blank. That is why attribute quality matters here, and why automatic product tagging sits next to the recommender rather than off to the side.

Live clicks then correct the estimate quickly, concentrating on the items where being wrong is expensive.

What it costs, and when it is worth it

Exploration is a real expense: on some fraction of impressions you are deliberately showing something less certain than your best performer. Size it and monitor it like the investment it is. It also asks more of the plumbing, since click and view events have to flow back promptly.

So the case for adaptivity is turnover, not novelty. A catalogue that changes twice a year, with deep history on everything in it, is served better and more cheaply by a well-tuned classic model. If your range moves faster than your retrain cycle, or a meaningful part of it has no history to rank on, the trained-once approach cannot catch up by working harder.

MLArt built this for a fashion platform in 2022: contextual bandits alongside attribute models, roughly 30% more click-through, and products with no sales record finally appearing in recommendations. The case study has the detail, Solutions covers what it looks like as a product, and Technology sets out the modules underneath. To work out whether your catalogue moves fast enough to need it, write to office@mlart.biz.