Advertising vs Recommendation

How targeted advertising is killing recommendation engines at your expense

Emile Contal, Mar 17, 2018

You spent a minute looking for a gift for your niece and from then you see pictures of dolls everywhere. Sound familiar? Targeted advertisement services are often framed as recommendation engines to blur the misalignment between businesses’ and individuals’ incentives. It’s so common that most people consider recommendation services to be advertisement platforms in disguise.

Let’s see how targeted advertising differs from a recommendation engine and what it means for a user: you.

What are recommendations and targeted advertising?

First, what do I mean by recommendation? This one is pretty straightforward: you are looking for something matching your personal taste, say music. You use a service where you can enter your preferences, like Spotify. This service then suggests new songs you are likely to love, given your past tastes. You just received recommendations.

Recommendation aims at optimizing your experience by delivering the best items based on your personal tastes.

Targeted advertising, on the other hand, involves third parties. Say advertiser A sells shoes and advertiser B sells travel packages. They each create their ad campaign and pay ad brokers to display their ads. Advertiser A uses Google’s DoubleClick and advertiser B uses AppNexus. You visit a website selling ad slots, such as the DailyNews. In a tenth of second these ad brokers buy your browser history on a data market and then bid to decide which ad will be displayed. If DoubleClick pays the most, you’ll see the fancy shoes. If AppNexus pays the most, you’ll see the cheap exotic travel packages and think about your next holidays.

Advertising aims at optimizing the profits of the advertiser, the publisher and the third parties based on your browsing history.

Let’s go down to the nuts and bolts for a moment. In advertising terms, an “ad broker” is called a demand-side platform (DSP). A DSP is paid when you click on the ad, therefore it computes the bids to maximize the probability of click. The similarity with recommendation comes when the DSP uses your personal data, bought from a data management platform (DMP).

The largest DMPs, like Oracle or Salesforce, know a lot about your age, the websites you visit, your search history, your offline purchases, or your activity in social networks.They use infamous tracking cookies to collect data everywhere and synchronise all your profiles together. The DSP uses this data to predict the probability of click. This is why after searching for shoes on Google, you will see shoes ads among your news articles.

Different technical frameworks

Targeted advertising differs significantly from recommendation for three main reasons: the data involved in these approaches, the over-simplification of user profiles, and computational limitations.

1. Implicit and explicit feedback

First and foremost, recommendation systems are interactive. You can adjust your preferences to modify what you get. Such services primarily use explicit feedback.

On the contrary, advertising systems collect only implicit feedback, often without asking for your permission. Seeing plenty of baby supply during several weeks is frustrating when you simply bought a present to your newly parent friend. The data received by implicit feedback is far less informative than from an interactive process.

As a music expert, would you be able to recommend a good music for someone after a short conversation on music? Surely. Would you be able to do it given only their recent searches on Google? I wouldn’t click on that.

2. Coarse pigeonholing of user profiles

A second major difference comes from the number of third parties involved in the process of real time bidding. Since DSPs (ad brokers) and DMPs (data platforms) are distinct businesses, the processing of your data is materially separated from the real time recommendation. The DMP does not know what are the current advertising campaigns when it compresses its myriads of data points into informative outlines. This process results in over-simplified segmentation of the user profiles in predefined coarse categories. Would you feel understood if I were to stereotype your tastes only by picking some of the following 44 traits?

For a DMP, it is often more profitable to accumulate billions of users’ summaries rather than spending time and money collecting in-depth taste preferences for a few people. A DMP may label you as “30 yo male looking for shoes”, but they don’t know what kind of shoes you prefer, nor whether your friend used your computer for this search.

3. Computational capacity

The third technical contrast I would like to point out is the extremely short time constraint imposed to DSPs to bid on an ad slot: often less than 100ms. This enforces the use of utterly naive algorithms compared to what recommender systems can afford. A prediction algorithm for Real Time Bidding (RTB) will typically select a sparse subset of 30–40 features features where an algorithm for recommendation would use dense representations with dozens of millions parameters.

Combined with the simplification explained above, it’s evident that common recommendation engines have a computational power orders of magnitude larger than the ones of RTB.

What it means for a user

Now that we have laid out the key differences, let’s see how it impacts the user experience. Due to the above technical limitations, ads are far from being tailored to the users taste. The perfect statistic to demonstrate this fact is that the average number of times an ad is clicked is less than 1/1000. Similarly, the average number of conversions (e.g. purchase, download) is less than 1/20,000. A recommendation engine with only one good guess ever 20,000 guesses could not look into the mirror on the morning.

Because they get paid by clicks, online advertisers use well known methods to grab the attention of not-interested users: flashing animations, high visual contrast and sound effects. This obviously harms the online experience and explains why people are rushing on ad blocking software. eMarketer expects 30% of internet users will use one in 2018.

This bidding system itself has an impact on the experience. The selected ad ends up being the one of the DSP that pays the most. What the user sees is not only irrelevant but also heavily biased toward wealthy businesses. Indie bands or film makers will never be promoted when competing with major industries, even if they are a better match for you.

In extreme cases, users don’t see anything without an expensive ad campaign paying for it. Ad slots are not a place for suggesting people to visit this hidden park your neighbors told you about, or the tiny public museum you discovered two years after passing by every day. By contrast, a good recommendation engine, not driven by profit, would definitely pick them up if it works for you. Especially free items and activities!

Let’s rebuild trust in recommendation engines

When trust is damaged, the relationship is changed forever. Harming the online experience makes us more and more suspicious about the algorithmic suggestions we receive. A genuine recommendation engine has to be concerned only about you, the individual, and work for your best interests. I wouldn’t trust recommendations from a platform offering sponsored products, or worse, selling my personal data to third party agencies.

Imagine two futures: one where advertising wins and popups are streamed directly into your optic nerves; the second where AI is here to help you discover things you love. Which future do you want to be part of building? This is why at Crossing Minds we are creating hai, an interactive recommendation engine for media and entertainment, free from any bias. You can sign-up here to try the alpha version: millions of parameters, trained by yourself, for you own profit!