Big data will give you what you need. Even if you don’t know what it is.

Big data is a term broad as the data sets, the processing of which it refers to. The scale of possible scenarios of application is blurring the term itself, making it too general for experts. Just another buzzword, a word-container, where you can throw in everything with just a little effort. Still, the processing of large data sets has very specific practical and fiendishly spreading applications. One of them is the editorial process of horizontal web portals.

The use of big data in the front-end of horizontal portals visited by millions of users daily, may not seem evident at first for a casual user or even a developer who have never had any experience with designing such sites. However, it is the processing of big data sets what makes the portals look as they are – both in terms of UI and UX, and even  to a significant degree  regarding their content quality. The CTR parameter plays the key role in this process.

The subtle magic of click-through-rate

CTR, i.e. click-through-rate is usually associated with the advertising industry. The comparison of the number of views with the number of clicks allows to measure the efficiency of a campaign. However, not everyone knows that it also serves the editorial process of the largest portals. Most of them look similar, single units of content presented by an image and a headline. These are the onlay instruments the editor has to get the reader’s attention. Most of the time the editors can create some variants of the combination. That is where the big data comes in.  

First the aalgorithms shuffle different combinations of headlines and images measuring the CTR rate continuously, i.e. the view and click ratio. After a few minutes the software used by the websites has enough data to evaluate which combination received the highest CTR. I.e. in which combination the number of clicks is closest to the number of views. One could seem that the process is accomplished by human editors, however that is incorrect. The last vote goes to the result of the processing of a data set assorted of thousands of clicks per minute.

Eat or be eaten

However, this is only the beginning of the adventure, the preliminary phase of the real game. After identifying the combination of the headline and the image with the highest CTR rate, it is set off to compete for a place on the main page of the popular horizontal portal with other units of content represented by their own best combinations of headlines and images. However, here a gradation takes place – there is no „everyone against everyone” struggle. The main pages are divided into sectors.

These sectors mostly represent the so-called folds. The length of a single fold is relative and depends on the screen resolution of the user. To put it simply, we can assume that the first fold is a fragment that is viewed in the browser window after entering the site. The second fold is a fragment of the site that is viewed after scrolling through the first one. In most situations the folds set the sectors of units of content with a similar rate of significance, which are processed on the basis of CTR.

Big data will give you what you need

Big data solutions subject each publication to the click ratio and if it reaches a high enough CTR ratio – they decide whether the given content will remain on the main page or be hidden. It takes but a few minutes for the algorithms to evaluate the effectiveness of a given combination. We are talking about constant A/B tests measuring the interest given in hundred of thousands of clicks, constant decisions regarding whether information will reach millions of users, or will be discarded forever.

Here the automatic processing of large numbers of data is an equivalent to the invisible hand mechanism, the high point of the democratisation of modern media. However, the process is not visible at first, it is the users not the editors that decide how the main pages of the largest portals look. The content selection is a mirror reflection of preferences and needs. Thanks to big data millions of readers can be provided with content that they expect. Even if those readers aren’t sure themselves, what they actually expect.

Author:

Agnieszka Szatanowska

Strategy and Project Management Department Director