Selection bias in marketing

Almost every decision we make tends to be in­flu­enced by sub­con­scious biases. The technical term for this psy­cho­log­i­cal phe­nom­e­non is “selection bias,” also known as sampling bias. The cognitive bias describes how in­ac­cu­rate results are caused by selection errors (e.g. in the selection of sample units). Selection bias affects market research and should, therefore, be taken into account by brands and companies.

Selection bias sig­nif­i­cant­ly affects the selection of in­for­ma­tion – and not just in market research or sci­en­tif­ic dis­ci­plines but also in everyday sit­u­a­tions. Only those who are aware of their own in­fal­li­bil­i­ty can benefit from the use of biases (also referred to as dis­tor­tions) and make use of their effects. Being aware of such errors in judgement can yield sig­nif­i­cant benefits for companies. By con­scious­ly trying to coun­ter­act selection biases, busi­ness­es can obtain more mean­ing­ful results from their consumer studies.

Selection bias is a sta­tis­ti­cal bias in the selection of sample units. This bias needs to be avoided to obtain mean­ing­ful data and results. For example, in marketing, selection bias jeop­ar­dizes the ob­jec­tiv­i­ty of customer surveys and other market research methods. There are numerous reasons why selection bias may occur, but they often involve either the par­tic­i­pants or the in­di­vid­u­als selecting the sample units. For example, one such bias involves people’s will­ing­ness to par­tic­i­pate in research, provided that this is not randomly conducted. Some well-known selection biases include “par­tic­i­pa­tion bias,” “self-selection bias,”, and “sur­vivor­ship bias.” An example of sur­vivor­ship bias is when an analysis of successes or failures un­in­ten­tion­al­ly only includes the data from the successes (i.e. the “survivors”) in the results.

The pre­cau­tions and coun­ter­mea­sures required to best minimize or com­plete­ly avoid selection bias are rel­a­tive­ly complex. Sta­tis­ti­cal tech­niques can be useful here, such as the Heckman cor­rec­tion, to obtain accurate results in empirical social research or market research.

Selection bias is not just important in research. Busi­ness­es and people in their private lives also tend to select in­for­ma­tion in a biased manner or are presented with biased data due to selection. Cognitive biases con­tribute to us making mistakes in the selection process which then in­evitably skews the results.

Such con­stant­ly recurring selection biases clearly show that we are not impartial and that im­par­tial­i­ty requires a lot of effort. The following examples of selection bias il­lus­trate the far-reaching con­se­quences of sampling bias.

In the first example, a survey on general brand awareness for a health dietary sup­ple­ment is to be conducted. If the survey is conducted in a gym, health food store, or organic su­per­mar­ket, the products’ target audiences are being surveyed. This can be useful. However, the results of such market research must be in­ter­pret­ed with care because selection bias has already occurred. People who frequent gyms, health food stores, and organic su­per­mar­kets are generally more open to the ef­fec­tive­ness and benefits of health products. Therefore, it can be assumed that brand awareness is higher among this group of people because they were not surveyed in a neutral en­vi­ron­ment.

The second example of selection bias il­lus­trates the far-reaching con­se­quences of not making a truly random selection. When studying the economy, economic re­searchers should use a sample that is rep­re­sen­ta­tive of all the companies in a country or region under study. If data selection is based on a register of limited and trading companies, for example, the final sample will exclude small busi­ness­es and suc­cess­ful free­lancers (e.g. lawyers, doctors, ar­chi­tects), artists, and part-time workers.

It is an obvious example and ex­pe­ri­enced re­searchers are unlikely to make such a mistake. However, numerous smaller sampling biases can add up and quickly skew a country’s economic forecast.

Selection bias is mainly a challenge in market research, and less so in active marketing campaigns. But brands should consider potential selection biases when assessing the success of an ad­ver­tis­ing campaign if they want to be able to show accurate marketing results, since campaigns are also a form of market research.

In customer and user surveys, one type of selection bias called self-selection bias plays an important role. This bias occurs where par­tic­i­pants are given the choice to par­tic­i­pate in a survey. If the non-par­tic­i­pants and the par­tic­i­pants have a sig­nif­i­cant­ly different opinion (e.g. dis­sat­is­fied versus satisfied with the company, re­spec­tive­ly), the selection bias can lead to customer sat­is­fac­tion being over­es­ti­mat­ed. The problem of selection bias can and must be minimized here using sta­tis­ti­cal weighting tech­niques.

Selection bias is tricky in marketing because the sampling bias often occurs in con­junc­tion with other kinds of biases. To be able to take the necessary sta­tis­ti­cal coun­ter­mea­sures, it is necessary to un­der­stand the com­bi­na­tion of biases at work. These may include pub­li­ca­tion bias (only positive results are published) or the pre­vi­ous­ly mentioned self-selection bias (only certain groups par­tic­i­pate).

The smaller the amount of available data and/or the smaller the sample taken from it, the more likely it is that selection bias will make data-driven marketing im­pos­si­ble. Any un­de­tect­ed errors can skew studies and surveys and lead to dis­as­trous decision-making errors for marketers. A properly designed study should always ensure that sub­con­scious effects, un­in­ten­tion­al biases, and potential tampering are avoided.

Click here for important legal dis­claimers.