Back to the Future: How Do Statisticians Make Predictions?

Naira Musallam, PhDFeb 7 2017
Back to the Future: How Do Statisticians Make Predictions?

Have you ever wondered how statisticians are able to make predictions about the future?

In our previous piece, When Science Gets Involved in Politics, we discussed the importance of adhering to scientific sampling techniques as a solid first step. Now that we have our well-defined sample, the natural next question is: how do we find answers about the population?

Welcome to inferential statistics!

Estimation in statistics refers to the process by which statisticians are able to make relatively accurate inferences about a population based on information obtained from a sample.

In order to understand how we make that move, it’s important to differentiate the three different distributions:

  1. The population distribution of the variable of interest (be it customer satisfaction or product popularity), while empirical, is actually unknown because it is extremely difficult to survey your entire population. 
  2. The sample consists of a set of data selected from the population of interest, ideally a representative one (More information on this is available on our last blog)
  3. The sampling distribution is a theoretical, probabilistic distribution of a statistic (such as the mean) for all possible samples which has a certain sample size.

It’s important to understand that sampling distribution is theoretical, meaning that the researcher never obtains it in reality, but it is critical for estimation.

Thanks to the laws of probability, a great deal is known about sampling distribution, such as its shape, central tendency, and dispersion. We know that its shape is a normal curve, but you may know this as a “Bell Curve”, which is a theoretical distribution of scores that is symmetrical and bell-shaped.

The standard normal curve always has a mean of 0 and a standard deviation of 1. Because one can assume that the shape of the sampling distribution is normal, we can calculate the probabilities of various outcomes. We can also assume things like the mean of the sampling distribution is the same value as the mean of the population.

Building on this is the Central Limit Theorem, a probability theory that says if a random sample of size N is drawn from any population with a mean and standard deviation, as N grows, the sampling distribution of the sample means will approach normality.

With a larger sample size, the mean of the sampling distribution becomes equal to the population mean, the standard error of the mean decreases in size, and the variability in the sample estimates from sample to sample decreases. So now you can start to see how researchers can have more and more confidence in their results. 

But with estimation, there is always a chance of error. 

The width of Confidence Intervals is a function of the risk we are willing to take of being wrong and the sample size. The larger the sample, the lower the chance of error.

In other words, it refers to the probability that a specified interval will contain the population parameter. A 95% confidence level means that there is a 0.95 probability that a specified interval does contain the population mean; accordingly, there are 5 chances out of 100 that the interval does not contain the population mean.

When the purpose of the statistical inference is to draw a conclusion about a population, the significance level measures how frequently the conclusion will be wrong. For example, a 5% significance level means that our conclusion will be wrong 5% of the time. It is always the case that Confidence Level + Significance Level = 1.

It is possible to make inferences about a population from a sample that is carefully selected. The sampling distribution, a theoretical one, links the known sample to a larger population through an estimation. Because of the properties of the sampling distribution, we are able to identify the probability of any statistic with a certain level of confidence.

Whether you realize it or not this is under our noses every day in the news!

Keep your eye out and next time someone talks about who is ahead in the polls at your next cocktail party, you’ll be armed with a heavy dose of skepticism.

Naira Musallam, PhD

Naira Musallam, PhD

Ready to meet the next generation
of market research technology?

More from SightX

Five Ways to Get Creative with Heatmaps

We’ve said it before- and we'll most likely say it again: consumers are changing.

It should come as no surprise that consumer behavior has evolved quite a bit in recent years, but that evolution was fast-tracked in 2020. From where they shop to how they want to connect with their favorite brands- consumers demand engagement on their terms.

Effective engagement can mean speed and efficiency, but more often than not, it also demands creativity.

For insights teams, in particular, this can be a challenge. However, a modern, effective, and creative way to get impactful feedback from consumers is through a heatmap experiment.

A heatmap is a visual storytelling exercise. It organizes data about an image using color-coded zones representing the frequency of activities, interactions, or sentiments.

Historically, heatmaps have been a popular visualization tool with data-driven researchers across industries. Given current consumer trends, it shouldn’t come as a surprise that heatmaps have been gaining popularity in recent years amongst leading researchers. While they remain a key tool in user interface and experience research, their usage in concept and product testing research continues to gain popularity.

To help spark some creativity and curiosity, we’ve put together a list of simple ways you can incorporate heatmap techniques in your own research:

Whitespace & Prototype Testing
Exploring white space and researching prototypes are important initial steps in the product innovation process. If you have some initial ideas or mock-ups for a product, heatmaps can be an important early indicator about which attributes your potential customers would be compelled by, or (just as importantly) be repelled by.

Efficient and effective prototype feedback allows you to refine your products earlier in the development process- before you even begin building your minimum viable product (MVP).

Design Testing
Getting feedback on visual design elements like fonts, colors, layouts, and imagery is an important step in the research process, and heatmap experiments are one of the most cost- and time-efficient ways to do it.

Using heatmaps for design testing allows you to identify what works and what doesn’t for any customer-facing visuals.

Package Testing
Most products go through many iterations of packaging designs before launch. Testing various concepts with heat mapping allows you to gain detailed insights into potential customers' preferences surrounding specific packaging attributes.

Respondents have the opportunity to select and react to design elements, logo placements, packaging types, and other details - allowing you to understand where consumers focus their attention and in what order.

Ad & Message Testing
Your go-to-market messaging and content strategy can make or break your product launch. However, message testing isn’t just about the words themselves - the taglines, logos, and other copy in the ad are just as important as the package and product designs.

Using heatmaps, you can test which ad or message garners the most positive or frequent interaction, and which drives more viewers to engage with the Call-to-Action. Consumers indicate to researchers where the messaging is catching their attention, if that attention is positive or negative, and why they feel that way.

Shelf Placement
Even though most of us are primarily shopping online, the in-store experience cannot be overlooked. Pandemics aside, consumers will continue walking into stores for the foreseeable future. By testing how a consumer responds to different shopping environments, you can understand how to maximize value both for the customer and your brand during in-store shopping experiences.

Of course, the shelf is a critical point in the in-store customer journey. Heatmaps are a great way to understand optimal shelf placement and product combinations that will entice consumers to reach for your products. They can also help with the design of the shelf itself!

These are just five primary examples of how heatmaps can enhance your consumer research to provide visual, data-driven insights. They are a quick, fun way for consumers to provide insights in a survey setting, and make a great addition to any research report.

Start exploring new use cases and research projects with heatmaps! And of course, reach out to the team at SightX to learn more.

by Naira Musallam, PhD
Image of a sample conjoint analysis featuring vacation packages and visualized results

Conjoint Analysis: Optimize Your Product & Pricing

If you work in or around market research, you have most likely used or (at the very least) heard of conjoint techniques. 

by Naira Musallam, PhD
Beyond Buzzwords: Machine Learning & Consumer Insights

Beyond Buzzwords: Machine Learning & Consumer Insights

Out of the Weeds, Part I

“Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.”

by Naira Musallam, PhD

Research Services