Every other week I come across yet another article purporting new ways to garner “insights” about consumers, markets, employees, or any other segment companies care about.

It is as if these insights are an enigma that all companies are on a quest to understand. What if I told you there is no magic recipe? All it takes is a simple principle that is often overlooked: being curious.

Before I explain further, let me share some background about me which is relevant to the bigger story here. I have been in the data analysis space for over fifteen years, long before it became a sexy buzzword. I have worked in the corporate sector, with governments and NGOs, and taught research at the university level.

Throughout my career I have had some big wins. I was able to uncover insights enabling companies to solve consumer related challenges they weren’t even aware of, help organizations figure out how to successfully scale initiatives through data-driven decision making, even prevent a major lawsuit for a large corporation, and everything in between.

Those wins weren’t because of some special skillset within my technical abilities. I knew people who were a lot smarter than me, better trained, and with more experience. I had one advantage that gave me the upper hand, an innate curiosity and an addiction to the journey of exploration.

What that meant for the datasets that I dealt with, consumer related or otherwise, is that I left no stone unturned. I manually conducted a surgical level of analysis.

I segmented the data in every way possible.

I then segmented the segments.

I controlled for every variable available. Then saw how my outcome variables were influenced, through regression models, correlations, or significant testing.

I reduced the analysis to the most meaningful unit.

If it meant I needed to run 100 regression models controlling for all variables to see if geographic location in the United States was a predictor of consumer behavior, that’s what was done.

Why 100 models and not 50?

If there were two variables for gender, male and female, and 50 U.S. states, in order to know for sure whether gender had an influence on consumer behavior in each state, 100 models it was.

That was only the beginning.

Imagine if you had variables for race, or job type, or others, the number of permutations grows into the thousands.

Certainly this was not something I could do manually, or even with the help from the team of data scientists I managed.

To the extent time allowed, before a presentation to a board of directors or a client representative, I conducted as many analyses as possible because I was genuinely curious to find out how the results changed or didn’t.

Impractical?

It was.

Guess what happened in the process? Insights were uncovered that companies were desperate to have and needed to grow their businesses. Major strategy decisions were made, findings were presented that changed the direction of lawsuits against Fortune500 companies, and people started to take notice.

A gift and a curse.

That gift of curiosity also became a curse as the demands on my time increased. The workload felt unbearable at times. This wasn’t sustainable. There had to be a way to automate some of my work. There were many sleepless nights before the proverbial light bulb went off in my own head.

Fast forward to a few years ago when I left the corporate world behind. With the help of a co-founder and ironically, more sleepless nights, we set out to build a platform for the old me.

The bright idea.

The vision, use the power of available technologies, machine learning, and artificial intelligence, to instantly uncover deep insights about human psychology – how people feel, act, and think.

Armed with that, we can make close to real-time strategic decisions informed by real statistical data and analysis of written feedback at scale.

Bringing the most important and relevant insights to life for our clients allows not only us, but them to sleep more too.

For the longest time I thought we were in the business of automating analysis, until it hit me, we are automating curiosity.

"Only with curiosity we can move the needle towards meaningful insights."

Time is the only commodity we never get back in life. Being able to give time back to our users so they can focus on growing their businesses with data based decisions is our metric of success.

Have you ever wanted to know which attributes of your product, service, or brand your customers prefer?

You may have a need to drill down during product R&D for development efficiency, or maybe you have a limited marketing budget and need better target your messaging for maximum engagement.

If you ask consumers to rate the features of your product, there is a chance that they might all end up somewhere in the middle of the scale. This won't tell you much about which features they value most and which they care for the least.

This is precisely where MaxDiff comes into play.

What is a MaxDiff Experiment? 

Whether you’re investigating brand preferences, product features, or messaging, a maximum difference (aka MaxDiff) experiment is a useful research approach for obtaining preference or importance scores for multiple attributes.

A MaxDiff experiment presents your respondents with a series of questions, each containing a group of attributes. Respondents then choose their favorite attribute (best) and the one they are least enthusiastic about (worst). This allows researchers to determine, across a representative sample, which attributes stand out the most.

Conducting a MaxDiff Experiment

In its simplest form, a MaxDiff experiment questions respondents about 4 to 7 attributes, asking them to select the best and worst options from each set. 

But, let’s say you have 10, 15, 20, or even 30 product features you want to test to better understand which your consumers find the most and least compelling. 

Presenting participants with a wealth of attributes and asking them to select only two items representing the anchoring choices is not necessarily a good research practice. This is because you’re missing out on a lot of information about the items that were not selected. Not to mention the fact that having someone pick two options out of 10 or more can be quite overwhelming. 

The alternative option is to divide the total claims into subgroups. For example, dividing 20 items into subgroups of 5.

But then comes the next challenge, which coincidentally brings us back to the title….

What is the one question you should ask before conducting a MaxDiff study?

It is often the case that consumer insights teams ask, “is it randomized?” This is a good and valid question to ask. You should want the items to be as randomized as possible. But, that’s not the question you should ask!

Randomization of 20 items presents its own challenges around combinations and permutations. Deep down, you may have a repressed memory of that probability and statistics course you had to take years ago, where the curriculum asked questions such as how many combinations of 5 item sub-groups you can make out of 20 total items?

No need to fetch your dusty old textbook, the formula is nCr = n!/r!*(n-r)!

No need to bookmark this post-the SightX platform will handle this for you!

For the sake of understanding, let’s put the formula to work. The number of 5 item groups you can create from 20 items is: 20!/5!*(20-5)!= 15,504.

There are a total of 15,504 possible combinations of variables!

Yes, that is the actual number of combinations that one can generate in this scenario. This number of possible combinations increases exponentially when the number of variables increases.

So how do you solve for this? 

Enter Professor Jordan Louviere, whose team at the Centre for the Study of Choice pioneered this work with the concept of “Balanced Design”.

The question that an insights professional should always make sure to ask is: “is the design balanced?”

The following are the criteria for a balanced design that your study should have to avoid inaccurate results:

• Frequency Balance: Each item should appear an equal number of times across your respondents;
• Orthogonality: Each item is paired with each other item an equal number of times;
• Connectivity: A set of items is connected if the items cannot be divided into two groups where any item within one group is never paired with any item within the other group. For example, items: A, B, C, D. We pair AB and CD. If we assign each pair in a separate group we won’t be able to know relative preferences across the items, since A was never used within the pairs of the other group. However, if we had asked pairs AB, BC, and CD, then all items would be interconnected. Even though many pairs (such as AC and AD) had not been asked, we could infer the relative order of preference of these;
• Positional Balance: Each item appears an equal number of times on the left and right.

While asking about randomization is good, asking about balanced design is crucial to the accuracy of your results.

So now with a balanced design, you’re able to conduct a much more reasonably sized study. Practical guidelines for a sample size may range between roughly 200 to 1,000 given your stated research goals and what you’re trying to measure, compare, etc.  

Ready to conduct your own MaxDiff experiments? Reach out to our team to get started today!

It’s always good to get a pulse of the market, the trends, the expectations, both the good and the bad. 

It helps brands, marketers, and advertisers anticipate and plan ahead. And effective planning helps improve your return-on-investment.

Recently, Nielsen conducted a survey of chief marketing officers around the globe. One of the big focuses of their study was on media and ad spend of various types.

A full 82% of CMOs said they are planning on increasing spending on digital media over the next year. Only 30% are planning to increase spend on the traditional media side of the house, with even some expecting a decrease.

This isn’t totally surprising given the trend of things like social media and streaming content. The average expected increase in digital spend was 49%, certainly not pocket change.

Would you jump out of a plane without your parachute?

One of the most surprising revelations in the Nielsen study was that most CMOs also admitted they will be jumping without a parachute. Only 1 in 4 marketers reported being confident that they can effectively measure their return-on-investment (ROI), BUT almost 80% are still planning on increasing their investment in marketing analytics.

If that seems a bit concerning to you, welcome to the club. With the depth of targeting that’s possible now, why would you want to be “fairly confident”?

Many of the CMOs in this study noted that it isn’t more data they’re looking for, they are already “flush with dashboards”, they need better insights and ROI.

Now, it is worth pointing out, even if just for our own sanity, that analytics does not equate to insights, understanding, and the power to act.

It is analysis of your appropriate or relevant consumer and marketing data that leads to real insights about your consumer segments. That is much easier said than done. This is especially true with the growing number of consumer touch points and data gathering tools.

So what to do?

Psychographic segmentation

Get a deeper understanding of your consumer segments. Consumers are not cleanly organized by gender or location or any other broad metric.

Consumers organize themselves into homogenous groups based on how they view your product, lifestyle, values, service, and the way they experience the world around them.

Some social media platforms have amazing targeting capabilities. To be able to micro target, you can increase not only your marketing reach, but effectiveness, and ROI if you segment properly.

Just because you can, doesn’t mean you should

Don’t go jumping out of the plane just yet. How do you know you’re targeting the optimal segment, whether it’s micro targeted or not? The data minded people prefer the mathematical approach to discover groups of similar customers. Make sure that you are not collecting any and all types of data.

Make sure you are collecting the relevant data to identify your ROI.

The goal being to accurately (and quickly) segment customers to achieve more effective marketing via personalization to them.

Target the most ideal group of consumers

If you want to actually improve your ROI, make sure that your targeting on either social or search is towards the most ideal group of customers. Take the time to design studies that collect data over time and measure increases in consumer conversion.

This will allow you to not only detect positive and negative changes, but also be able to pinpoint when a change happens and figure out the “why” quicker and more accurately.

Better consumer segmentation drives ROI through:

Improved prospecting

Better conversion rates

Increased customer loyalty

Higher net promoter scores

The bottom line, it is an iterative process. Conduct thoughtful and insightful studies, thinking through relevant variables, designs, and measurements, that identify tangible drivers of top line growth. Really understanding your customers can’t happen with one data point.

Have you ever wondered how statisticians are able to make predictions about the future?

In our previous piece, When Science Gets Involved in Politics, we discussed the importance of adhering to scientific sampling techniques as a solid first step. Now that we have our well-defined sample, the natural next question is: how do we find answers about the population?

Welcome to inferential statistics!

Estimation in statistics refers to the process by which statisticians are able to make relatively accurate inferences about a population based on information obtained from a sample.

In order to understand how we make that move, it’s important to differentiate the three different distributions:

1. The population distribution of the variable of interest (be it customer satisfaction or product popularity), while empirical, is actually unknown because it is extremely difficult to survey your entire population. 
2. The sample consists of a set of data selected from the population of interest, ideally a representative one (More information on this is available on our last blog)
3. The sampling distribution is a theoretical, probabilistic distribution of a statistic (such as the mean) for all possible samples which has a certain sample size.

It’s important to understand that sampling distribution is theoretical, meaning that the researcher never obtains it in reality, but it is critical for estimation.

Thanks to the laws of probability, a great deal is known about sampling distribution, such as its shape, central tendency, and dispersion. We know that its shape is a normal curve, but you may know this as a “Bell Curve”, which is a theoretical distribution of scores that is symmetrical and bell-shaped.

The standard normal curve always has a mean of 0 and a standard deviation of 1. Because one can assume that the shape of the sampling distribution is normal, we can calculate the probabilities of various outcomes. We can also assume things like the mean of the sampling distribution is the same value as the mean of the population.

Building on this is the Central Limit Theorem, a probability theory that says if a random sample of size N is drawn from any population with a mean and standard deviation, as N grows, the sampling distribution of the sample means will approach normality.

With a larger sample size, the mean of the sampling distribution becomes equal to the population mean, the standard error of the mean decreases in size, and the variability in the sample estimates from sample to sample decreases. So now you can start to see how researchers can have more and more confidence in their results. 

But with estimation, there is always a chance of error. 

The width of Confidence Intervals is a function of the risk we are willing to take of being wrong and the sample size. The larger the sample, the lower the chance of error.

In other words, it refers to the probability that a specified interval will contain the population parameter. A 95% confidence level means that there is a 0.95 probability that a specified interval does contain the population mean; accordingly, there are 5 chances out of 100 that the interval does not contain the population mean.

When the purpose of the statistical inference is to draw a conclusion about a population, the significance level measures how frequently the conclusion will be wrong. For example, a 5% significance level means that our conclusion will be wrong 5% of the time. It is always the case that Confidence Level + Significance Level = 1.

It is possible to make inferences about a population from a sample that is carefully selected. The sampling distribution, a theoretical one, links the known sample to a larger population through an estimation. Because of the properties of the sampling distribution, we are able to identify the probability of any statistic with a certain level of confidence.

Whether you realize it or not this is under our noses every day in the news!

Keep your eye out and next time someone talks about who is ahead in the polls at your next cocktail party, you’ll be armed with a heavy dose of skepticism.

In research, weighting is used to adjust the results of a study to bring them more in line with what is known about a population. And it’s a frequent topic of conversation.

It’s often the case that there is confusion around when to use it, how to use it, and its purpose.

So What is Weighting?

Weighting is an applied correction technique. It is done by multiplying each response that is collected from a consumer by a ratio to bring the total results more in line with what is known about the population.

For example, only 20% of the consumers that responded to your survey were males. However, we know that males make up roughly 50% of the general census population. So that 20% needs to be adjusted to bring the results more in line with the overall characteristics of the population.

In general, for responses that belong to an under-represented category, you assign a certain weight that is larger than the value of 1, and for responses that belong to an over-represented category, you assign a weight less than 1.

But Why Weight?

Ideally, you collect your survey data from a representative sample at the start, so that no weighting is needed.

If you’ve done the work upfront and you simply sample your target population in a representative way that meets the goals of your research, then the analysis is the very next step. No weighting, cleaning, or re-coding is needed.

Weighting techniques become important when there are discrepancies between the actual population you are trying to analyze, and the breakdown of the consumers you engaged. When this happens, no reliable conclusion can be drawn from your collected responses- so we weight.

To apply weighting properly, consumer insights teams and market researchers need to rely on what are called “auxiliary variables”. These are variables that have been measured in the survey itself, and their distribution in the population is already known. Like males in the example above.

Typically, these include demographic variables such as age, sex, gender, marital status, etc. obtained from credible national statistical institutions such as the Census Bureau. They can help consumer insights and research teams make estimates on the larger population.

Populations can be anything from all voting-age citizens of the United States to craft beer enthusiasts in Portland, Oregon, or cat owners in Pennsylvania. Each is unique, and a representative sample for each will look different. 

Now for the tough question: to weight or not to weight?

In published research, top-notch empirical scholars make conflicting choices about whether and how to weight, often providing little or no rationale for their choices.

Additionally, in private discussions among experts, it has been repeatedly found that accomplished researchers express confusion or provide faulty reasons for their weighting choices. This debate is captured below:

Pros:

• Enable researchers to argue for more accuracy and representation of results for the larger population by adjusting for over-or under-represented segments of the population

Cons:

• It increases the standard errors of the statistical analysis, making the overall findings less precise and more variable
• In other words, if you are up-weighting respondents, and counting each as more than one person, the more their answers are exaggerated
• All analysis is effected, to include reported descriptive statistics (means, percentiles, medians, modes), and inferential statistics such as regressions and coefficients
  
  

Again, this is a correction technique. So by that definition, you are fundamentally altering the data you’ve collected from your consumers.

There is a general consensus in statistics literature that weights can be useful for descriptive statistics information like mean, median, mode, and standard deviation.

Our view is that insights professionals and researchers should avoid relying on weighting as much as possible. Opt for focusing more time upfront, identifying and targeting a representative and balanced sample before you get yourself into the muddy area of weighting. 

It will not only save you time on the back-end, but generally make your life a little easier.

Out of the Weeds, Part II

In the first article of our Beyond Buzzwords series, we set out to demystify the meaning of machine learning, showcasing its relevance and practicality in the consumer insights space. This second installment is all about doing the same with natural language processing (NLP), also known as text analytics.

What is Natural Language Processing? 

Natural language processing is widely acknowledged as a subfield of artificial intelligence. Its focus is on enabling computers to process and understand human languages. This allows NLP to perform functions like translations, semantic analysis, text classification, extraction, and summarization.

In practice, NLP relies on multiple disciplines, including computer science, computational power, statistics, and linguistics to understand human communications.

When we talk about NLP, we are usually referring to:

Content & topic categorization

The ability to organize a piece of text into meaningful themes or categories. It could be behaviors, products, or any organizing factor of importance to the end-user.

Speech-to-text and text-to-speech

Converting audio into written text and vice versa.

Document summarization

The ability to extract and create accurate textual summaries based on a large quantity of text.

Named Entity Recognition

The tagging of specific words (e.g. names, organizations, etc.) & parts of speech. 

Sentiment analysis

Identifying the emotional reactions to include types of emotions, frequency, and intensity

How Does Natural Language Processing Work? 

You might be asking yourself, this sounds great, but how does it work? How could a machine understand human language? What type of processes and algorithms are applied?

You might be surprised to know that accurate and actionable NLP outcomes often take hard work. Specifically, the work of computer scientists, engineers, linguists, and industry-specific experts, who do a significant amount of manual (and not-so-sexy) work to get the software to perform “artificially intelligent” tasks.

So, let’s say you have a million product reviews you want to analyze or 1,000 pages of text from consumer interviews, and you would like to extract the sentiments and/or understand the most popular topics

If the NLP software you are using is any good, the first step would be to clean the data. Just like organizing a messy excel sheet, NLP software combs through your results to clean the data- or at least reduce the “level of noise” to a minimum.

This critical first step is called pre-processing and it involves both “normalization” and “tokenization”.

Normalization

Normalization involves tasks like removing non-alphabetical characters, converting letters to lowercase, removing stop words (e.g. the, a, in, etc.), converting numbers to words, and stemming and lemmatization.

For further context, stemming and lemmatization work to reduce words to a common variant- the “stem” or “lemma”. The “stem” is the part of the word to which you add influential affixes such as -ed, -ize, mis, etc. Sometimes this results in words that are not actual words. The “lemma” is the base or dictionary form of the word.

Tokenization 

Tokenization refers to segmenting the text into smaller chunks. This means paragraphs can be tokenized into sentences, and sentences into categories, sentiments, parts of speech, or parsing and then tagged the text with anything meaningful to the user (e.g. name recognition, sentiment, behaviors, etc.).

While there are readily available libraries with codes and algorithms that can perform the above tasks- if you are building your own lexical analysis and framework, you should tokenize your own text.

You might want to do this either because your framework is new, or you want to enhance the accuracy. Or, you could work with a software platform that has custom data sets relevant to your space, already built-in.

Tokenized text becomes a “golden dataset”, which is then used to “train” a statistical model, applied to any new text. This is where you may come across the term “supervised machine learning”.

Statistical Models for Natural Language Processing 

Depending on what you are trying to achieve, there are a variety of statistical models that can be applied. These range from logistic regression models, to Support Vector Machine (SVM), or deep neural learning.

The type of statistical model you choose depends on the structure and complexity of your data and frankly is the result of continuous experimentation to increase the accuracy.  

Hopefully, now, you feel a little better prepared to know what is available for your research. 

But more importantly, be able to evaluate future solutions with a clearer understanding of the science and the technology supporting it.

If you’ve ever worked in the military, cybersecurity, or even corporate compliance- you have most likely heard a debate about the differences between information and intelligence. 

Two very different, but often confused, concepts. Both concepts have very broad meanings when it comes to data and what types of data your organization deals with.

Here, when we talk about either, we’re talking about information or intelligence that comes from direct engagement with any group that your organization connects with. 

Whether it be your customers, employees, or any other beneficiary of your business model, there are vast amounts of information to be gained and intelligence to be acquired.

What is the Difference Between Information and Intelligence 

Information helps describe the world around us, whether in the present moment or perceptions about the past or future. It tells us how the world is. It tells us the what. Being able to find out that what is often the easy part. There are many ways to acquire it. However, information for information’s sake isn’t worth a whole lot if you stop there. Information only becomes intelligence when you start to add context and some actual analysis around it all.

Intelligence guides, predicts, and advises. It is the capacity to use the information you've collected to solve problems. It can tell us the why. Arguably, finding out the why is often the toughest part. Intelligence not only helps us understand why things are the way they are, but it also helps to guide decisions that enact effective change. The keyword being effective.

Decision-Making with Information vs Intelligence

Decision-makers can (and certainly do) enact change just from information alone. But intelligence-driven decisions allow you to have more confidence in what the outcome is likely to be, e.g. effective change. When an organization is making any type of a decision, time and money are always at play. Being confident in those decisions should be a top priority.

The truth of the matter is that regardless of the industry or sector that you are working in, information is most likely coming at you so fast you don’t know what to do with it. Quite possibly, adding “analyzing data for some sort of intelligence” doesn’t squeeze nicely into your already packed schedule. 

You need a better way, a faster way.

Intelligence Driven Decisions with SightX

At SightX, we set out to do one thing- have a major impact on each organization that we partner with. We provide a virtual data scientist that helps analyze the deluge of data you are gathering so you can gain the intelligence needed to make effective decisions.

We understand that the majority of analysts’ time goes to cleaning and organizing data. We are making sure that all of your time- in some cases up to 80% of working hours- is now spent on actual analysis.

Intelligence, not information, is what you need to better understand the people, customers, or employees that you’re engaging with. In today’s world, the most powerful organizations are the ones that combine the best human levels of intelligence, emotional insight, and ability to handle uncertainty with software that automates busy work.

Whether you are trying to gain insights about your employees, customers, operations within your business, or other data based research, there is a pretty good chance that at some point in your career you have been presented with a bar chart similar to the one above.

Maybe the bars represented the levels of employee satisfaction across business units, product preferences, or maybe business performance among various geographic locations.

These types of graphs are widely used across industries, and tend to be helpful in shedding quick insights. Yet, they can be some of the most misleading graphs out there…IF not complemented by other key statistics.

Why? Let’s start by thinking about the meaning of each bar. Suppose the above bar chart represents product testing ratings of four new smoothie flavors.  A market research company asks a sample of 100 individuals to rate the four flavors on a scale of 1 to 10. Overall, the purple flavor gets a 9, orange a 5, yellow a 3 and the green an 8. The market research results are presented and Juice Co. concludes that purple is the top favorite followed by the green, where participants had no particular strong preference for the orange, and disliked yellow.

Now let’s think about the orange for a second. For simplicity’s sake let us assume that the market research company sampled 10 individuals. One scenario to get an average of 5 could be: 5, 5, 4, 5, 6, 5, 3, 5, 7, 5. A second scenario to get an average of 5 for the orange could be: 10, 10, 10, 10, 1, 1, 2, 2, 2, 2.

Notice that while the average of both scenarios is still 5, the distribution of the data leads to a very different conclusion. In the first scenario the conclusion is that for the most part people don’t have a strong preference for the orange flavor at all, while in the second scenario the conclusion is that orange is divisive. People either hate it or love it.

The solution to avoiding misleading results is actually pretty simple, yet not widely practiced. You always want to make sure that you ask for all measures of Central Tendency. You’ve heard of them before, but here they are again in all of their glory: mean, mode, and median. All three summarize an entire distribution of scores by describing the most common score (the mode), the score of the middle case (the median), and the average score (the mean) of that distribution.  Here are some basic characteristics about each one of them:

1. The mode is useful when you are interested in the most common score and when you are working with a limited number of variables. The mode becomes less meaningful when the distribution has many
2. The median is always at the exact center of a distribution of scores. Half of the cases are higher and half of the cases are lower
3. The mean as you well know is the average score of a distribution, and we’re pretty sure no further explanation is needed

The mean, which gets reported the most can be misleading if the distribution is skewed. It is also affected by every score in the distribution, while the mode and the median not so much.

So, next time you get presented with bar charts, make sure that you know more about the central tendency of the distribution (mean, mode, and median). You should also know something about the dispersion of the distribution, such as the standard deviation and sample size. We will discuss these fun concepts and their applications to your business in a later post. Statistics shouldn’t be scary. They can and should be used in everyday business decisions.

“Curiosity is the most superficial of all of the affections; it changes its object perpetually; it has an appetite which is very sharp, but very easily satisfied; and it has always an appearance of giddiness, restlessness and anxiety”

- Edmond Burke, Irish Philosopher and Statesman

Throughout human history, curiosity has been a key element in our evolution. Curiosity has been recognized as a critical driver for learning, understanding, problem-solving, and innovating across all societal contexts ranging from child development, scientific discovery, and into the commercial realm.

George Loewenstein, an American educator and economist explained that curiosity arises when attention becomes focused on a gap in one’s knowledge. All of those information gaps produce a feeling of deprivation. This is an aversive psychological state, thus triggering motivation to find the missing knowledge.

Regardless of your business context, if you are curious, that means you are driven to explore, learn, and ultimately understand your respective audiences and the business challenges you face.

When a culture of curiosity exists amongst your consumer insights team, you ignite a thirst for knowledge, creating competitive advantages like those below: 

Asking “Why” 

Curiosity leads with “why”. 

As a direct result, you create the space and framework to ask questions to better understand, why your consumers are unsatisfied, why consumers prefer the products and services they do, why or consumers behave the way they do.

Innovation

If you don’t know the classic story- the inspiration for the Polaroid instant camera was a result of a question posed by the 3-year-old daughter of the inventor Edwin Land, who got impatient to see a photo her father had just taken. When Edwin explained that the film needed to be processed, his daughter asked “why?”

Curiosity leads to challenging the business status quo and invites teams to disrupt and innovate.

Thorough Analysis 

Data is the most valuable commodity in your business-some say it’s the new oil. Yet the question remains, what do you do with it? 

Curiosity leads to the crucial practice of examining your data in numerous ways until some insight is uncovered.

It is no secret that while overall results are a useful signal, the value of your dataset comes to its fruition when you start dissecting and rotating it in new ways.

By applying a range of analytics to uncover those hidden gems that will drive business decisions, you turn that oil into digital gasoline for your business.  

But without curiosity, the analysis will fall flat.

Brilliant consumer insights teams and marketers will continue to lead the revolution in curiosity. 

They will hire talent with curiosity, they will develop the curiosity skill among their staff, they will create a culture of curiosity, all leading to a better and smarter world.

If we are really able to predict, how is it that we repeatedly fail?

Why is it that ‘experts’ continually miss signs of looming disaster? In his Market Watch article, Brett Arends (August 24, 2015) noted that Wall Street experts failed to predict the housing bust, the majority of economists polled in early 2008 failed to predict the biggest recession in 70 years, all of the experts at the International Monetary Fund failed to predict the financial crisis, and since 2011 most Wall Street experts have missed the crashes in emerging market stocks and commodities.

Chances are, you’re not a statistician and have very little interest in formulas. But understanding their conceptual meanings are critical to the way you engage when presented with predictions, forecasts, and projections. 

In statistics, there is a concept known as “margin of error." It tells us something about confidence levels and the degree of uncertainty one is accounting for when forecasting.

The reality is, every single forecast you see is built upon a model that includes mathematical assumptions, like estimated error. Generally, these are determined by human judgment. Some judgments turn out to be more accurate than others, but time has shown that even experts are pretty lousy at it.

The concept of the assumed margin of error and its implications have been discussed most eloquently by Nassim Nicholas Taleb in his book, The Black Swan: The Impact of the Highly Improbable. In the chapter “The Scandal of Prediction," he argues that forecasting without incorporating an error rate uncovers three fallacies, all arising from the same misconception about the nature of uncertainty. 

Fallacy One: Variability matters. This first error lies in taking a projection too seriously, without heading its accuracy. Yet, for planning purposes, the accuracy in your forecast matters far more than the forecast itself. Therefore, the policies we need to make decisions on should depend far more on the range of possible outcomes than on the expected final number. He shares the dire consequences of financial and government institutions projecting cash flows without wrapping them in the thinnest layer of uncertainty.

Fallacy Two: Failing to take into account forecast degradation as the projected period lengthens. We do not realize the full extent of the difference between the near and far futures. Historically, forecasting errors have been enormous, and there is no reason for us to believe that we are suddenly in a more privileged position to see into the future compared to our predecessors. From Facebook to Apple to Ali Baba -in their early days very few would have predicted their dominance in their respective markets.

Fallacy Three: Misunderstanding the random character of the variables being forecasted. Owing to the Black Swan, these variables can accommodate far more optimistic or far more pessimistic scenarios than currently expected. While there are numerous examples in the tech world, Fab stands out as the quintessential example of a bad prediction: a company that was at one point valued at ~$1bn was acquired in a fire sale for about $20 million.

But what does all of that mean for you?  

Whether you are a venture capitalist, corporation, or non-profit- the next time you're presented with forecasts, projections, and predictions, make sure you put a significant amount of thought into the assumptions and margin of error.