Deciding on a Goal in the Beginning Stages of a Data Analysis Project

Asking the Right Questions

Before beginning an analysis project, it is crucial to identify the questions being asked. This requires communication skills above all.

In a survey, 400 recruiters from technical companies were asked to answer the question, “When I recruit for Business Intelligence/Business Analysis roles, it is important that the students have the following coursework/knowledge…” Their ranked answers to this question follow.

  1. Communication Skills
  2. SQL and Query Skills
  3. Basic Analytics

Another study, run by Gartner, attempted to determine what proportions of analytics project failures were due to organizational failures versus technical failures. The organizations’ responses were as follows.

  • 19% - 100% organizational
  • 59% - 75% organizational, 25% technical
  • 21% - 50% organizational, 50% technical
  • 1% - 25% organizational, 75% technical

So, 99% of organizations responded that at least half of the reasons their data analytics projects failed were due to poor organizational skills, not technical skills! And, of course, one of the most important organizational skills is the ability to communicate with stakeholders. Stakeholders are anyone who could be influenced by the results of a data analysis. In the initial stages of the project, Professor Borg suggests to begin by asking many questions. In particular, the goal should be to determine all the things that can either solve the business problem, effect your data interpretation, or influence the eventual recommendation. The following sections provide an overview of the key questions to ask before even touching data.

S.M.A.R.T. Objectives

The goal of initial investigations, before touching data, is to clearly articulate what problems the data analyst is trying to solve. To accomplish this, the analyst should set a meeting with the primary people who are funding the project. A few questions to ask:

  • What problem is this business having that you hope to solve by developing this project?
  • Can you tell me more about how this problem is affecting the business?
  • What is your ideal outcome of this project?

After the meeting, the analysts job is to to synthesize what was said in the meeting into a S.M.A.R.T. goal. These goals are Specific, Measurable, Attainable, Relevant, and Time-bound.

Vague Goal Increase the number of returning visitors to the website.
SMART Goal 1 Increase the number of returning visitors on a month-by-month basis by 15% compared to the same month last year.

Suppose that further conversations and questions indicate that what the project manager really wants is to increase revenues, and that the reason she has requested increasing the returning visitors is because returning visitors are more likely to spend money on the site. This indicates that the previous SMART goal may not actually be Relevant. With this in mind, the goal is changed to the following.

SMART Goal 1 Increase the number of returning visitors on a month-by-month basis by 15% compared to the same month last year.
SMART Goal 2 Within 2 months, determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.

Now, depending on the data available to the company, this goal may still not be Attainable. If, for example, the company does not currently collect clickstream data, then it will be necessary to first install an infrastructure system to gather that data. With this in mind, the SMART goal could be refined as follows.

SMART Goal 2 Within 2 months, determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.
SMART Goal 3 Within 3 months, install a system that will collect and store click-stream data in a cloud-based relational database. By 2 months after the system is installed, analyze that data to determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.

Elicitation

“Elicitation” is the process by which information is gathered from stakeholders. There are a few goals for the elicitation process, including the following:

  1. Identify your key stakeholders
  2. Identify independent variables to test
  3. Determine whether stakeholders agree about the problem to be solved. Work to create sufficient consensus, if they do not agree.

A few pertinent questions during elicitation:

  • What has been tried before?
  • How did it turn out?
  • What do you think might solve this business problem?

Levels of Analysis

Depending on the company’s culture and level of analytics sophistication, certain approaches may not be convincing. Doug Laney, part of the data strategies research team at Gartner, identifies several levels of the “analytics continuum.”

  1. Descriptive analytics - answers the questions of “what happened.” It is hindsight oriented analytics, represented by typical bar charts and pie charts.
  2. Diagnostic analytics - answers questions about “why things are happening.” This approach includes looking for root causes and doing root cause kinds of analyses.
  3. Predictive analytics - answers the question of what “what is going to happen?” This approach involves forecasting the future.
  4. Prescriptive analytics - This approach involves the system actually giving recommendations.

Diagnostic, predictive, and prescriptive analytics are sometimes referred to as advanced analytics. These may include techniques such as linear regression analysis, machine learning, and heuristic processing.

Doug Laney’s point is that it may be difficult to get organizational buy-in to take action based on the conclusions of advanced analytics techniques that are not understood, at least at a high level, by the appropriate business decision makers. This is especially the case if conclusions reached by the advanced analytics techniques are not in line with those decision makers’ intuitions.

Outcomes of Initial Elicitation

The outcome of the initial elicitation process should be organizational buy-in to a goal for the project that is S.M.A.R.T. There should also be a high-level conceptual agreement on the level of analytics that will be employed to meet the goals of the project.

This does not mean that the elicitation process is over. The analytics process frequently becomes iterative in nature. Initial insights can be shared with stakeholders in subsequent “elicitation” meetings. Those conversations may may result in goals being refined or new goals being developed.

In light of this, what is important is that SMART goals be defined by which the process can be deemed a success or failure, and lessons learned generated and incorporated in subsequent analytics projects.

This content is taken from my notes on the Coursera course “Data Visualization and Communication with Tableau.” It is part of the “Excel to MySQL: Analytic Techniques for Business Specialization” specialization.

The specialization is sponsored by Duke University and this particular course is presented by Professor Jana Schaich Borg.