Goal Setting

Asking the Right Questions

Before beginning an analysis project, it is crucial to identify the questions being asked. This requires communication skills above all.

In a survey, 400 recruiters from technical companies were asked to answer the question, “When I recruit for Business Intelligence/Business Analysis roles, it is important that the students have the following coursework/knowledge…” Their ranked answers to this question follow.

  1. Communication Skills
  2. SQL and Query Skills
  3. Basic Analytics

Another study, run by Gartner, attempted to determine what proportions of analytics project failures were due to organizational failures versus technical failures. The organizations' responses were as follows.

  • 19% - 100% organizational
  • 59% - 75% organizational, 25% technical
  • 21% - 50% organizational, 50% technical
  • 1% - 25% organizational, 75% technical

So, 99% of organizations responded that at least half of the reasons their data analytics projects failed were due to poor organizational skills, not technical skills! And, of course, one of the most important organizational skills is the ability to communicate with stakeholders. Stakeholders are anyone who could be influenced by the results of a data analysis. In the initial stages of the project, Professor Borg suggests to begin by asking many questions. In particular, the goal should be to determine all the things that can either solve the business problem, effect your data interpretation, or influence the eventual recommendation. The following sections provide an overview of the key questions to ask before even touching data.

S.M.A.R.T. Objectives

The goal of initial investigations, before touching data, is to clearly articulate what problems the data analyst is trying to solve. To accomplish this, the analyst should set a meeting with the primary people who are funding the project. A few questions to ask:

  • What problem is this business having that you hope to solve by developing this project?
  • Can you tell me more about how this problem is affecting the business?
  • What is your ideal outcome of this project?

After the meeting, the analysts job is to to synthesize what was said in the meeting into a S.M.A.R.T. goal. These goals are Specific, Measurable, Attainable, Relevant, and Time-bound.

Vague Goal: Increase the number of returning visitors to the website.
SMART Goal: Increase the number of returning visitors on a month-by-month basis by 15% compared to the same month last year.

Suppose that further conversations and questions indicate that what the project manager really wants is to increase revenues, and that the reason she has requested increasing the returning visitors is because returning visitors are more likely to spend money on the site. This indicates that the previous SMART goal may not actually be Relevant. With this in mind, the goal is changed to the following.

SMART Goal 1: Increase the number of returning visitors on a month-by-month basis by 15% compared to the same month last year.
SMART Goal 2: Within 2 months, determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.

Now, depending on the data available to the company, this goal may still not be Attainable. If, for example, the company does not currently collect clickstream data, then it will be necessary to first install an infrastructure system to gather that data. With this in mind, the SMART goal could be refined as follows.

SMART Goal 2: Within 2 months, determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.
SMART Goal 3: Within 3 months, install a system that will collect and store click-stream data in a cloud-based relational database. By 2 months after the system is installed, analyze that data to determine the website changes that will most efficiently increase revenues by 15% on a month-by-month basis compared to the same month last year.

Elicitation

“Elicitation” is the process by which information is gathered from stakeholders. There are a few goals for the elicitation process, including the following:

  1. Identify your key stakeholders
  2. Identify independent variables to test
  3. Determine whether stakeholders agree about the problem to be solved. Work to create sufficient consensus, if they do not agree.

A few pertinent questions during elicitation:

  • What has been tried before?
  • How did it turn out?
  • What do you think might solve this business problem?

Outcomes of Initial Elicitation

The outcome of the initial elicitation process should be organizational buy-in to a goal for the project that is S.M.A.R.T. There should also be a high-level conceptual agreement on the level of analytics that will be employed to meet the goals of the project.

This does not mean that the elicitation process is over. The analytics process frequently becomes iterative in nature. Initial insights can be shared with stakeholders in subsequent “elicitation” meetings. Those conversations may may result in goals being refined or new goals being developed.

In light of this, what is important is that SMART goals be defined by which the process can be deemed a success or failure, and lessons learned generated and incorporated in subsequent analytics projects.

Example S.M.A.R.T. Goal

In 2 months, analyze archived click-stream data to determine the website changes that will most efficiently increase revenue by 15% on a month-by-month basis compared to the same month last year.

Relevant Dependent Variables

  1. Total $ spent per transaction: Clickstream DB, “total spent” field aggregated by SUM over each transaction.
  2. Avg $ spent per transaction: Clickstream DB, “total spent” field aggregated by AVG over each transaction.
  3. Total $ spent per customer: Clickstream DB, “total spent” field aggregated by SUM over each customer.

Relevant Questions and Independent Variables

  • Dos specific demographics disproportionately contribute to revenue?
    • Age: “Age” in clickstream DB, divided into 5 groups
    • Gender: “Sex” in clickstream DB
    • Income: “Income” in clickstream DB
  • Do specific behaviors disproportionately contribute to revenue?
    • Longer time on site: Need to find out how to calculate from clickstream DB
    • More visits to site: “Customer ID” in clickstream DB, aggregated by SUM over unique values in “entry”
  • Did specific marketing strategies contribute disproportionately to revenue?
    • Promotional emails: Independent variable?
    • Facebook ads: Independent variable?
    • Tweets: Independent variable?