Rationale for an Analysis Plan

Jana Schaich Borg describes the place of an analysis plan within an analytics project as follows:

“Analysis plans ensure that you will address questions that are relevant to your business objectives as quickly and efficiently as possible. The quickest way to narrow in the factors in your analysis plan that are likely to create new insights is to combine simple SQL calculations with visualization programs, like Tableau, to identify which factors under consideration have the strongest effects on the business metric you are tasked with improving. You can then design more nuanced statistical models in other software, such as R, based on the factors you have confirmed are likely to be important for understanding and changing your business metric.”

Necessity of a Plan

The job of a business analyst is to do something that will clarify what a business should do to solve a given business problem. Like all employees, business analysts should work to create value for their employer. This value should be created in the most efficient and structured manner possible. This way, progress is reasonably predictable, and stakeholders can be updated on that progress.

Given those realities, a structured analysis plan is necessary prior to beginning a data analysis project. The larger and more complex the data, the more structured this plan will need to be. Recall that data analysis needs to provide insights into processes that the business has both the ability and willingness to improve. Recall three key questions:

  1. What questions will provide actionable insight?
  2. Will the answers to these questions actually matter?
  3. Are the answers correct?

Question 3 was discussed here. Questions 1 and 2 are considered below.

Approaches to Avoid

Analysis paralysis is the phenomenon where someone is so overwhelmed by all the ways a dataset can be analyzed, that they do nothing at all.

The opposite problem is termed an “analysis free-for-all” by the instructor, and she considers it more dangerous. This is the situation where the analyst just starts analyzing whatever seems fun, interesting, or relevant. This will probably produce results, but they will lack coherence or focus. Again, an analysis project should be started with a business problem in mind.

Structured Pyramid Analysis Plan

The high-level overview of an sPAP is that it is a pyramid, or hierarchy, of variables. The top of the pyramid is a S.M.A.R.T. Goal (Specific, Measurable, Attainable, Relevant, and Time-Bound). The variables that will be used to assess the goal are written under it, in expanding layers of categories and subcategories of issues that will be important to analyze. These categories are derived from conversations with stakeholders in the analysis. This “elicitation” process ensures the categories are relevant to the solutions the company can implement.

More specifically, dependent variables are underneath the S.M.A.R.T. Goal. Independent variables are underneath the dependent variables. Finally, specific analyses and graphs form the base of the pyramid. The bottom levels of the pyramid should be articulated to the point that they include exact variables and column names required to address a question about a specific categories of factors. Ultimately, this level of planning will result in more efficient project execution.

Example S.M.A.R.T. Goal

In 2 months, analyze archived click-stream data to determine the website changes that will most efficiently increase revenue by 15% on a month-by-month basis compared to the same month last year.

Relevant Dependent Variables

  1. Total $ spent per transaction: Clickstream DB, “total spent” field aggregated by SUM over each transaction.
  2. Avg $ spent per transaction: Clickstream DB, “total spent” field aggregated by AVG over each transaction.
  3. Total $ spent per customer: Clickstream DB, “total spent” field aggregated by SUM over each customer.

Relevant Questions and Independent Variables

  • Do specific demographics disproportionately contribute to revenue?
    • Age: “Age” in clickstream DB, divided into 5 groups
    • Gender: “Sex” in clickstream DB
    • Income: “Income” in clickstream DB
  • Do specific behaviors disproportionately contribute to revenue?
    • Longer time on site: Need to find out how to calculate from clickstream DB
    • More visits to site: “Customer ID” in clickstream DB, aggregated by SUM over unique values in “entry”
  • Did specific marketing strategies contribute disproportionately to revenue?
    • Promotional emails: Independent variable?
    • Facebook ads: Independent variable?
    • Tweets: Independent variable?