Healthy Skepticism

Business analysts creates value out of data. Gathering data, whether by writing SQL queries or obtaining spreadsheets, is important because it is one of the skills that will help you create business value. But data gathering is a means to an end, only. Even if the SQL syntax an analyst writes is perfect, the data’s results won’t create business value unless the questions it answers are meaningful and the answers are interpreted appropriately.

Further, it is important that the analysis provide insight into business processes that a business has the ability and willingness to improve. Some important questions an analyst should consider include:

  • What questions will provide actionable insight?
  • Will the answers to my questions matter?
  • Are the answers to my questions correct?

The third question, in particular, can be addressed now. The only systematic way to avoid generating incorrect data is to build appropriate habits.

In a brief interview for the course on which these notes are based, Elena Grewal, Data Science Manager at Airbnb, suggests:

  • “…being kind of single-mindedly paranoid about your data quality…”,
  • “…developing a data intuition, there’s this kind of sense that you have that something’s not quite right…”, and
  • “…getting really familiar with the data that you’re looking at.”

Note that each of those suggestions require time on the job and experience.

The instructor for the course, Jana Schaich Borg, had the following practical advice:

  • “…Whenever you see a really dramatic or surprising effect in your data that nobody in your team or company expected… it’s probably due to a bookkeeping or coding mistake.”
  • “…Whenever somebody tells you something with extreme confidence, temper your expectations. It’s probably more complicated and messy than they are making it out to be.”
  • “…Seek details. Don’t take your data at face value… figure out what your data really looks like.”
  • “…Challenge yourself to think of different ways to make sure your results and interpretations are correct… Look at subsets in the data and query outputs with your eyes, before … strongly interpreting calculations and aggregate functions.”
  • “…Try to cultivate an insatiable desire to find the right answers to your business problem.”

Jana Schaich Borg concludes the course with this:

“If you are motivated by getting the correct answers, you will be more likely to be motivated to catch mistakes in your queries along the way, that could impact your analysis conclusions. So whenever you are getting frustrated with SQL queries, or disheartened by the messiness of the data you are working with, just remember the broader perspective. You are creating brand new value for your business, that it didn’t even know existed. That’s really exciting. And it’s also worth getting right. So pay attention to details, and be ready to conquer any mistakes in the data or your queries that come your way.”