Companies’ Relationships to Big Data Analytics

This course presents a five group classification scheme based on how they relate to big data. The rankings are assigned based upon least to most reliant on proprietary software IP for strategic advantage. Practically, this means that the rankings are in order of most to least likely to hire traditional business analysts, and least to most likely to be hiring for more technical software engineering roles.

These classifications include

  1. Strategic consulting firms with a general business focus,
  2. Traditional bricks-and-mortar companies in various industries,
  3. Strategic consulting firms with an IT systems focus,
  4. Companies that sell hardware or software, including big data analytics software, and
  5. Digital businesses, where the core competency involves real-time data analysis.

Strategic Consulting Firms with General Business Focus

  • Provide expert advice about best practices
  • Collect and analyze data to recommend changes
  • Business analysts in large numbers are needed

These companies are paid for advice. They focus on ways to improve business processes in order to increase revenues and profitability and/or minimize risk.

Examples of these types of companies include Bain & Company, The Boston Consulting Group, McKinsey & Company, and Deloitte Consulting. These types of companies are distinguished by analyzing data that belongs to their client companies. They use that data to recommend business process changes. In order to make those recommendations, they need many business analysts.

A minor variation on this type of company are general business consulting firms who have developed their own in-house, proprietary databases. The best example of this kind of consulting firm are credit bureaus, Equifax, TransUnion, and Experian, and FICO, which uses the credit bureau’s data. Two other examples are Argus, which utilizes transaction-level credit card information, and comScore, which monitors web and mobile click streams. Both those organizations essentially sell insights based upon their unique information to client companies.

Brick-and-Mortar Companies

  • Under serious threat from digital Companies
  • Most to gain from best practices
  • Most to lose from failing to adapt
  • Need business analysts, business data analysts, and data scientists

Brick-and-Mortar companies may compete in the same markets as digital companies (Walmart versus Amazon), but in general they will be older and have higher costs. Retail stores, big box stores, banks, and hotels are examples of brick-and-mortar companies. Generally speaking, many of these companies face competition from digital businesses who are more nimble and have less overhead. Brick-and-Mortar companies are by far the largest group of companies in this list.

There are certain sectors of the traditional brick and mortar economy that are under serious threat from digital companies. They also have the most to gain by adopting best practices for big data analytics (and lose, by failing to do so!). Examples of these sectors include all forms of consumer retail purchases, financial services, short-term lodging, and taxi services. The specific companies threatening domination of those sectors are Amazon.com, the entire FinTech group of companies, AirBnB, and Uber/Lyft.

There are incredible opportunities if the large, established, moneyed organizations that are part of this category will properly invest in big data analytics. Professor Daniel Egger provides a 20-item checklist to evaluate how well a given company is embracing big data analytics.

Strategic Consulting Firms with IT Systems Focus

  • Work to introduce new technology into existing market sectors
  • Need some data analysts, more data scientists, and more teams of experience software engineers

This category includes the relatively small number of strategic consulting firms that build custom software systems for client companies. While small, this group tends to be highly influential, as Fortune 500 companies tend to hire them for advice. They are one of the most important forces driving worldwide adoption of analytics.

The world leader in this market is a company called Accenture. An example of the type of project this company would take on is overhauling the federal health insurance website healthcare.gov. Other companies in this category include Palantir and Opera Solutions. Palantir “specializes in analysis, but we don’t actually do the analysis ourselves, we write software that enables experts in their respective fields to extract insight from information.” Opera Solutions “applies advanced analytics to big data flows to extract predictive patterns.”

The goal of IT focused consulting firms is to introduce the newest technologies for achieving competitive advantage into a particular vertical market, such as retail grocery. They will convince one of the major players in that particular market to upgrade. Provided the implementation is successful, the company’s competitors will have no real choice but to upgrade their own systems or risk falling behind. In this way, eventually all the surviving companies in that sector adopt the new technology.

Hardware and Software Companies

  • All of these companies have big data strategies
  • Value depends upon the efficient development and delivery of proprietary technology
  • Need a few business data analysts and a few data scientists, primarily, they need software engineers and project managers

These companies are critical because they provide the software and hardware means of gathering, storing, managing, searching, processing, analyzing, visualizing, and reporting data. This category is also referred to as the “big data information technology” sector. The pricing dynamics of this sector are unusual. Technology has high fixed development costs, but variable costs that are often nearly zero. To survive, these companies need to find ways to be the lowest cost provider of a commodity good or service, or they need to dominate a particular niche so completely that the perceived or actual costs of changing to a cheaper alternative outweigh the benefits.

Egger tells an anecdote about the “million fold” decrease in data storage costs he’s seen in his career. In 1993, he visited the largest commercial database in the world, at the time owned by Mead Data Central. Their acre of mainframe Unix computers and associated software cost over $100M. The 1 Terabyte of test storage was accessible via dial-up modem. The same volume of storage is now routinely purchased by consumers for on the order of $50, and is stored in a smaller volume than most consumer laptops. This rapid improvement is a result of the efforts of these hardware and software companies.

These companies work on different parts of the “big data stack,” a (very) simplified version of which is presented and discussed here. That post also presents examples of the 20 or so companies that make up the majority of the big data commercial landscape.

Digital Companies

  • Use new business models to disrupt traditional markets or forge entirely new markets
  • Driven primarily by developing new information technology
  • Core value proposition is offering products or services better, faster, or cheaper than competitors, by virtue of their grasp or real-time data processing and machine learning
  • Require many software engineers and a few data scientists, but have little need for business analysts and business data analysts

These companies primarily compete against established players in traditional markets, but have a much lower fixed cost structure (consider Redbox versus Blockbuster). These companies rely on massive data processing and real-time business analytics as a core part of their value proposition.

Amazon is the best example of this type of company. It has reinvented multiple retail industries, in 2015 surpassing Walmart to become the largest retailer in the world by market capitalization. Aside from offering products at lower prices and with greater convenience than brick-and-mortar retail stores, they also pursue recurring revenue via their paid Prime membership. More recently, it has dominated the cloud computing sector with its Amazon Web Services. Revenues from this service are estimated to exceed the cloud storage revenues of its three closest rivals, IBM, Google, and Microsoft, combined.

The business model for many of these companies actually offer their services and information for free to consumers, and monetize the content by selling targeted advertising. They, fundamentally, are selling attention and eyeballs to client companies. Analytics forms a core part of their value proposition because they use analytics to make their services as useful and attractive to users as possible. Leaders of these sorts of companies are Google ($50B+ out of ~$70B revenue from advertising), Facebook ($13.5B revenue from advertising), and LinkedIn (over 20% of revenue from advertising).


This content is taken from my notes on the Coursera course “Business Metrics for Data-Driven Companies.” It is sponsored by Duke University and the course content is presented by Professor Daniel Egger.