2016 US Bike Share Activity Snapshot

Table of Contents

Introduction

Tip: Quoted sections like this will provide helpful instructions on how to navigate and use a Jupyter notebook.

Over the past decade, bicycle-sharing systems have been growing in number and popularity in cities across the world. Bicycle-sharing systems allow users to rent bicycles for short trips, typically 30 minutes or less. Thanks to the rise in information technologies, it is easy for a user of the system to access a dock within the system to unlock or return bicycles. These technologies also provide a wealth of data that can be used to explore how these bike-sharing systems are used.

In this project, you will perform an exploratory analysis on data provided by Motivate, a bike-share system provider for many major cities in the United States. You will compare the system usage between three large cities: New York City, Chicago, and Washington, DC. You will also see if there are any differences within each system for those users that are registered, regular users and those users that are short-term, casual users.

Posing Questions

Before looking at the bike sharing data, you should start by asking questions you might want to understand about the bike share data. Consider, for example, if you were working for Motivate. What kinds of information would you want to know about in order to make smarter business decisions? If you were a user of the bike-share service, what factors might influence how you would want to use the service?

Question 1: Write at least two questions related to bike sharing that you think could be answered by data.

Answer:

1. When does peak demand occur?

2. How does peak demand compare to a more average (baseline) demand level?

This information could help inform decision about variable pricing that scales depending upon the expected level of demand. Consider both these questions for both types of users (registered/regular and short-term/casual) and all three large cities (New York, Chicago, Washington DC).

Tip: If you double click on this cell, you will see the text change so that all of the formatting is removed. This allows you to edit this block of text. This block of text is written using Markdown, which is a way to format text using headers, links, italics, and many other options using a plain-text syntax. You will also use Markdown later in the Nanodegree program. Use Shift + Enter or Shift + Return to run the cell and show its rendered form.

Data Collection and Wrangling

Now it's time to collect and explore our data. In this project, we will focus on the record of individual trips taken in 2016 from our selected cities: New York City, Chicago, and Washington, DC. Each of these cities has a page where we can freely download the trip data.:

  • New York City (Citi Bike): Link
  • Chicago (Divvy): Link
  • Washington, DC (Capital Bikeshare): Link

If you visit these pages, you will notice that each city has a different way of delivering its data. Chicago updates with new data twice a year, Washington DC is quarterly, and New York City is monthly. However, you do not need to download the data yourself. The data has already been collected for you in the /data/ folder of the project files. While the original data for 2016 is spread among multiple files for each city, the files in the /data/ folder collect all of the trip data for the year into one file per city. Some data wrangling of inconsistencies in timestamp format within each city has already been performed for you. In addition, a random 2% sample of the original data is taken to make the exploration more manageable.

Question 2: However, there is still a lot of data for us to investigate, so it's a good idea to start off by looking at one entry from each of the cities we're going to analyze. Run the first code cell below to load some packages and functions that you'll be using in your analysis. Then, complete the second code cell to print out the first trip recorded from each of the cities (the second line of each data file).

Tip: You can run a code cell like you formatted Markdown cells above by clicking on the cell and using the keyboard shortcut Shift + Enter or Shift + Return. Alternatively, a code cell can be executed using the Play button in the toolbar after selecting it. While the cell is running, you will see an asterisk in the message to the left of the cell, i.e. In [*]:. The asterisk will change into a number to show that execution has completed, e.g. In [1]. If there is output, it will show up as Out [1]:, with an appropriate number to match the "In" number.

In [1]:
## import all necessary packages and functions.
import csv # read and write csv files
from datetime import datetime # operations to parse dates
from pprint import pprint # use to print data structures like dictionaries in
                          # a nicer way than the base print function.
In [2]:
def print_first_point(filename):
    """
    This function prints and returns the first data point (second row) from
    a csv file that includes a header row.
    """
    # print city name for reference
    city = filename.split('-')[0].split('/')[-1]
    print('\nCity: {}'.format(city))
    
    with open(filename, 'r') as f_in:
        ## TODO: Use the csv library to set up a DictReader object. ##
        ## see https://docs.python.org/3/library/csv.html           ##
        trip_reader = csv.DictReader(f_in)
        
        ## TODO: Use a function on the DictReader object to read the     ##
        ## first trip from the data file and store it in a variable.     ##
        ## see https://docs.python.org/3/library/csv.html#reader-objects ##
        first_trip = trip_reader.__next__()
        
        ## TODO: Use the pprint library to print the first trip. ##
        ## see https://docs.python.org/3/library/pprint.html     ##
        pprint(first_trip)
        
    # output city name and first trip for later testing
    return (city, first_trip)

# list of files for each city
data_files = ['./data/NYC-CitiBike-2016.csv',
              './data/Chicago-Divvy-2016.csv',
              './data/Washington-CapitalBikeshare-2016.csv',]

# print the first trip from each file, store in dictionary
example_trips = {}
for data_file in data_files:
    city, first_trip = print_first_point(data_file)
    example_trips[city] = first_trip
City: NYC
OrderedDict([('tripduration', '839'),
             ('starttime', '1/1/2016 00:09:55'),
             ('stoptime', '1/1/2016 00:23:54'),
             ('start station id', '532'),
             ('start station name', 'S 5 Pl & S 4 St'),
             ('start station latitude', '40.710451'),
             ('start station longitude', '-73.960876'),
             ('end station id', '401'),
             ('end station name', 'Allen St & Rivington St'),
             ('end station latitude', '40.72019576'),
             ('end station longitude', '-73.98997825'),
             ('bikeid', '17109'),
             ('usertype', 'Customer'),
             ('birth year', ''),
             ('gender', '0')])

City: Chicago
OrderedDict([('trip_id', '9080545'),
             ('starttime', '3/31/2016 23:30'),
             ('stoptime', '3/31/2016 23:46'),
             ('bikeid', '2295'),
             ('tripduration', '926'),
             ('from_station_id', '156'),
             ('from_station_name', 'Clark St & Wellington Ave'),
             ('to_station_id', '166'),
             ('to_station_name', 'Ashland Ave & Wrightwood Ave'),
             ('usertype', 'Subscriber'),
             ('gender', 'Male'),
             ('birthyear', '1990')])

City: Washington
OrderedDict([('Duration (ms)', '427387'),
             ('Start date', '3/31/2016 22:57'),
             ('End date', '3/31/2016 23:04'),
             ('Start station number', '31602'),
             ('Start station', 'Park Rd & Holmead Pl NW'),
             ('End station number', '31207'),
             ('End station', 'Georgia Ave and Fairmont St NW'),
             ('Bike number', 'W20842'),
             ('Member Type', 'Registered')])

If everything has been filled out correctly, you should see below the printout of each city name (which has been parsed from the data file name) that the first trip has been parsed in the form of a dictionary. When you set up a DictReader object, the first row of the data file is normally interpreted as column names. Every other row in the data file will use those column names as keys, as a dictionary is generated for each row.

This will be useful since we can refer to quantities by an easily-understandable label instead of just a numeric index. For example, if we have a trip stored in the variable row, then we would rather get the trip duration from row['duration'] instead of row[0].

Condensing the Trip Data

It should also be observable from the above printout that each city provides different information. Even where the information is the same, the column names and formats are sometimes different. To make things as simple as possible when we get to the actual exploration, we should trim and clean the data. Cleaning the data makes sure that the data formats across the cities are consistent, while trimming focuses only on the parts of the data we are most interested in to make the exploration easier to work with.

You will generate new data files with five values of interest for each trip: trip duration, starting month, starting hour, day of the week, and user type. Each of these may require additional wrangling depending on the city:

  • Duration: This has been given to us in seconds (New York, Chicago) or milliseconds (Washington). A more natural unit of analysis will be if all the trip durations are given in terms of minutes.
  • Month, Hour, Day of Week: Ridership volume is likely to change based on the season, time of day, and whether it is a weekday or weekend. Use the start time of the trip to obtain these values. The New York City data includes the seconds in their timestamps, while Washington and Chicago do not. The datetime package will be very useful here to make the needed conversions.
  • User Type: It is possible that users who are subscribed to a bike-share system will have different patterns of use compared to users who only have temporary passes. Washington divides its users into two types: 'Registered' for users with annual, monthly, and other longer-term subscriptions, and 'Casual', for users with 24-hour, 3-day, and other short-term passes. The New York and Chicago data uses 'Subscriber' and 'Customer' for these groups, respectively. For consistency, you will convert the Washington labels to match the other two.

Question 3a: Complete the helper functions in the code cells below to address each of the cleaning tasks described above.

In [3]:
def duration_in_mins(datum, city):
    """
    Takes as input a dictionary containing info about a single trip (datum) and
    its origin city (city) and returns the trip duration in units of minutes.
    
    Remember that Washington is in terms of milliseconds while Chicago and NYC
    are in terms of seconds. 
    
    HINT: The csv module reads in all of the data as strings, including numeric
    values. You will need a function to convert the strings into an appropriate
    numeric type when making your transformations.
    see https://docs.python.org/3/library/functions.html
    """
    
    # YOUR CODE HERE
    if city == 'NYC' or city == 'Chicago':
        duration = float(datum['tripduration'])/60     # raw data: seconds
    elif city == 'Washington':
        duration = float(datum['Duration (ms)'])/60000 # raw data: miliseconds
        
    return duration    # returns the trip duration in minutes, as an integer

# Some tests to check that your code works. There should be no output if all of
# the assertions pass. The `example_trips` dictionary was obtained from when
# you printed the first trip from each of the original data files.
tests = {'NYC': 13.9833,
         'Chicago': 15.4333,
         'Washington': 7.1231}

for city in tests:
    assert abs(duration_in_mins(example_trips[city], city) - tests[city]) < .001
In [4]:
def time_of_trip(datum, city):
    """
    Takes as input a dictionary containing info about a single trip (datum) and
    its origin city (city) and returns the month, hour, and day of the week in
    which the trip was made.
    
    Remember that NYC includes seconds, while Washington and Chicago do not.
    
    HINT: You should use the datetime module to parse the original date
    strings into a format that is useful for extracting the desired information.
    see https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
    """
    
    if city == 'NYC':
        dt = datetime.strptime(datum['starttime'],'%m/%d/%Y %H:%M:%S')
    elif city == 'Chicago':
        dt = datetime.strptime(datum['starttime'],'%m/%d/%Y %H:%M')
    elif city == 'Washington':
        dt = datetime.strptime(datum['Start date'],'%m/%d/%Y %H:%M')
        
    return (dt.month, dt.hour, dt.strftime('%A'))

# Some tests to check that your code works. There should be no output if all of
# the assertions pass. The `example_trips` dictionary was obtained from when
# you printed the first trip from each of the original data files.
tests = {'NYC': (1, 0, 'Friday'),
         'Chicago': (3, 23, 'Thursday'),
         'Washington': (3, 22, 'Thursday')}

for city in tests:
    assert time_of_trip(example_trips[city], city) == tests[city]
In [5]:
def type_of_user(datum, city):
    """
    Takes as input a dictionary containing info about a single trip (datum) and
    its origin city (city) and returns the type of system user that made the
    trip.
    
    Remember that Washington has different category names compared to Chicago
    and NYC. 
    """
    
    if city == 'Washington':  # only executes for Washington.
        if datum['Member Type'] == 'Registered':
            return 'Subscriber'
        elif datum['Member Type'] == 'Casual':
            return 'Customer'
    
    return datum['usertype']   # only executes for NYC and Chicago.

# Some tests to check that your code works. There should be no output if all of
# the assertions pass. The `example_trips` dictionary was obtained from when
# you printed the first trip from each of the original data files.
tests = {'NYC': 'Customer',
         'Chicago': 'Subscriber',
         'Washington': 'Subscriber'}

for city in tests:
    assert type_of_user(example_trips[city], city) == tests[city]

Question 3b: Now, use the helper functions you wrote above to create a condensed data file for each city consisting only of the data fields indicated above. In the /examples/ folder, you will see an example datafile from the Bay Area Bike Share before and after conversion. Make sure that your output is formatted to be consistent with the example file.

In [6]:
def condense_data(in_file, out_file, city):
    """
    This function takes full data from the specified input file
    and writes the condensed data to a specified output file. The city
    argument determines how the input file will be parsed.
    
    HINT: See the cell below to see how the arguments are structured!
    """
    
    with open(out_file, 'w') as f_out, open(in_file, 'r') as f_in:
        # set up csv DictWriter object - writer requires column names for the
        # first row as the "fieldnames" argument
        out_colnames = ['duration', 'month', 'hour', 'day_of_week', 'user_type']        
        trip_writer = csv.DictWriter(f_out, fieldnames = out_colnames)
        trip_writer.writeheader()
        
        ## TODO: set up csv DictReader object ##
        trip_reader = csv.DictReader(f_in)

        # collect data from and process each row
        for row in trip_reader:
            # set up a dictionary to hold the values for the cleaned and trimmed
            # data point
            new_point = {}

            ## TODO: use the helper functions to get the cleaned data from  ##
            ## the original data dictionaries.                              ##
            ## Note that the keys for the new_point dictionary should match ##
            ## the column names set in the DictWriter object above.         ##
            new_point[out_colnames[0]] = duration_in_mins(row, city)
            
            month, hour, day_of_week = time_of_trip(row, city)
            new_point[out_colnames[1]] = month
            new_point[out_colnames[2]] = hour
            new_point[out_colnames[3]] = day_of_week
            
            new_point[out_colnames[4]] = type_of_user(row, city)

            ## TODO: write the processed information to the output file.     ##
            ## see https://docs.python.org/3/library/csv.html#writer-objects ##
            trip_writer.writerow(new_point)
In [7]:
# Run this cell to check your work
city_info = {'Washington': {'in_file': './data/Washington-CapitalBikeshare-2016.csv',
                            'out_file': './data/Washington-2016-Summary.csv'},
             'Chicago': {'in_file': './data/Chicago-Divvy-2016.csv',
                         'out_file': './data/Chicago-2016-Summary.csv'},
             'NYC': {'in_file': './data/NYC-CitiBike-2016.csv',
                     'out_file': './data/NYC-2016-Summary.csv'}}

for city, filenames in city_info.items():
    condense_data(filenames['in_file'], filenames['out_file'], city)
    print_first_point(filenames['out_file'])
City: Washington
OrderedDict([('duration', '7.123116666666666'),
             ('month', '3'),
             ('hour', '22'),
             ('day_of_week', 'Thursday'),
             ('user_type', 'Subscriber')])

City: Chicago
OrderedDict([('duration', '15.433333333333334'),
             ('month', '3'),
             ('hour', '23'),
             ('day_of_week', 'Thursday'),
             ('user_type', 'Subscriber')])

City: NYC
OrderedDict([('duration', '13.983333333333333'),
             ('month', '1'),
             ('hour', '0'),
             ('day_of_week', 'Friday'),
             ('user_type', 'Customer')])

Tip: If you save a jupyter Notebook, the output from running code blocks will also be saved. However, the state of your workspace will be reset once a new session is started. Make sure that you run all of the necessary code blocks from your previous session to reestablish variables and functions before picking up where you last left off.

Exploratory Data Analysis

Now that you have the data collected and wrangled, you're ready to start exploring the data. In this section you will write some code to compute descriptive statistics from the data. You will also be introduced to the matplotlib library to create some basic histograms of the data.

Statistics

First, let's compute some basic counts. The first cell below contains a function that uses the csv module to iterate through a provided data file, returning the number of trips made by subscribers and customers. The second cell runs this function on the example Bay Area data in the /examples/ folder. Modify the cells to answer the question below.

Question 4a: Which city has the highest number of trips? Which city has the highest proportion of trips made by subscribers? Which city has the highest proportion of trips made by short-term customers?

Answer:

1. New York City has the highest number of trips (276,798).

2. New York City has the highest proportion of trips taken by subscribers (89%).

3. Chicago has the highest proportion of trips taken by customers (24%).

In [8]:
# Dictionary of city names and filenames, to be used throughout the rest of this project
city_to_filename = {
    'Washington': './data/Washington-2016-Summary.csv',
    'Chicago': './data/Chicago-2016-Summary.csv',
    'NYC': './data/NYC-2016-Summary.csv'
}

# Helper method to convert ratios to percentages, to be used throughout the rest of this project
def ratio_to_percent( numerator_or_ratio, denominator=""):
    '''
    Returns a two digit percent (no decimal component) given a numerator and denominator of a ratio.
    '''
    if denominator == "":      # executes if denominator is not defined b/c ratio (as a float) is passed in.
        return int( round( numerator_or_ratio*100 ) )
    else:                      # executes if numerator and denominator are passed in.
        return int( round( numerator_or_ratio/denominator*100 ) )
In [9]:
# Helper Methods for this section
def number_of_trips(filename):
    """
    This function reads in a file with trip data and reports the number of
    trips made by subscribers, customers, and total overall.
    """
    with open(filename, 'r') as f_in:
        # set up csv reader object
        reader = csv.DictReader(f_in)
        
        # initialize count variables
        subscriber_trips = 0
        customer_trips = 0
        
        # tally up ride types
        for row in reader:
            if row['user_type'] == 'Subscriber':
                subscriber_trips += 1
            else:
                customer_trips += 1
        
        # return tallies as a list
        return (subscriber_trips, customer_trips)
In [10]:
# Obtain necessary data
trip_counts = {}
for city in city_to_filename.keys():
    trip_counts[city] = {}
    subscriber_trips, customer_trips = number_of_trips( city_to_filename[city] )
    trip_counts[city]['sub_trips'] = subscriber_trips
    trip_counts[city]['cust_trips'] = customer_trips
In [11]:
# Print results
for city in city_to_filename.keys():
    total_trips = trip_counts[city]['sub_trips'] + trip_counts[city]['cust_trips']
    print( "{} - {} total trips".format(total_trips, city) )

print("\r")
for city in city_to_filename.keys():
    total_trips = trip_counts[city]['sub_trips'] + trip_counts[city]['cust_trips']
    subscriber_percent = ratio_to_percent ( trip_counts[city]['sub_trips'], total_trips )
    print( "{} - {} Subscriber %".format(subscriber_percent, city) )

print("\r")
for city in city_to_filename.keys():
    total_trips = trip_counts[city]['sub_trips'] + trip_counts[city]['cust_trips']
    customer_percent = ratio_to_percent ( trip_counts[city]['cust_trips'], total_trips )
    print( "{} - {} Customer %".format(customer_percent, city) )
66326 - Washington total trips
72131 - Chicago total trips
276798 - NYC total trips

78 - Washington Subscriber %
76 - Chicago Subscriber %
89 - NYC Subscriber %

22 - Washington Customer %
24 - Chicago Customer %
11 - NYC Customer %

Tip: In order to add additional cells to a notebook, you can use the "Insert Cell Above" and "Insert Cell Below" options from the menu bar above. There is also an icon in the toolbar for adding new cells, with additional icons for moving the cells up and down the document. By default, new cells are of the code type; you can also specify the cell type (e.g. Code or Markdown) of selected cells from the Cell menu or the dropdown in the toolbar.

Now, you will write your own code to continue investigating properties of the data.

Question 4b: Bike-share systems are designed for riders to take short trips. Most of the time, users are allowed to take trips of 30 minutes or less with no additional charges, with overage charges made for trips of longer than that duration. What is the average trip length for each city? What proportion of rides made in each city are longer than 30 minutes?

Answer:

1. The average trip lengths are 18.9, 16.6, and 15.8 minutes for Washington, Chicago, and NYC, respectively.

2. The proportion of rides that are longer than 30 minutes are 11, 8, and 7 percent for Washington, Chicago, and NYC, respectively.

In [12]:
## Use this and additional cells to answer Question 4b.                 ##
##                                                                      ##
## HINT: The csv module reads in all of the data as strings, including  ##
## numeric values. You will need a function to convert the strings      ##
## into an appropriate numeric type before you aggregate data.          ##
## TIP: For the Bay Area example, the average trip length is 14 minutes ##
## and 3.5% of trips are longer than 30 minutes.                        ##
def analyze_durations(filename, user_type=""):
    '''
    This function reads in a file with trip data and returns the average 
    duration of the trips in that file, as well as the proportion of trips
    that exceed a duration of 30 minutes.
    '''
    with open(filename, 'r') as file:
        reader = csv.DictReader(file)
        cumulative_duration = 0
        total_trip_count = 0
        long_trip_count = 0
        long_trip_threshold = 30
        
        for row in reader:
            if user_type == "":
                cumulative_duration += float(row['duration'])
                total_trip_count += 1
                if float( row['duration'] ) > long_trip_threshold:
                    long_trip_count += 1
            elif user_type == row['user_type']:
                cumulative_duration += float(row['duration'])
                total_trip_count += 1
                if float( row['duration'] ) > long_trip_threshold:
                    long_trip_count += 1
            
        average_trip_duration = cumulative_duration / total_trip_count
        proportion_long_trips = long_trip_count / total_trip_count
        
        return ( average_trip_duration, proportion_long_trips )
In [13]:
# Obtain necessary data
trip_durations = {}
for city in city_to_filename.keys():
    trip_durations[city] = {}
    average_duration, long_trip_ratio = analyze_durations( city_to_filename[city] )
    trip_durations[city]['avg_dur'] = average_duration
    trip_durations[city]['long_ratio'] = long_trip_ratio
In [14]:
# Print results
print("\r")
for city in city_to_filename.keys():
    average_duration = round(trip_durations[city]['avg_dur'],1)
    print( "{} - {} average trip duration".format(average_duration, city) )

print("\r")
for city in city_to_filename.keys():
    long_trip_ratio = ratio_to_percent(trip_durations[city]['long_ratio'])
    print( "{} - {} long trip ratio %".format(long_trip_ratio, city) )
18.9 - Washington average trip duration
16.6 - Chicago average trip duration
15.8 - NYC average trip duration

11 - Washington long trip ratio %
8 - Chicago long trip ratio %
7 - NYC long trip ratio %

Question 4c: Dig deeper into the question of trip duration based on ridership. Choose one city. Within that city, which type of user takes longer rides on average: Subscribers or Customers?

Answer:

1. The average customer trip is much longer than the average subscriber trip, by a large margin.

In [15]:
# Obtain necessary data
users = ('Customer', 'Subscriber')

trip_durations_by_user = {}
for city in city_to_filename.keys():
    trip_durations_by_user[city] = {}
    for user in users:
        trip_durations_by_user[city][user] = {}
        average_duration, long_trip_ratio = analyze_durations( city_to_filename[city], user )
        trip_durations_by_user[city][user]['avg_dur'] = average_duration
        trip_durations_by_user[city][user]['long_ratio'] = long_trip_ratio
In [16]:
## Use this and additional cells to answer Question 4c. If you have    ##
## not done so yet, consider revising some of your previous code to    ##
## make use of functions for reusability.                              ##
##                                                                     ##
## TIP: For the Bay Area example data, you should find the average     ##
## Subscriber trip duration to be 9.5 minutes and the average Customer ##
## trip duration to be 54.6 minutes. Do the other cities have this     ##
## level of difference?                                                ##

print("\nAverage Trip Durations (minutes)")

for city in city_to_filename.keys():
    average_duration_customers = round(trip_durations_by_user[city]['Customer']['avg_dur'],1)
    average_duration_subscribers = round(trip_durations_by_user[city]['Subscriber']['avg_dur'],1)
    print( "Customers: {}, Subscribers: {} - {}".format(average_duration_customers, average_duration_subscribers, city) )

print("\nLong Trip Ratio (%)")

for city in city_to_filename.keys():
    average_long_ratio_customers = ratio_to_percent(trip_durations_by_user[city]['Customer']['long_ratio'])
    average_long_ratio_subscribers = ratio_to_percent(trip_durations_by_user[city]['Subscriber']['long_ratio'])
    print( "Customers: {},   Subscribers:    {} - {}".format(average_long_ratio_customers, average_long_ratio_subscribers, city) )
Average Trip Durations (minutes)
Customers: 41.7, Subscribers: 12.5 - Washington
Customers: 31.0, Subscribers: 12.1 - Chicago
Customers: 33.0, Subscribers: 13.7 - NYC

Long Trip Ratio (%)
Customers: 39,   Subscribers:    3 - Washington
Customers: 28,   Subscribers:    2 - Chicago
Customers: 23,   Subscribers:    5 - NYC

Visualizations

The last set of values that you computed should have pulled up an interesting result. While the mean trip time for Subscribers is well under 30 minutes, the mean trip time for Customers is actually above 30 minutes! It will be interesting for us to look at how the trip times are distributed. In order to do this, a new library will be introduced here, matplotlib. Run the cell below to load the library and to generate an example plot.

In [17]:
# load library
import matplotlib.pyplot as plt

# this is a 'magic word' that allows for plots to be displayed
# inline with the notebook. If you want to know more, see:
# http://ipython.readthedocs.io/en/stable/interactive/magics.html
%matplotlib inline 

# example histogram, data taken from bay area sample
data = [ 7.65,  8.92,  7.42,  5.50, 16.17,  4.20,  8.98,  9.62, 11.48, 14.33,
        19.02, 21.53,  3.90,  7.97,  2.62,  2.67,  3.08, 14.40, 12.90,  7.83,
        25.12,  8.30,  4.93, 12.43, 10.60,  6.17, 10.88,  4.78, 15.15,  3.53,
         9.43, 13.32, 11.72,  9.85,  5.22, 15.10,  3.95,  3.17,  8.78,  1.88,
         4.55, 12.68, 12.38,  9.78,  7.63,  6.45, 17.38, 11.90, 11.52,  8.63,]
plt.hist(data)
plt.title('Distribution of Trip Durations')
plt.xlabel('Duration (m)')
plt.show()

In the above cell, we collected fifty trip times in a list, and passed this list as the first argument to the .hist() function. This function performs the computations and creates plotting objects for generating a histogram, but the plot is actually not rendered until the .show() function is executed. The .title() and .xlabel() functions provide some labeling for plot context.

You will now use these functions to create a histogram of the trip times for the city you selected in question 4c. Don't separate the Subscribers and Customers for now: just collect all of the trip times and plot them.

In [18]:
## Use this and additional cells to collect all of the trip times as a list ##
## and then use pyplot functions to generate a histogram of trip times.     ##
def plot_histogram(city, user_type=""):
    '''
    This function reads in a file with trip data and returns the average 
    duration of the trips in that file, as well as the proportion of trips
    that exceed a duration of 30 minutes.
    '''
    filename = city_to_filename[city]
    with open(filename, 'r') as file:
        reader = csv.DictReader(file)
        data = []
        for row in reader:
            if user_type == "":
                data.append( float( row['duration'] ) )
            elif user_type == row['user_type']:
                data.append( float( row['duration'] ) )
        plt.hist(data, range=(0,75), bins=15)
        if user_type == '':
            plt.title('Distribution of Trip Durations - {}'.format(city) )
        else:
            plt.title('Distribution of Trip Durations - {} {}s'.format(city, user_type) )
        plt.xlabel('Duration (minutes)')
        plt.show()
In [19]:
# Plot histograms for all cities
for city in city_to_filename.keys():
    plot_histogram(city)

If you followed the use of the .hist() and .show() functions exactly like in the example, you're probably looking at a plot that's completely unexpected. The plot consists of one extremely tall bar on the left, maybe a very short second bar, and a whole lot of empty space in the center and right. Take a look at the duration values on the x-axis. This suggests that there are some highly infrequent outliers in the data. Instead of reprocessing the data, you will use additional parameters with the .hist() function to limit the range of data that is plotted. Documentation for the function can be found [here].

Question 5: Use the parameters of the .hist() function to plot the distribution of trip times for the Subscribers in your selected city. Do the same thing for only the Customers. Add limits to the plots so that only trips of duration less than 75 minutes are plotted. As a bonus, set the plots up so that bars are in five-minute wide intervals. For each group, where is the peak of each distribution? How would you describe the shape of each distribution?

Answer:

1. For subscribers, the peak of the distribution is in the 5- to 10-minute "bucket," regardless of the city being analyzed. For customers, the peak of the distribution is in the 15- to 20-minute "bucket" for Washington only. For customers in Chicago and NYC, the peak of the distribution is in the 20- to 25-minute "bucket." For each of the cities and each user group, the distribution is unimodal, with a positive (right) skew.

2. The shape of the histogram for subscribers is unimodal with a skew to the right. For customers in each city, the mode is further to the right than for subscribers, but the shape still remains positively skewed. The tail is much fatter for customers than for subscribers, as well.

In [20]:
# Use this and additional cells to answer Question 5.
# Plot histograms for both user types in all three cities
for user in users:
    for city in city_to_filename.keys():
        plot_histogram(city, user)

Performing Your Own Analysis

So far, you've performed an initial exploration into the data available. You have compared the relative volume of trips made between three U.S. cities and the ratio of trips made by Subscribers and Customers. For one of these cities, you have investigated differences between Subscribers and Customers in terms of how long a typical trip lasts. Now it is your turn to continue the exploration in a direction that you choose. Here are a few suggestions for questions to explore:

  • How does ridership differ by month or season? Which month / season has the highest ridership? Does the ratio of Subscriber trips to Customer trips change depending on the month or season?
  • Is the pattern of ridership different on the weekends versus weekdays? On what days are Subscribers most likely to use the system? What about Customers? Does the average duration of rides change depending on the day of the week?
  • During what time of day is the system used the most? Is there a difference in usage patterns for Subscribers and Customers?

If any of the questions you posed in your answer to question 1 align with the bullet points above, this is a good opportunity to investigate one of them. As part of your investigation, you will need to create a visualization. If you want to create something other than a histogram, then you might want to consult the Pyplot documentation. In particular, if you are plotting values across a categorical variable (e.g. city, user type), a bar chart will be useful. The documentation page for .bar() includes links at the bottom of the page with examples for you to build off of for your own use.

Question 6: Continue the investigation by exploring another question that could be answered by the data available. Document the question you want to explore below. Your investigation should involve at least two variables and should compare at least two groups. You should also use at least one visualization as part of your explorations.

Is the pattern of ridership different on the weekends versus weekdays?

The histograms below aggregate trip durations for each city based on whether the rides occur on a weekday or a weekend. The histograms are qualitatively very similar for each category of trip, with 5-10 minute trips remaining the most common types of trips, and distributions remaining right-skewed.

The city with the largest difference between weekend and weekday trip length durations is Chicago, where the distribution shows a larger number of trips in the 10-25 minute range on weekends.

It is reasonable to expect that the trip durations would skew toward longer trips on the weekends, as customers utilize the bikes for more casual, longer-distance trips. Weekday usage, by contrast, could be expected to be comparatively shorter as customers take direct, time-efficient, commuting trips. The overall similarity between weekday and weekend usage is somewhat surprising. But, the tails are fatter on weekends than weekdays, as predicted by the foregoing reasoning.

In [21]:
# Define function to plot histograms
def plot_durations_by_day_type_histogram(city, day_type=""):
    '''
    Plots a histogram for a given city and day type (weekday or weekend)
    '''
    filename = city_to_filename[city]
    
    with open(filename, 'r') as file:
        reader = csv.DictReader(file)
        durations = []
        
        for row in reader:
            if day_type == 'Weekend':                
                if row['day_of_week'] == 'Saturday' or row['day_of_week'] == 'Sunday':
                    durations.append( float( row['duration'] ) )
            elif day_type == 'Weekday':
                if row['day_of_week'] == 'Monday' or row['day_of_week'] == 'Tuesday' or row['day_of_week'] == 'Wednesday' or row['day_of_week'] == 'Thursday' or row['day_of_week'] == 'Friday':
                    durations.append( float( row['duration'] ) )
            else:
                print("Must input day_type.")
        
        plt.hist(durations, range=(0,75), bins=15)
        plt.title('Durations of Trips - All {} Users on {}s'.format(city, day_type) )
        plt.xlabel('Duration (minutes)')
        plt.show()
In [22]:
# Plot histograms for both types of days in all three cities
day_types = ('Weekend', 'Weekday')

for city in city_to_filename.keys():
    for day_type in day_types:
        plot_durations_by_day_type_histogram(city, day_type)

On what days are Subscribers most likely to use the system? What about Customers?

The bar charts below aggregate the number of trips taken by Subscribers and Customers for all three cities (Washington, Chicago, and NYC) and organizes them by the day of week of the trip. As shown, subscribers are most likely to use the service during the week, particularly Tuesday through Thursday. By contrast, customers are most likely to use the service on Weekends, with Tuesday through Thursday being the least days when customers are least likely to use the service.

This reflects the typical usage for customers and subscribers. Customers are more likely to use the system for infrequent, casual, longer-duration trips. Subscribers, on the other hand, are more likely to utilize the service as an important part of their dairly routine, particularly for direct, purposeful, and shorter-duration trips.

In [23]:
# Define a function to compute number of counts and total duration for a given city, user_type, and day of week.
def trip_info_by_day(city, user_type, day):
    '''
    Returns a tuple with trip count and total duration for a given city, user type, and day of week.
    '''
    filename = city_to_filename[city]
    
    with open(filename, 'r') as file:
        reader = csv.DictReader(file)
        trip_count = 0
        trip_duration = 0.0

        for row in reader:
            if day == row['day_of_week']:
                if user_type == row['user_type']:
                    trip_count += 1
                    trip_duration += float( row['duration'] )
                
        return (trip_count, trip_duration)
In [24]:
# Compute necessary data.
cities = ['Washington', 'Chicago', 'NYC']
users = ['Customer', 'Subscriber']
days = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']

trips = {}
            
for user in users:
    trips[user] = {}
    trips[user]['count'] = {}
    trips[user]['total_duration'] = {}
    for day in days:
        trips[user]['count'][day] = {}
        trips[user]['total_duration'][day] = {}
        trips[user]['count'][day] = 0
        trips[user]['total_duration'][day] = 0.0
        for city in cities:
            count, duration = trip_info_by_day(city, user, day)
            trips[user]['count'][day] += count
            trips[user]['total_duration'][day] += duration
In [25]:
# Plot trip data.
abr_days = ['Sun', 'Mon', 'Tues', 'Wed', 'Thurs', 'Fri', 'Sat']

plt.bar(range(len(abr_days)),trips['Customer']['count'].values())
plt.xticks(range(len(days)),abr_days)
plt.ylabel('Trip Count')
plt.xlabel('Day of Week')
plt.title('Customer Trips - Combined Washington, Chicago, and NYC')
plt.show()

plt.bar(range(len(abr_days)),trips['Subscriber']['count'].values())
plt.xticks(range(len(days)),abr_days)
plt.ylabel('Trip Count')
plt.xlabel('Day of Week')
plt.title('Subscriber Trips - Combined Washington, Chicago, and NYC')
plt.show()

Does the average duration of rides change depending on the day of the week?

When each user group is examined individually, average trip duration does not vary much from day to day during the week. Customers take average trips of 32-34 minutes, regardless of the day of week. For customers, there is no discernable pattern from day to day. Subscribers exhibit clearer trends, with slightly longer duration trips (13-14 minutes) on Fridays and weekends, and shorter duration trips (12-13 minutes) from Monday to Thursday. It is interesting to note how dramatically different the average ride duration is for the different user types.

When the average ride duration is considered for all users, the distribution resembles that of the Customer trip count plot, with longer duration trips on weekends (~20 minutes) and shorter trips during the week, especially Tuesday through Thursday (~15 minutes). This is due to the ridership mix shifting from consisting primarily of shorter trips taken by subscribers during the week, to longer trips taken by customers on weekends.

In [26]:
# Compute average trip duration for subscribers, customers, and a combination of both, with the data combined for all
# three cities.
sub_avg_dur = []
cust_avg_dur = []
comb_avg_dur = []

for day in days:
    sub_dur_on_day = trips['Subscriber']['total_duration'][day]
    sub_count_on_day = trips['Subscriber']['count'][day]
    cust_dur_on_day = trips['Customer']['total_duration'][day]
    cust_count_on_day = trips['Customer']['count'][day]
    
    sub_avg_dur_on_day = sub_dur_on_day / sub_count_on_day
    cust_avg_dur_on_day = cust_dur_on_day / cust_count_on_day
    
    sub_avg_dur.append( sub_avg_dur_on_day )
    cust_avg_dur.append( cust_avg_dur_on_day )
    comb_avg_dur.append( ( sub_avg_dur_on_day * sub_count_on_day + cust_avg_dur_on_day * cust_count_on_day ) / ( sub_count_on_day + cust_count_on_day ) )
In [27]:
# Plot the data on histograms
plt.bar(range(len(abr_days)),comb_avg_dur)
plt.xticks(range(len(days)),abr_days)
plt.ylabel('Average Trip Duration (mins)')
plt.xlabel('Day of Week')
plt.title('All Users Average Trip Duration - Combined Washington, Chicago, and NYC')
plt.show()

plt.bar(range(len(abr_days)),cust_avg_dur)
plt.xticks(range(len(days)),abr_days)
plt.ylabel('Average Trip Duration')
plt.xlabel('Day of Week')
plt.title('Customer Average Trip Duration - Combined Washington, Chicago, and NYC')
plt.show()

plt.bar(range(len(abr_days)),sub_avg_dur)
plt.xticks(range(len(days)),abr_days)
plt.ylabel('Average Trip Duration (mins)')
plt.xlabel('Day of Week')
plt.title('Subscriber Average Trip Duration - Combined Washington, Chicago, and NYC')
plt.show()

Conclusions

Congratulations on completing the project! This is only a sampling of the data analysis process: from generating questions, wrangling the data, and to exploring the data. Normally, at this point in the data analysis process, you might want to draw conclusions about the data by performing a statistical test or fitting the data to a model for making predictions. There are also a lot of potential analyses that could be performed on the data which are not possible with only the data provided. For example, detailed location data has not been investigated. Where are the most commonly used docks? What are the most common routes? As another example, weather has potential to have a large impact on daily ridership. How much is ridership impacted when there is rain or snow? Are subscribers or customers affected more by changes in weather?

Question 7: Putting the bike share data aside, think of a topic or field of interest where you would like to be able to apply the techniques of data science. What would you like to be able to learn from your chosen subject?

Answer:

I recently conducted a study on the data science labor market as part of the culminating project for my MBA. The analysis I did was fairly rudimentary, but it did demonstrate fairly conclusively that the employment opportunities in data science are rapidly growing. The analysis was inhibited by limited occupation data form the Bureau of Labor Statistics, which could be expected due to the newness of the data science field.

I would like to take a deeper dive into the available data, to especially determine the geographic distribution of data science jobs. I think they would be heavily concentrated in urban centers and wherever large companies are headquartered. The reason for my interest in this subject is that I will probably need to relocate from my relatively small hometown of Bakersfield, CA to a larger city in order to pursue my Data Science career. It would be interesting to know where I would be most likely to find these jobs.

Tip: If we want to share the results of our analysis with others, we aren't limited to giving them a copy of the jupyter Notebook (.ipynb) file. We can also export the Notebook output in a form that can be opened even for those without Python installed. From the File menu in the upper left, go to the Download as submenu. You can then choose a different format that can be viewed more generally, such as HTML (.html) or PDF (.pdf). You may need additional packages or software to perform these exports.

If you are working on this project via the Project Notebook page in the classroom, you can also submit this project directly from the workspace. Before you do that, you should save an HTML copy of the completed project to the workspace by running the code cell below. If it worked correctly, the output code should be a 0, and if you click on the jupyter icon in the upper left, you should see your .html document in the workspace directory. Alternatively, you can download the .html copy of your report following the steps in the previous paragraph, then upload the report to the directory (by clicking the jupyter icon).

Either way, once you've gotten the .html report in your workspace, you can complete your submission by clicking on the "Submit Project" button to the lower-right hand side of the workspace.

In [28]:
from subprocess import call
call(['python', '-m', 'nbconvert', 'Bike_Share_Analysis.ipynb'])
Out[28]:
0