Using PCA on Cars Dataset

import pandas as pd
import numpy as np
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

import matplotlib.pyplot as plt

The dataset for this example consists of 387 cars with 18 features. 7 of these features are dummy variables.

c = pd.read_csv('pca-on-cars-dataset/cars.csv')
c.info()
c.head().T

<class 'pandas.core.frame.DataFrame'>
Index: 387 entries, Acura 3.5 RL to Volvo XC90 T6
Data columns (total 18 columns):
 #   Column      Non-Null Count  Dtype  
---  ------      --------------  -----  
 0   Sports      387 non-null    int64  
 1   SUV         387 non-null    int64  
 2   Wagon       387 non-null    int64  
 3   Minivan     387 non-null    int64  
 4   Pickup      387 non-null    int64  
 5   AWD         387 non-null    int64  
 6   RWD         387 non-null    int64  
 7   Retail      387 non-null    int64  
 8   Dealer      387 non-null    int64  
 9   Engine      387 non-null    float64
 10  Cylinders   387 non-null    int64  
 11  Horsepower  387 non-null    int64  
 12  CityMPG     387 non-null    int64  
 13  HighwayMPG  387 non-null    int64  
 14  Weight      387 non-null    int64  
 15  Wheelbase   387 non-null    int64  
 16  Length      387 non-null    int64  
 17  Width       387 non-null    int64  
dtypes: float64(1), int64(17)
memory usage: 57.4+ KB

Helper function that performs the scaling and actual PCA.

def do_pca(n_components, data):
    X = StandardScaler().fit_transform(data)
    pca = PCA(n_components)
    X_pca = pca.fit_transform(X)
    
    return pca, X_pca

Helper function that:

• Creates a DataFrame of the PCA results
• Includes feature weights and explained variance
• Visualizes the PCA results

def pca_results(full_dataset, pca, plot=True):
    '''
    Create a DataFrame of the PCA results
    Includes dimension feature weights and explained variance
    Visualizes the PCA results
    '''

    # Dimension indexing
    dimensions = dimensions = \
    ['Dimension {}'.format(i) for i in range(1,len(pca.components_)+1)]

    # PCA components
    components = pd.DataFrame(np.round(pca.components_, 4), columns = full_dataset.keys())
    components.index = dimensions

    # PCA explained variance
    ratios = pca.explained_variance_ratio_.reshape(len(pca.components_), 1)
    variance_ratios = pd.DataFrame(np.round(ratios, 4), columns = ['Explained Variance'])
    variance_ratios.index = dimensions

    if plot:
        # Create a bar plot visualization
        fig, ax = plt.subplots(figsize = (14,8))

        # Plot the feature weights as a function of the components
        components.plot(ax = ax, kind = 'bar');
        ax.set_ylabel("Feature Weights")
        ax.set_xticklabels(dimensions, rotation=0)

        # Display the explained variance ratios
        for i, ev in enumerate(pca.explained_variance_ratio_):
            ax.text(i-0.40, ax.get_ylim()[1] + 0.05, 
                    "Explained Variance\n          %.4f"%(ev))

    # Return a concatenated DataFrame
    return pd.concat([variance_ratios, components], axis = 1)

For context, as outlined in my other notes on PCA:

• Principal components span the directions of maximum variability.
• Principal components are always orthogonal to one another.
• Eigenvalues tell us the amount of information a principal component holds.

First, consider the first 3 principal components:

pca, X_pca = do_pca(3, c)
pca_results(c, pca).T

print('Number of Components :   Variance Explained')
for component_count in range(1,17):
    pca, X_pca = do_pca(component_count, c)
    results = pca_results(c, pca, False)
    print('{:20} :   {:2.1%}'.format(component_count,
                                     results['Explained Variance'].sum()))

Number of Components :   Variance Explained
                   1 :   43.5%
                   2 :   60.2%
                   3 :   70.5%
                   4 :   76.8%
                   5 :   82.4%
                   6 :   86.8%
                   7 :   89.7%
                   8 :   92.5%
                   9 :   95.0%
                  10 :   96.8%
                  11 :   97.7%
                  12 :   98.5%
                  13 :   99.0%
                  14 :   99.5%
                  15 :   99.8%
                  16 :   100.0%

Note that the first two components, alone, account for 60% of the variance in the dataset.