Lab: Titanic EDA
This week was all about Pandas and plotting. At this point you should be chomping at the bit to get your hands dirty on a real-world dataset.
For this lab, we're going to take a look at the Titanic manifest. We'll be exploring this data to see what we can learn regarding the survival rates of different groups of people.
Fork and clone this repo. At the end of this lab, you'll submit a pull request using the
Titanic.ipynb notebook to answer the questions below.
Step 1: Reading the data
- Go to https://www.kaggle.com/c/titanic/data
- If you scroll down the page a bit, you'll see a data dictionary explaining each of the columns. Take a minute to familiarize yourself with how the csv is structured.
- Download the
train.csvfile into this project
- Create an iPython notebook and load the csv into pandas.
Step 2: Cleaning the data
- Create a bar chart showing how many missing values are in each column
- Which column has the most
NaNvalues? How many cells in that column are empty?
- Delete all rows where
- Fill all empty cabins with ¯\(ツ)/¯
NaN, empty, and missing are synonymous.
Step 3: Feature extraction
- There are two columns that pertain to how many family members are on the boat for a given person. Create a new column called
FamilyCountwhich will be the sum of those two columns.
- Reverends have a special title in their name. Create a column called
IsReverend: 1 if they're a preacher, 0 if they're not.
- In order to feed our training data into a classification algorithm, we need to convert our categories into 1's and 0's using
- Create 3 columns:
Embarked_S. These columns will have 1's and 0's that correspond to the
Svalues in the
- Do the same thing for
- BONUS: Extract the title from everyone's name and create dummy columns
Step 4: Exploratory analysis
- What was the survival rate overall?
- Which gender fared the worst? What was their survival rate?
- What was the survival rate for each
- Did any reverends survive? How many?
- What is the survival rate for cabins marked ¯\(ツ)/¯
- What is the survival rate for people whose
- What is the survival rate for each port of embarkation?
- What is the survival rate for children (under 12) in each
- Did the captain of the ship survive? Is he on the list?
- Of all the people that died, who had the most expensive ticket? How much did it cost?
- Does having family on the boat help or hurt your chances of survival?
Step 5: Plotting
Using Matplotlib and Seaborn, create several charts showing the survival rates of different groups of people. It's fine if a handful of charts are basic (Gender, Age, etc), but what we're really looking for is something beneath the surface.