Changing a data format

When analyzing or exploring data, the type of analysis we perform on our data is highly dependent on the data formats or data types within our dataset. Typically, numerical data requires specific analytical techniques, while categorical data requires other analytical techniques. Hence, it is important that data types are properly captured before analysis commences.

In pandas, the dtypes attribute helps us to inspect the data types within our dataset, while the astype attribute helps us to convert our dataset between various data types.

Getting ready

We will work with the Marketing Campaign data again for this recipe.

How to do it…

We will change the format of our data using the pandas library:

1. Import the pandas library:

   import pandas as pd

2. Load the .csv file into a dataframe using read_csv. Then, subset the dataframe to include only relevant columns:

   marketing_data = pd.read_csv("data/marketing_campaign.csv")

   marketing_data = marketing_data[['ID', 'Year_Birth','Marital_Status','Income']]

3. Inspect the data. Check the first few rows. Check the number of columns and rows:

       ID    Year_Birth    Marital_Status    Income    Income_changed

   0    5524    1957    Single    58138.0    58138

   1    2174    1954    Single    46344.0    46344

   2    4141    1965    Together    71613.0    71613

   3    6182    1984    Together    26646.0    26646

   4    5324    1981    Married    58293.0    58293

   marketing_data.shape

   (2240, 5)

4. Fill NAs in the Income column:

   marketing_data['Income'] = marketing_data['Income'].fillna(0)

5. Change the data type of the Income column from float to int:

   marketing_data['Income_changed'] = marketing_data['Income'].astype(int)

6. Inspect the output using the head method and dtypes attribute:

   marketing_data[['Income','Income_changed']].head()

       Income    Income_changed

   0    58138.0    58138

   1    46344.0    46344

   2    71613.0    71613

   3    26646.0    26646

   4    58293.0    58293

   marketing_data[['Income','Income_changed']].dtypes

       0

   Income    float64

   Income_changed    int32

Now we have changed the format of our dataset.

How it works...

We refer to pandas as pd in step 1. In step 2, we use read_csv to load the csv file into a pandas dataframe and call it marketing_data. We also subset the dataframe to include only four relevant columns. In step 3, we inspect the dataset using head() to see the first five rows in the dataset. Using the shape method, we get a sense of the number of rows and columns.

In step 4, we fill NaN values with zeros using the fillna method. This is an important step before we can change data types in pandas. We have provided the fillna method with the just argument, which is the value to replace NaN values with. In step 5, we change the data type of the Income column from float to int using the astype method. We supply the data type we wish to convert to as the argument for the method.

In step 6, we subset the dataframe and inspect the result.

There’s more...

When converting data types, we may encounter errors in conversion. The astype method gives us options through the errors parameter to raise or ignore errors. By default, the method raises errors; however, we can ignore errors so that the method returns the original values for each error identified.

See also

Here is a great article by PB Python that provides more details on converting data types in pandas: https://pbpython.com/pandas_dtypes.html.