Data structure

Let’s get our hands dirty and start coding. Create a new Jupyter notebook following this guide. You can copy fragments of the code, but make sure each code block is a different cell in you notebook. Also remember that you can add Markdown cells in between code cells, which are really useful to document your code.

The data we will use here is a csv file containing selected eruptions of the past 50 years. The first 5 rows of the data are illustrated in Table 1.

Table 1: First 5 rows of the dataset.

Name	Country	Date	VEI	Latitude	Longitude
St. Helens	USA	1980-05-18 00:00:00	5	46.1914	-122.196
Pinatubo	Philippines	1991-04-02 00:00:00	6	15.1501	120.347
El Chichón	Mexico	1982-03-28 00:00:00	5	17.3559	-93.2233
Galunggung	Indonesia	1982-04-05 00:00:00	4	-7.2567	108.077
Nevado del Ruiz	Colombia	1985-11-13 00:00:00	3	4.895	-75.322

What is the VEI?

The Volcanic Explosivity Index - or VEI - is a scale to measure the magnitude of explosive eruptions based on the volume of tephra ejected during an eruption. It is a logarithmic scale in base 10:

Table 2: VEI scale with minimum and maximum erupted volume and approximate frequency.

VEI	Min Volume (km³)	Max Volume (km³)	Approx. Frequency
0	<0.00001	0.0001	Daily
1	0.0001	0.001	Weekly
2	0.001	0.01	Yearly
3	0.01	0.1	Few per year
4	0.1	1	~10 per decade
5	1	10	~1 per decade
6	10	100	~1 per century
7	100	1000	~1 per several centuries
8	>1000	-	~1 per 10,000 years

Packages, modules and functions

Python’s capabilities can be extended by importing packages. A package groups related tools and can contain modules, which organise code into separate components. These tools include functions, which perform particular tasks.

In the following example:

Function reference: read_csv (doc).

import pandas as pd
df = pd.read_csv('data.csv')

pandas is the package we import.
pd is the short name we choose to refer to it.
read_csv() (doc) is a function provided by pandas.
'data.csv' is an argument telling the function which file to read.
df is the variable in which we store the resulting DataFrame.

Later, we will also call methods: functions associated with an object, such as df.head() (doc).

Importing the library and the data

As always, we start by importing the pandas library as pd.

import pandas as pd

We load the dataset using the pd.read_csv (doc) function into a variable called df (for DataFrame). Remember that functions can take different arguments, which are extra keywords you can pass to make the behaviour of the function more specific to your need. Here, we pass one arguments to the read_csv() (doc) function: parse_dates=['Date'] Specifies that the Date column should be treated as a date object.

Function reference: read_csv (doc); head (doc).

Listing 1: Loading data from a csv file

df = pd.read_csv('https://raw.githubusercontent.com/ELSTE-Master/Data-Science/main/Data/dummy_volcanoes.csv', parse_dates=['Date'])
df.head()

1: Load data
2: Show the first 5 rows

	Name	Country	Date	VEI	Latitude	Longitude
0	St. Helens	USA	1980-05-18	5	46.1914	-122.1956
1	Pinatubo	Philippines	1991-04-02	6	15.1501	120.3465
2	El Chichón	Mexico	1982-03-28	5	17.3559	-93.2233
3	Galunggung	Indonesia	1982-04-05	4	-7.2567	108.0771
4	Nevado del Ruiz	Colombia	1985-11-13	3	4.8950	-75.3220

Setting up the index

The output of Listing 1 shows the first 5 rows in our DataFrame. As displayed here, the first column is the index - which is currently just integer numbers. That can be acceptable in some cases, but for the sake of the exercise we will choose one column to become the index - here Name.

Listing 2 Illustrates the use of two useful functions:

.set_index() (doc): Uses a column as the DataFrame’s index
.reset_index() (doc): Removes the column’s index back to a sequential numbering as in Listing 1.

Function reference: set_index (doc); reset_index (doc); head (doc).

Listing 2: Common functions to set the index of a DataFrame

df = df.set_index('VEI')
df = df.reset_index()
df = df.set_index('Name')
df.head()

1: Set the ‘VEI’ column as an index
2: Shoots, I meant to set the ‘Name’ columns as an index
3: Here we go.

	VEI	Country	Date	Latitude	Longitude
Name
St. Helens	5	USA	1980-05-18	46.1914	-122.1956
Pinatubo	6	Philippines	1991-04-02	15.1501	120.3465
El Chichón	5	Mexico	1982-03-28	17.3559	-93.2233
Galunggung	4	Indonesia	1982-04-05	-7.2567	108.0771
Nevado del Ruiz	3	Colombia	1985-11-13	4.8950	-75.3220

Basic data exploration

Let’s now explore the structure of the dataset with the following functions:

Function	Description
`df.head()` (doc)	Prints the first 5 rows of the DataFrame
`df.tail()` (doc)	Prints the last 5 rows of the DataFrame
`df.info()` (doc)	Displays some info about the DataFrame, including the number of rows (entries) and columns. Note the `Dtype` column: this is the type variable stored in each column including strings (`object`), integer (`int64`) and float (`int64`) numbers. See that the `Date` column is indeed stored as a `datetime` variable as requested above.
`df.shape` (doc)	Returns a list containing the number of rows and columns of the DataFrame.
`df.index` (doc)	Returns a list containing the index along the rows of the DataFrame.
`df.columns` (doc)	Returns a list containing the index along the columns of the DataFrame.

Why do some expressions have parentheses?

Compare these two expressions:

Function reference: head (doc); shape (doc).

df.head()
df.shape

head() (doc) is a method: we call it with parentheses to perform an operation. We can also pass arguments, such as df.head(3) (doc) to display the first three rows.

shape (doc) is an attribute: we access it without parentheses to obtain information about the DataFrame. It returns a pair of numbers: the number of rows and the number of columns.

The same distinction applies to df.info() (doc) and df.tail() (doc)—methods—and to df.index (doc) and df.columns (doc)—attributes.

Your turn!

Try these functions on df and get familiar with the output.

Sorting data

The main function to sort data is .sort_values (doc). It is necessary to review how three arguments can alter the function’s behaviour:

by: First argument (required) is the label of index/row used to sort the data. It is possible to sort by multiple columns by passing a list of values.
axis: Specifies whether sorting rows (axis = 0 - in which case by is a column name) or sorting columns (axis = 1, in which case by is an index value). The documentation specifies axis = 0, which means that rows will be sorted if axis is not specified.
ascending: Using a bool (remember, this is a True/False behaviour), specifies if values are sorted in ascending (ascending = True, default behaviour is not specified) or descending (ascending = False) order.

Function reference: sort_values (doc).

Listing 3: Basic sorting operations

df.sort_values('VEI')
df.sort_values('Date', ascending=False)
df.sort_values('Country')
df.sort_values(['Latitude', 'Longitude'])

1: Sort volcanoes by VEI in ascending number
2: Sort volcanoes by eruption dates from recent to old
3: .sort_values also work on strings to sort alphabetically
4: Sorting using multiple columns

Question

After sorting the data in descending order by VEI and time, what are the three first volcanoes?