NumPy and SciPy
This reference explains the NumPy and SciPy tools used in classes 1–4. Each entry gives a purpose, the arguments most useful in this course, an example, and a practical note. Follow Official documentation for the complete API.
Examples use the course datasets: df is the eruption table, df_traces the geochemical table, reading the reading-score table, and df_icecover the lake table. Samples such as dietA and dietB have already been selected. Variables such as data, ax, and mod1 refer to the array, axes, or fitted model described in the lesson. Snippets illustrate individual operations rather than forming a script to run from top to bottom.
Import numpy as np and from scipy import stats. Numerical transformations apply element by element. Statistical tests require a defensible sampling design and appropriate assumptions.
np.arange()
Purpose: Create a sequence with a chosen step.
Main arguments: start, stop, step: range; stop is excluded.
Example:
ages = np.arange(6, 13)Remember: Produces ages 6 through 12.
np.linspace()
Purpose: Create evenly spaced values across an interval.
Main arguments: start, stop: endpoints; num: number of values.
Example:
x = np.linspace(0, 100, 200)Remember: Includes both endpoints by default.
np.percentile()
Also used as: .percentiles.
Purpose: Find thresholds below which chosen percentages of values fall.
Main arguments: a: observations; q: percentages from 0 to 100.
Example:
quartiles = np.percentile(data, [25, 50, 75])Remember: This is percentile, not percentiles. Missing values require separate handling.
np.power()
Purpose: Raise each value to a power.
Main arguments: x1: values; x2: exponent.
Example:
squared = np.power(df_traces['Zr'], 2)Remember: The operation applies element by element.
np.sqrt()
Purpose: Take square roots element by element.
Main arguments: x: values.
Example:
root = np.sqrt(df_traces['Zr'])Remember: Real-valued square roots require nonnegative values.
np.log()
Purpose: Take natural logarithms.
Main arguments: x: positive values.
Example:
logged = np.log(df_traces['Zr'])Remember: Zero and negative values are not valid for a finite real logarithm.
np.log10()
Purpose: Take base-10 logarithms.
Main arguments: x: positive values.
Example:
logged = np.log10(df_traces['Zr'])Remember: Document the transformation and its scientific meaning.
np.exp()
Purpose: Calculate the exponential of each value.
Main arguments: x: values.
Example:
original_scale = np.exp(logged)Remember: Inverts a natural logarithm, not a base-10 logarithm.
np.floor()
Purpose: Round each value down to the next integer.
Main arguments: x: values.
Example:
rounded = np.floor(df['Latitude'])Remember: Rounding down is towards negative infinity.
np.ceil()
Purpose: Round each value up to the next integer.
Main arguments: x: values.
Example:
rounded = np.ceil(df['Latitude'])Remember: Rounding up is towards positive infinity.
np.ones_like()
Purpose: Create ones with the same shape as another array.
Main arguments: a: reference array; dtype: optional output type.
Example:
mask = np.ones_like(correlation, dtype=bool)Remember: The dtype controls whether the ones are numbers or Boolean values.
np.triu()
Purpose: Keep the upper triangle of a matrix and zero the remainder.
Main arguments: m: matrix; k: diagonal offset.
Example:
mask = np.triu(np.ones_like(correlation, dtype=bool))Remember: Used to hide duplicate correlations in a heatmap.
np.random.normal()
Purpose: Simulate observations from a normal distribution.
Main arguments: loc: mean; scale: standard deviation; size: number or shape.
Example:
data = np.random.normal(loc=0, scale=1, size=1000)Remember: Simulated values change between runs unless the generator is seeded.
np.random.lognormal()
Purpose: Simulate positive observations with a log-normal distribution.
Main arguments: mean, sigma: mean and standard deviation on the log scale; size: sample size.
Example:
data = np.random.lognormal(mean=0, sigma=0.4, size=1000)Remember: Parameters describe the underlying normal distribution, not the observed mean and standard deviation.
array.mean()
Also used as: np.mean.
Purpose: Average array values.
Main arguments: axis: direction of the summary.
Example:
average = data.mean()Remember: Unlike pandas, ordinary NumPy means do not automatically ignore NaN.
array.min()
Also used as: np.min.
Purpose: Find the smallest array value.
Main arguments: axis: direction of the summary.
Example:
minimum = data.min()Remember: Returns an observed extreme.
array.max()
Also used as: np.max.
Purpose: Find the largest array value.
Main arguments: axis: direction of the summary.
Example:
maximum = data.max()Remember: Returns an observed extreme.
array.std()
Also used as: np.std.
Purpose: Compute the standard deviation of array values.
Main arguments: axis: direction; ddof: denominator adjustment.
Example:
spread = data.std(ddof=1)Remember: NumPy defaults to ddof=0; pandas defaults to ddof=1.
stats.ttest_ind()
Also used as: ttest_ind, scipy.stats.ttest_ind.
Purpose: Compare the means of two independent samples.
Main arguments: a, b: samples; equal_var=False: Welch test; alternative: two-sided, less or greater.
Example:
result = stats.ttest_ind(dietA, dietB, equal_var=False)Remember: A large p-value does not establish equal means. Independence and the suitability of the mean still matter.
stats.mannwhitneyu()
Also used as: mannwhitneyu, scipy.stats.mannwhitneyu.
Purpose: Compare the relative ordering of two independent samples.
Main arguments: x, y: samples; alternative: two-sided, less or greater.
Example:
result = stats.mannwhitneyu(dietA, dietC, alternative="less")Remember: A median-shift interpretation needs additional distributional assumptions; the test is not universally a test of medians.
stats.kruskal()
Also used as: kruskal, scipy.stats.kruskal.
Purpose: Compare ranks across two or more independent groups.
Main arguments: Pass one sample per group.
Example:
result = stats.kruskal(dietA, dietB, dietC)Remember: A significant result does not identify the differing pairs; follow-up comparisons require multiplicity control.
stats.probplot()
Also used as: probplot, scipy.stats.probplot.
Purpose: Compare ordered sample values with theoretical quantiles.
Main arguments: x: sample; dist: distribution, such as "norm"; plot: optional plotting target.
Example:
stats.probplot(mod1.resid, dist="norm", plot=ax)Remember: A Q–Q plot is a diagnostic, not proof that a distribution is normal.
stats.f_oneway()
Also used as: f_oneway, scipy.stats.f_oneway.
Purpose: Perform a one-way ANOVA comparing group means.
Main arguments: Pass the independent groups as separate samples.
Example:
result = stats.f_oneway(dietA, dietB, dietC)Remember: Classical ANOVA assumes equal population variances; a significant result does not identify which means differ.