Scientific Python Lecture Notes

One document to learn numerics, science, and data with Python

 Download

 PDF, 2 pages per side

 PDF, 1 page per side

 HTML and example files

 Source code (github)

Tutorials on the scientific Python ecosystem: a quick introduction to central tools and techniques. The different chapters each correspond to a 1 to 2 hours course with increasing level of expertise, from beginner to expert.

Release: 2022.1

• About the Scientific Python Lecture Notes
  • Authors
  • What’s new
  • License
  • Contributing

• 1. Getting started with Python for science
  • 1.1. Python scientific computing ecosystem
    • 1.1.1. Why Python?
      • 1.1.1.1. The scientist’s needs
      • 1.1.1.2. Python’s strengths
      • 1.1.1.3. How does Python compare to other solutions?
        • Compiled languages: C, C++, Fortran…
        • Matlab scripting language
        • Julia
        • Other scripting languages: Scilab, Octave, R, IDL, etc.
        • Python
    • 1.1.2. The scientific Python ecosystem
    • 1.1.3. Before starting: Installing a working environment
    • 1.1.4. The workflow: interactive environments and text editors
      • 1.1.4.1. Interactive work
      • 1.1.4.2. Elaboration of the work in an editor
      • 1.1.4.3. IPython and Jupyter Tips and Tricks
  • 1.2. The Python language
    • 1.2.1. First steps
    • 1.2.2. Basic types
      • 1.2.2.1. Numerical types
      • 1.2.2.2. Containers
        • Lists
        • Strings
        • Dictionaries
        • More container types
      • 1.2.2.3. Assignment operator
    • 1.2.3. Control Flow
      • 1.2.3.1. if/elif/else
      • 1.2.3.2. for/range
      • 1.2.3.3. while/break/continue
      • 1.2.3.4. Conditional Expressions
      • 1.2.3.5. Advanced iteration
        • Iterate over any sequence
        • Keeping track of enumeration number
        • Looping over a dictionary
      • 1.2.3.6. List Comprehensions
    • 1.2.4. Defining functions
      • 1.2.4.1. Function definition
      • 1.2.4.2. Return statement
      • 1.2.4.3. Parameters
      • 1.2.4.4. Passing by value
      • 1.2.4.5. Global variables
      • 1.2.4.6. Variable number of parameters
      • 1.2.4.7. Docstrings
      • 1.2.4.8. Functions are objects
      • 1.2.4.9. Methods
      • 1.2.4.10. Exercises
    • 1.2.5. Reusing code: scripts and modules
      • 1.2.5.1. Scripts
      • 1.2.5.2. Importing objects from modules
      • 1.2.5.3. Creating modules
      • 1.2.5.4. ‘__main__’ and module loading
      • 1.2.5.5. Scripts or modules? How to organize your code
        • How modules are found and imported
      • 1.2.5.6. Packages
      • 1.2.5.7. Good practices
    • 1.2.6. Input and Output
      • 1.2.6.1. Iterating over a file
        • File modes
    • 1.2.7. Standard Library
      • 1.2.7.1. os module: operating system functionality
        • Directory and file manipulation
        • os.path: path manipulations
        • Running an external command
        • Walking a directory
        • Environment variables:
      • 1.2.7.2. shutil: high-level file operations
      • 1.2.7.3. glob: Pattern matching on files
      • 1.2.7.4. sys module: system-specific information
      • 1.2.7.5. pickle: easy persistence
    • 1.2.8. Exception handling in Python
      • 1.2.8.1. Exceptions
      • 1.2.8.2. Catching exceptions
        • try/except
        • try/finally
        • Easier to ask for forgiveness than for permission
      • 1.2.8.3. Raising exceptions
    • 1.2.9. Object-oriented programming (OOP)
  • 1.3. NumPy: creating and manipulating numerical data
    • 1.3.1. The NumPy array object
      • 1.3.1.1. What are NumPy and NumPy arrays?
        • NumPy arrays
        • NumPy Reference documentation
        • Import conventions
      • 1.3.1.2. Creating arrays
        • Manual construction of arrays
        • Functions for creating arrays
      • 1.3.1.3. Basic data types
      • 1.3.1.4. Basic visualization
      • 1.3.1.5. Indexing and slicing
      • 1.3.1.6. Copies and views
      • 1.3.1.7. Fancy indexing
        • Using boolean masks
        • Indexing with an array of integers
    • 1.3.2. Numerical operations on arrays
      • 1.3.2.1. Elementwise operations
        • Basic operations
        • Other operations
      • 1.3.2.2. Basic reductions
        • Computing sums
        • Other reductions
      • 1.3.2.3. Broadcasting
      • 1.3.2.4. Array shape manipulation
        • Flattening
        • Reshaping
        • Adding a dimension
        • Dimension shuffling
        • Resizing
      • 1.3.2.5. Sorting data
      • 1.3.2.6. Summary
    • 1.3.3. More elaborate arrays
      • 1.3.3.1. More data types
        • Casting
        • Different data type sizes
      • 1.3.3.2. Structured data types
      • 1.3.3.3. maskedarray: dealing with (propagation of) missing data
    • 1.3.4. Advanced operations
      • 1.3.4.1. Polynomials
        • More polynomials (with more bases)
      • 1.3.4.2. Loading data files
        • Text files
        • Images
        • NumPy’s own format
        • Well-known (& more obscure) file formats
    • 1.3.5. Some exercises
      • 1.3.5.1. Array manipulations
      • 1.3.5.2. Picture manipulation: Framing a Face
      • 1.3.5.3. Data statistics
      • 1.3.5.4. Crude integral approximations
      • 1.3.5.5. Mandelbrot set
      • 1.3.5.6. Markov chain
    • 1.3.6. Full code examples
      • 1.3.6.1. Full code examples for the numpy chapter
  • 1.4. Matplotlib: plotting
    • 1.4.1. Introduction
      • 1.4.1.1. IPython, Jupyter, and matplotlib modes
      • 1.4.1.2. pyplot
    • 1.4.2. Simple plot
      • 1.4.2.1. Plotting with default settings
      • 1.4.2.2. Instantiating defaults
      • 1.4.2.3. Changing colors and line widths
      • 1.4.2.4. Setting limits
      • 1.4.2.5. Setting ticks
      • 1.4.2.6. Setting tick labels
      • 1.4.2.7. Moving spines
      • 1.4.2.8. Adding a legend
      • 1.4.2.9. Annotate some points
      • 1.4.2.10. Devil is in the details
    • 1.4.3. Figures, Subplots, Axes and Ticks
      • 1.4.3.1. Figures
      • 1.4.3.2. Subplots
      • 1.4.3.3. Axes
      • 1.4.3.4. Ticks
        • Tick Locators
    • 1.4.4. Other Types of Plots: examples and exercises
      • 1.4.4.1. Regular Plots
      • 1.4.4.2. Scatter Plots
      • 1.4.4.3. Bar Plots
      • 1.4.4.4. Contour Plots
      • 1.4.4.5. Imshow
      • 1.4.4.6. Pie Charts
      • 1.4.4.7. Quiver Plots
      • 1.4.4.8. Grids
      • 1.4.4.9. Multi Plots
      • 1.4.4.10. Polar Axis
      • 1.4.4.11. 3D Plots
      • 1.4.4.12. Text
    • 1.4.5. Beyond this tutorial
      • 1.4.5.1. Tutorials
      • 1.4.5.2. Matplotlib documentation
      • 1.4.5.3. Code documentation
      • 1.4.5.4. Galleries
      • 1.4.5.5. Mailing lists
    • 1.4.6. Quick references
      • 1.4.6.1. Line properties
      • 1.4.6.2. Line styles
      • 1.4.6.3. Markers
      • 1.4.6.4. Colormaps
    • 1.4.7. Full code examples
      • 1.4.7.1. Code samples for Matplotlib
      • 1.4.7.2. Code for the chapter’s exercises
      • 1.4.7.3. Example demoing choices for an option
      • 1.4.7.4. Code generating the summary figures with a title
  • 1.5. SciPy : high-level scientific computing
    • 1.5.1. File input/output: scipy.io
    • 1.5.2. Special functions: scipy.special
    • 1.5.3. Linear algebra operations: scipy.linalg
    • 1.5.4. Interpolation: scipy.interpolate
    • 1.5.5. Optimization and fit: scipy.optimize
      • 1.5.5.1. Root Finding
      • 1.5.5.2. Curve fitting
      • 1.5.5.3. Optimization
    • 1.5.6. Statistics and random numbers: scipy.stats
      • 1.5.6.1. Statistical Distributions
      • 1.5.6.2. Sample Statistics and Hypothesis Tests
    • 1.5.7. Numerical integration: scipy.integrate
      • 1.5.7.1. Quadrature
      • 1.5.7.2. Initial Value Problems
    • 1.5.8. Fast Fourier transforms: scipy.fftpack
    • 1.5.9. Signal processing: scipy.signal
    • 1.5.10. Image manipulation: scipy.ndimage
      • 1.5.10.1. Geometrical transformations on images
      • 1.5.10.2. Image filtering
      • 1.5.10.3. Mathematical morphology
      • 1.5.10.4. Connected components and measurements on images
    • 1.5.11. Summary exercises on scientific computing
      • 1.5.11.1. Maximum wind speed prediction at the Sprogø station
        • Statistical approach
        • Computing the cumulative probabilities
        • Prediction with UnivariateSpline
        • Exercise with the Gumbell distribution
      • 1.5.11.2. Non linear least squares curve fitting: application to point extraction in topographical lidar data
        • Introduction
        • Loading and visualization
        • Fitting a waveform with a simple Gaussian model
          • Model
          • Initial solution
          • Fit
        • Going further
      • 1.5.11.3. Image processing application: counting bubbles and unmolten grains
        • Statement of the problem
      • 1.5.11.4. Example of solution for the image processing exercise: unmolten grains in glass
    • 1.5.12. Full code examples for the SciPy chapter
      • 1.5.12.18. Solutions of the exercises for SciPy
  • 1.6. Getting help and finding documentation
• 2. Advanced topics
  • 2.1. Advanced Python Constructs
    • 2.1.1. Iterators, generator expressions and generators
      • 2.1.1.1. Iterators
      • 2.1.1.2. Generator expressions
      • 2.1.1.3. Generators
      • 2.1.1.4. Bidirectional communication
      • 2.1.1.5. Chaining generators
    • 2.1.2. Decorators
      • 2.1.2.1. Replacing or tweaking the original object
      • 2.1.2.2. Decorators implemented as classes and as functions
      • 2.1.2.3. Copying the docstring and other attributes of the original function
      • 2.1.2.4. Examples in the standard library
      • 2.1.2.5. Deprecation of functions
      • 2.1.2.6. A while-loop removing decorator
      • 2.1.2.7. A plugin registration system
    • 2.1.3. Context managers
      • 2.1.3.1. Catching exceptions
      • 2.1.3.2. Using generators to define context managers
  • 2.2. Advanced NumPy
    • 2.2.1. Life of ndarray
      • 2.2.1.1. It’s…
      • 2.2.1.2. Block of memory
      • 2.2.1.3. Data types
        • The descriptor
        • Example: reading .wav files
        • Casting and re-interpretation/views
          • Casting
          • Re-interpretation / viewing
      • 2.2.1.4. Indexing scheme: strides
        • Main point
          • C and Fortran order
          • Slicing with integers
        • Example: fake dimensions with strides
        • Broadcasting
        • More tricks: diagonals
        • CPU cache effects
      • 2.2.1.5. Findings in dissection
    • 2.2.2. Universal functions
      • 2.2.2.1. What they are?
        • Parts of an Ufunc
        • Making it easier
      • 2.2.2.2. Exercise: building an ufunc from scratch
      • 2.2.2.3. Solution: building an ufunc from scratch
      • 2.2.2.4. Generalized ufuncs
    • 2.2.3. Interoperability features
      • 2.2.3.1. Sharing multidimensional, typed data
      • 2.2.3.2. The old buffer protocol
      • 2.2.3.3. The old buffer protocol
      • 2.2.3.4. Array interface protocol
    • 2.2.4. Array siblings: chararray, maskedarray
      • 2.2.4.1. chararray: vectorized string operations
      • 2.2.4.2. masked_array missing data
        • The mask
        • Domain-aware functions
      • 2.2.4.3. recarray: purely convenience
    • 2.2.5. Summary
    • 2.2.6. Contributing to NumPy/SciPy
      • 2.2.6.1. Why
      • 2.2.6.2. Reporting bugs
        • Good bug report
      • 2.2.6.3. Contributing to documentation
      • 2.2.6.4. Contributing features
      • 2.2.6.5. How to help, in general
  • 2.3. Debugging code
    • 2.3.1. Avoiding bugs
      • 2.3.1.1. Coding best practices to avoid getting in trouble
      • 2.3.1.2. pyflakes: fast static analysis
        • Running pyflakes on the current edited file
        • A type-as-go spell-checker like integration
    • 2.3.2. Debugging workflow
    • 2.3.3. Using the Python debugger
      • 2.3.3.1. Invoking the debugger
        • Postmortem
        • Step-by-step execution
        • Other ways of starting a debugger
      • 2.3.3.2. Debugger commands and interaction
        • Getting help when in the debugger
    • 2.3.4. Debugging segmentation faults using gdb
  • 2.4. Optimizing code
    • 2.4.1. Optimization workflow
    • 2.4.2. Profiling Python code
      • 2.4.2.1. Timeit
      • 2.4.2.2. Profiler
      • 2.4.2.3. Line-profiler
    • 2.4.3. Making code go faster
      • 2.4.3.1. Algorithmic optimization
        • Example of the SVD
    • 2.4.4. Writing faster numerical code
      • 2.4.4.1. Additional Links
  • 2.5. Sparse Matrices in SciPy
    • 2.5.1. Introduction
      • 2.5.1.1. Why Sparse Matrices?
      • 2.5.1.2. Sparse Matrices vs. Sparse Matrix Storage Schemes
      • 2.5.1.3. Typical Applications
      • 2.5.1.4. Prerequisites
      • 2.5.1.5. Sparsity Structure Visualization
    • 2.5.2. Storage Schemes
      • 2.5.2.1. Common Methods
      • 2.5.2.2. Sparse Matrix Classes
        • Diagonal Format (DIA)
          • Examples
        • List of Lists Format (LIL)
          • Examples
        • Dictionary of Keys Format (DOK)
          • Examples
        • Coordinate Format (COO)
          • Examples
        • Compressed Sparse Row Format (CSR)
          • Examples
        • Compressed Sparse Column Format (CSC)
          • Examples
        • Block Compressed Row Format (BSR)
          • Examples
      • 2.5.2.3. Summary
    • 2.5.3. Linear System Solvers
      • 2.5.3.1. Sparse Direct Solvers
        • Examples
      • 2.5.3.2. Iterative Solvers
        • Common Parameters
        • LinearOperator Class
        • A Few Notes on Preconditioning
      • 2.5.3.3. Eigenvalue Problem Solvers
        • The eigen module
    • 2.5.4. Other Interesting Packages
  • 2.6. Image manipulation and processing using NumPy and SciPy
    • 2.6.1. Opening and writing to image files
    • 2.6.2. Displaying images
    • 2.6.3. Basic manipulations
      • 2.6.3.1. Statistical information
      • 2.6.3.2. Geometrical transformations
    • 2.6.4. Image filtering
      • 2.6.4.1. Blurring/smoothing
      • 2.6.4.2. Sharpening
      • 2.6.4.3. Denoising
      • 2.6.4.4. Mathematical morphology
    • 2.6.5. Feature extraction
      • 2.6.5.1. Edge detection
      • 2.6.5.2. Segmentation
    • 2.6.6. Measuring objects properties: scipy.ndimage.measurements
    • 2.6.7. Full code examples
    • 2.6.8. Examples for the image processing chapter
  • 2.7. Mathematical optimization: finding minima of functions
    • 2.7.1. Knowing your problem
      • 2.7.1.1. Convex versus non-convex optimization
      • 2.7.1.2. Smooth and non-smooth problems
      • 2.7.1.3. Noisy versus exact cost functions
      • 2.7.1.4. Constraints
    • 2.7.2. A review of the different optimizers
      • 2.7.2.1. Getting started: 1D optimization
      • 2.7.2.2. Gradient based methods
        • Some intuitions about gradient descent
        • Conjugate gradient descent
      • 2.7.2.3. Newton and quasi-newton methods
        • Newton methods: using the Hessian (2nd differential)
        • Quasi-Newton methods: approximating the Hessian on the fly
    • 2.7.3. Full code examples
    • 2.7.4. Examples for the mathematical optimization chapter
      • 2.7.4.12. Gradient-less methods
        • A shooting method: the Powell algorithm
        • Simplex method: the Nelder-Mead
      • 2.7.4.13. Global optimizers
        • Brute force: a grid search
    • 2.7.5. Practical guide to optimization with SciPy
      • 2.7.5.1. Choosing a method
      • 2.7.5.2. Making your optimizer faster
      • 2.7.5.3. Computing gradients
      • 2.7.5.4. Synthetic exercices
    • 2.7.6. Special case: non-linear least-squares
      • 2.7.6.1. Minimizing the norm of a vector function
      • 2.7.6.2. Curve fitting
    • 2.7.7. Optimization with constraints
      • 2.7.7.1. Box bounds
      • 2.7.7.2. General constraints
    • 2.7.8. Full code examples
    • 2.7.9. Examples for the mathematical optimization chapter
  • 2.8. Interfacing with C
    • 2.8.1. Introduction
    • 2.8.2. Python-C-Api
      • 2.8.2.1. Example
      • 2.8.2.2. NumPy Support
    • 2.8.3. Ctypes
      • 2.8.3.1. Example
      • 2.8.3.2. NumPy Support
    • 2.8.4. SWIG
      • 2.8.4.1. Example
      • 2.8.4.2. NumPy Support
    • 2.8.5. Cython
      • 2.8.5.1. Example
      • 2.8.5.2. NumPy Support
    • 2.8.6. Summary
    • 2.8.7. Further Reading and References
    • 2.8.8. Exercises
      • 2.8.8.1. Python-C-API
      • 2.8.8.2. Ctypes
      • 2.8.8.3. SWIG
      • 2.8.8.4. Cython
• 3. Packages and applications
  • 3.1. Statistics in Python
    • 3.1.1. Data representation and interaction
      • 3.1.1.1. Data as a table
      • 3.1.1.2. The pandas data-frame
        • Creating dataframes: reading data files or converting arrays
        • Manipulating data
        • Plotting data
    • 3.1.2. Hypothesis testing: comparing two groups
      • 3.1.2.1. Student’s t-test: the simplest statistical test
        • 1-sample t-test: testing the value of a population mean
        • 2-sample t-test: testing for difference across populations
      • 3.1.2.2. Paired tests: repeated measurements on the same individuals
    • 3.1.3. Linear models, multiple factors, and analysis of variance
      • 3.1.3.1. “formulas” to specify statistical models in Python
        • A simple linear regression
        • Categorical variables: comparing groups or multiple categories
      • 3.1.3.2. Multiple Regression: including multiple factors
      • 3.1.3.3. Post-hoc hypothesis testing: analysis of variance (ANOVA)
    • 3.1.4. More visualization: seaborn for statistical exploration
      • 3.1.4.1. Pairplot: scatter matrices
      • 3.1.4.2. lmplot: plotting a univariate regression
    • 3.1.5. Testing for interactions
    • 3.1.6. Full code for the figures
    • 3.1.7. Solutions to this chapter’s exercises
  • 3.2. Sympy : Symbolic Mathematics in Python
    • 3.2.1. First Steps with SymPy
      • 3.2.1.1. Using SymPy as a calculator
      • 3.2.1.2. Symbols
    • 3.2.2. Algebraic manipulations
      • 3.2.2.1. Expand
      • 3.2.2.2. Simplify
    • 3.2.3. Calculus
      • 3.2.3.1. Limits
      • 3.2.3.2. Differentiation
      • 3.2.3.3. Series expansion
      • 3.2.3.4. Integration
    • 3.2.4. Equation solving
    • 3.2.5. Linear Algebra
      • 3.2.5.1. Matrices
      • 3.2.5.2. Differential Equations
  • 3.3. Scikit-image: image processing
    • 3.3.1. Introduction and concepts
      • 3.3.1.1. scikit-image and the SciPy ecosystem
      • 3.3.1.2. What’s to be found in scikit-image
    • 3.3.2. Input/output, data types and colorspaces
      • 3.3.2.1. Data types
      • 3.3.2.2. Colorspaces
    • 3.3.3. Image preprocessing / enhancement
      • 3.3.3.1. Local filters
      • 3.3.3.2. Non-local filters
      • 3.3.3.3. Mathematical morphology
    • 3.3.4. Image segmentation
      • 3.3.4.1. Binary segmentation: foreground + background
        • Histogram-based method: Otsu thresholding
        • Labeling connected components of a discrete image
      • 3.3.4.2. Marker based methods
        • Watershed segmentation
        • Random walker segmentation
    • 3.3.5. Measuring regions’ properties
    • 3.3.6. Data visualization and interaction
    • 3.3.7. Feature extraction for computer vision
    • 3.3.8. Full code examples
    • 3.3.9. Examples for the scikit-image chapter
  • 3.4. scikit-learn: machine learning in Python
    • 3.4.1. Introduction: problem settings
      • 3.4.1.1. What is machine learning?
      • 3.4.1.2. Data in scikit-learn
        • The data matrix
        • A Simple Example: the Iris Dataset
          • The application problem
          • Loading the Iris Data with Scikit-learn
    • 3.4.2. Basic principles of machine learning with scikit-learn
      • 3.4.2.1. Introducing the scikit-learn estimator object
        • Fitting on data
      • 3.4.2.2. Supervised Learning: Classification and regression
      • 3.4.2.3. A recap on Scikit-learn’s estimator interface
      • 3.4.2.4. Regularization: what it is and why it is necessary
        • Prefering simpler models
        • Simple versus complex models for classification
    • 3.4.3. Supervised Learning: Classification of Handwritten Digits
      • 3.4.3.1. The nature of the data
      • 3.4.3.2. Visualizing the Data on its principal components
      • 3.4.3.3. Gaussian Naive Bayes Classification
      • 3.4.3.4. Quantitative Measurement of Performance
    • 3.4.4. Supervised Learning: Regression of Housing Data
      • 3.4.4.1. A quick look at the data
      • 3.4.4.2. Predicting Home Prices: a Simple Linear Regression
    • 3.4.5. Measuring prediction performance
      • 3.4.5.1. A quick test on the K-neighbors classifier
      • 3.4.5.2. A correct approach: Using a validation set
      • 3.4.5.3. Model Selection via Validation
      • 3.4.5.4. Cross-validation
      • 3.4.5.5. Hyperparameter optimization with cross-validation
        • Basic Hyperparameter Optimization
        • Automatically Performing Grid Search
        • Built-in Hyperparameter Search
        • Nested cross-validation
    • 3.4.6. Unsupervised Learning: Dimensionality Reduction and Visualization
      • 3.4.6.1. Dimensionality Reduction: PCA
      • 3.4.6.2. Visualization with a non-linear embedding: tSNE
    • 3.4.7. The eigenfaces example: chaining PCA and SVMs
    • 3.4.8. The eigenfaces example: chaining PCA and SVMs
      • 3.4.8.1. Preprocessing: Principal Component Analysis
      • 3.4.8.2. Doing the Learning: Support Vector Machines
      • 3.4.8.3. Pipelining
    • 3.4.9. Parameter selection, Validation, and Testing
      • 3.4.9.1. Hyperparameters, Over-fitting, and Under-fitting
        • Bias-variance trade-off: illustration on a simple regression problem
      • 3.4.9.2. Visualizing the Bias/Variance Tradeoff
        • Validation Curves
        • Learning Curves
      • 3.4.9.3. Summary on model selection
        • High Bias
        • High Variance
      • 3.4.9.4. A last word of caution: separate validation and test set
    • 3.4.10. Examples for the scikit-learn chapter

ScientificPythonLectures.pdf ScientificPythonLectures-simple.pdf