Data Smoothing and Filtering

Table of Contents

  1. Chapter 1: Sensor data smoothing and filtering
    1. Part 1: Introduction to Smoothing and Filtering (notes) (slides)
    2. Part 2: Time-series Smoothing and Filtering (notes) (slides)
    3. Part 3: Frequency-domain Filtering (notes) (slides)
    4. Notebook 1: Time Domain Noise Removal (html) (ipynb)
    5. Notebook 2: Frequency Domain Noise Removal (html) (ipynb)

Chapter 1: Sensor data smoothing and filtering

Most sensor data is affected to some extent by noise, that is unexplained variations in the data that in many cases is uninterpretable, and in almost all cases is not of interest to us. Data analysis is often considerably simpler if this noise can be removed from the data. This chapter provides an overview of the sources of noise in typical sensor data and the methods by which noise can be removed.

Part 1: Introduction to Smoothing and Filtering (notes) (slides)

We start by explaining the difference between information and noise, and describe what causes noise in sensor data. We then show different examples of noisy signals and what are some common sources of noise in these signals.

Part 2: Time-series Smoothing and Filtering (notes) (slides)

Noise removal techniques can be divided into two class. The first is time-domain approaches, where the information is contained in the amplitude of a signal. For example, you may be interested in the temperature of this room, the orientation of your phone, your location and driving trajectory, and so on. All of these contain information in the time domain.

Part 3: Frequency-domain Filtering (notes) (slides)

The second is frequency-domain approaches, which removes noise that is periodic in nature. For example, accelerometer readings when you walk show periodic motion and your heart beats in a quasi-periodic manner. Many noise sources are similarly periodic and can be removed using frequency domain filtering methods.

Notebook 1: Time Domain Noise Removal (html) (ipynb)

This notebook shows a few examples of time-series signals and how different time-domain smoothing methods (moving average, exponentially weighted moving average, and median filtering) work on this data.

Notebook 2: Frequency Domain Noise Removal (html) (ipynb)

This notebook shows a few examples of time-series signals and how different frequency-domain noise removal methods (low-pass, high-pass, and notch filter) work on this data.

Table of contents