Finite word length effects

Introduction

In implementing a discrete-time system in hardware or software, it is important to consider the finite word length effects.

For example, if a filter is to be implemented on a fixed point processor, both the signal samples and the filter coefficients must be quantized to a finite number of bits. This quantization introduces round-off noise and quantized filter coefficients will change the frequency response characteristics of the filter.

In this reader we first present the most common binary representations, quantization- and overflow- methods. Then we analyze round-off noise. Finally we discuss how finite word length effects could influence filter coefficients and we show how this influences the filter characteristics and its sensitivity.

A/D conversion

First let's take a closer look where finite word length effects play a role when converting an analog, or continuous time signal $x_a(t)$ to the digital domain. For this we refer to Fig. 1.

Up to now we assumed an ideal C-to-D for this conversion, in which the resulting discrete-time samples $x[n]$ have infinite precision. These infinite precision samples are quantized with quantization step $q$ which results in a quantization error which is in the range $\pm \frac{q}{2}$. To overcome overflow, the quantization is limited up to a maximum and finally each quantized sample is encoded into $B$ bits.

In the following sections we will first discuss some binary representations, some quantization- and overflow- methods. Then we will analyse round-off noise. Finally we will discuss how finite word length effects can influence filter coefficients and we will show how this influences the filter characteristics and its sensitivity.

Screencast video [⯈]

Module overview

This module will cover the following topics:

1. Binary representation of numbers - As a start we will discuss the origin of the quantization errors, caused by the inevitable conversion of samples into bits.
2. Quantization and overflow - Here we will discuss how sample values are processed into bits, taking into account the non-linear quantization and overflow characteristics.
3. Analysis of round-off noise - The consequences of the signal quantization will be discussed in this section.
4. Quantization of filter coefficients - Filter coefficients also need to be quantized to allow for implementation in a digital systems. This quantization has an effect at the frequency response of the filter.
5. Influence filter structure - Finally, different filter structures are discussed with regards to the quantization of filter coefficients and the corresponding effects on the frequency response.

Summary

Binary representation of numbers

Fixed point representation

• Sign Magnitude (SM): $\color{red}{(-1)^{b_0}} \sum_{i=1}^{B-1} b_i 2^{B-1-i}$
• One's complement (1C): Negative = $\overline{\mbox{positive}}$
• Two's complement (2C): Negative = $\overline{\mbox{positive}} + 1$

Floating point representation

$x=M \cdot 2^{E}$ with mantissa $M$ ($\frac{1}{2} \leq |M| < 1$) and exponent $E$, positive or negative integer.

Quantization and overflow

• Three common quantization characteristics: Rounding, Value truncation and Magnitude truncation.
• Three common overflow characteristics: Saturation, Nulling and Sawtooth overflow.

Analysis of Round-off noise

$$ \frac{P_x}{P_e} = \frac{3}{16} 2^{2B} \hspace{2mm} \rightarrow \hspace{2mm} 10 \cdot ^{10}\text{log} \left ( \frac{P_x}{P_e} \right ) \hat{=} \color{red}{6 B} \text{[dB]} $$

Influence filtering of round-off noise

$$ \frac{P_x}{P_e} \neq \frac{P_v}{P_g} \hspace{3mm} \Leftrightarrow \text{ SNR$_{in}$ $\neq$ SNR$_{out}$} $$

Pairing and ordering

• Pair pole/zero couple which is closest to unit circle.
• Order according to closeness poles to unit circle. Increasing or decreasing ordering depends a.o. on shape output noise.

Quantization of filter coefficients

• Does not introduce non-linearity effects.
• Will influence the filter characteristics.

Influence filter structure

The accuracy with which the filter coefficients can be specified depends upon the word length of the processor, and the sensitivity of the filter to coefficient quantization depends on the structure of the filter, as well as on the locations of the poles and zeros.