1. Introduction

This article deals with harmonic decomposition as a denoising technique for speech processing. In the following the idea of harmonic decomposition is described shortly, followed by the description of one approach of how to implement it in a preprocessing step of a speech recognition system.

Harmonic decomposition uses the fact that a speech signal consists mainly of harmonic frequencies and therefore adapts the weights for the harmonic and random components of the speech signal. By decomposing the speech signal into a harmonic and a random component, the two components are processed independently and then recombined to generate an enhanced signal [3].

Harmonic decomposition is based on the property of periodicity of speech signals and the classification of the speech into voiced and unvoiced states, which is described in the Speech Production article [2] [4]. By exploiting these inherent properties of the speech signal itself and not making any assumptions about the noise, a priori knowledge of the noise is not needed. Therefore harmonic decomposition is able to improve robustness of speech recognition in situations where occurring noise is stationary as well as non-stationary [1].

2. Exemplary realization

2.1 Model of Speech

A clean speech signal is considered as the summation of a harmonic signal and a random signal 

, where the harmonic signal can be modelled as a weighted sum of harmonically-related sinusoids


with being the fundamental frequency, also called the pitch, and being the total number of harmonics in the signal. The fundamental frequency of a periodic signal is defined as its lowest frequency and harmonic frequency is an integer multiple of this fundamental frequency. For every frame of the speech signal an estimate of  and the amplitude parameters is needed.

The pitch is estimated using a pitch detection algorithm (PDA) e.g. RAPT - A robust algorithm for pitch tracking’ [5].  is approximated with  and then the equation


is solved, where is a vector of samples. Since the number of samples this yields an overdetermined system and therefore an approximated solution can be found by using a least squares approach:

where this set of amplitude parameters is used to approximate the periodic component 

and by simply subtracting from you can get an estimate for the residual or random component :

2.2 Harmonic+Noise case

For the case of a noisy speech signal  the decomposition is done as follows

, where  is the noise at the harmonic component and  is the noise at the random component. An entire block diagram of the denoising approach based on harmonic decomposition is depicted below.


Figure 1: Block diagram of the harmonic decomposition for denoising of a speech signal


The windowed speech segments are decomposed using the least square approach described above into approximations of and . Now assuming that the harmonic and random component are uncorrelated and also that the clean speech and noise are uncorrelated, a transformation of the speech signal into the Mel spectrum representation is made. This transformation consists of the two first stages of the feature extraction using Mel Frequency Cepstral Coefficients (MFCC). Briefly summarized the windowed signals are transformed into the frequency domain and the computed power spectrum of each window is filtered using triangular overlapping (bandpass) filters. The outcome is described as the Mel-frequency-spectrum. This spectrum can be considered as the power spectrum of the windowed signals mapped on the mel scale [4]. It is described in greater detail in the respective article. Therefore the Mel-frequency-spectrum can be considered as a measure of energy.

Denote as the Mel-frequency-spectrum of a clean speech segment x. Due to the assumption about correlation, the following equation can be derived .

Now it is possible to derive an estimate of the clean speech component of a noise corrupted speech segment easily as a weighted summation of its harmonic and random component.


The final task is to determine and . For an highly voiced speech segment for example it is assumed that the clean speech will be captured almost entirely by the harmonic component and at the same time the random component captures mostly noise.

An estimate for can therefore be found by using the energy of the harmonic component for every frame: with being the index of the sample of a frame.

If noise corrupts the signal, the energy of the harmonic component will be smaller while the energy of the random component will be bigger.

For the weighting factor of the random component it is not that easy to find an estimate, since there is no predictable structure of the underlying structure of this component. An estimate can learned from data, empirically studying the recognition performance of an susbsequent classifier using different values for . Results from [1] and [3] show best results setting . The following figure belongs to the results of Seltzer et al. showing the improvement of the word accuracy of a MFCC based recognition system over different Signal-to-noise ratios.

Figure 2: Improvement of the word accuracy over different Signal-to-noise ratios [1]



[1] M. Seltzer, J. Droppo, A. Acero. A harmonic-model-based front end for robust speech recognition. In Proc. Eurospeech Conference, Geneva, Switzerland, September 2003. International Speech Communication Association.

[2] Q. Hu, M. Liang. On the harmonic-plus-noise decomposition of speech. In Signal Processing, 2006 8th International Conference, volume 1, 2006.

[3] U. Imtiaz. Robust Speech Recognition Using Harmonic Components. Catholic University of Leuven, Belgium, June, 2004.

[4] B. Schuller. Voice and speech analysis in search of states and traits. In A. A. Salah, T. Gevers (eds.) Computer Analysis of Human Behaviour, Advances in Pattern Recognition, chapter 9, pp. 227-253, Springer, Berlin (2011).

[5] D. Talkin. A robust algorithm for pitch tracking (RAPT). In Speech Coding and Synthesis, pp. 495-517, 1995.