The Least Mean Square (LMS) algorithm and its variants are currently the most frequently used adaptation algorithms; therefore, it is desirable to understand them thoroughly from both theoretical and practical points of view. One of the main aspects studied in the literature is the influence of the step size on stability or convergence of LMS-based algorithms. Different publications provide different stability upper bounds, but a lower bound is always set to zero. However, they are mostly based on statistical analysis. In this paper we show, by means of control theoretic analysis confirmed by simulations, that for the leaky LMS algorithm, a small negative step size is allowed. Moreover, the control theoretic approach alows to minimize the number of assumptions necessary to prove the new condition. Thus, although a positive step size is fully justified for practical applications since it reduces the mean-square error, knowledge about an allowed small negative step size is important from a cognitive point of view.
For many adaptive noise control systems the Filtered-Reference LMS, known as the FXLMS algorithm is used to update parameters of the control filter. Appropriate adjustment of the step size is then important to guarantee convergence of the algorithm, obtain small excess mean square error, and react with required rate to variation of plant properties or noise nonstationarity. There are several recipes presented in the literature, theoretically derived or of heuristic origin. This paper focuses on a modification of the FXLMS algorithm, were convergence is guaranteed by changing sign of the algorithm steps size, instead of using a model of the secondary path. A TakagiSugeno-Kang fuzzy inference system is proposed to evaluate both the sign and the magnitude of the step size. Simulation experiments are presented to validate the algorithm and compare it to the classical FXLMS algorithm in terms of convergence and noise reduction.