N2 - The known standard recursion methods of computing the full normalized associated Legendre functions do not give the necessary precision due to application of IEEE754-2008 standard, that creates a problems of underflow and overflow. The analysis of the problems of the calculation of the Legendre functions shows that the problem underflow is not dangerous by itself. The main problem that generates the gross errors in its calculations is the problem named the effect of “absolute zero”. Once appeared in a forward column recursion, “absolute zero” converts to zero all values which are multiplied by it, regardless of whether a zero result of multiplication is real or not. Three methods of calculating of the Legendre functions, that removed the effect of “absolute zero” from the calculations are discussed here. These methods are also of interest because they almost have no limit for the maximum degree of Legendre functions. It is shown that the numerical accuracy of these three methods is the same. But, the CPU calculation time of the Legendre functions with Fukushima method is minimal. Therefore, the Fukushima method is the best. Its main advantage is computational speed which is an important factor in calculation of such large amount of the Legendre functions as 2 401 336 for EGM2008
JO - Geodesy and Cartography
L1 - http://journals.pan.pl/Content/107428/PDF/292.pdf
L2 - http://journals.pan.pl/Content/107428
IS - No 2
KW - Fully Normalized Associated Legendre Functions
KW - recursion
KW - underflow problem
KW - overflow problem
KW - Fukushima method
ER -
A1 - Novikova, Elena
A1 - Dmitrenko, Alexander
PB - Commitee on Geodesy PAS
VL - vol. 65
JF - Geodesy and Cartography
T1 - Problems and methods of calculating the Legendre functions of arbitrary degree and order
UR - http://journals.pan.pl/dlibra/docmetadata?id=107428
DOI - 10.1515/geocart-2016-0017