noise filter.pdfVIP

The Use of Principal Components Analysis (PCA) in Processing AIRS Data Mitchell D. Goldberg Walter Wolf, Lihang Zhou NOAA/NESDIS/Office of Research and Applications 05/22/2003 Applications of Eigenvectors to AIRS Data ? NESDIS is processing and distributing AIRS data and products in near-real time to NWP centers. ? Data size is very large compared with current sounders (1 orbit ~ 2GB vs. 8 MB) ? Information is not independent. Eigenvectors provide an effective way to extract independent pieces of information. ? Eigenvector expansion coefficients (principal component scores) can be provided instead of the individual channels. ? Individual channels can be reconstructed with minimal signal loss. NWP Users ? NCEP ? ECMWF ? Met. Office ? Meteo-France ? Goddard DAO ? Meteor. Service of Canada ? Bureau of Meteorology Research Centre (Australia) ? FNMOC Principal Component Analysis ? Principal component analysis (PCA) is often used to reduce data vectors with many components to a different set of data vectors with much fewer components that still retains most of the variability and information of the original data ? R = r1.i1 + r2.i2 + r3.i3 + …. + rn.in where i1 = (1,0,0,0,0….0n) ; i2 = (0,1,0,0,0….0n) ? Data are rotated onto a new set of axes, such that the first few axes have the most explained variance. ? R = p1.E1 + p2.E2 + p3.E3 + …. + pn.En where E are eigenvectors and p1 = R E1 ? So instead of R vectors of length n, we can have a truncated P vectors of length m, where m n Eigenvectors are used for ? Data compression ? Quality control ? Retrievals of geophysical parameters ? Atmospheric temperature, moisture and ozone ? Surface temperature and emissivity Generating AIRS eigenvectors ? Each AIRS data vector has 1524 radiance values (Efforts are being made to expand it to over 2000). ? The radiances are normalized by expected instrumental noise (signal to noise) ? Compute the covariance matrix S ? Compute the eigenvectors E and