On determining the statistical parameters for pollution concentration from a truncated data set

If required, greater accuracy in the determination of the geometric mean and standard deviation can be obtained using a method based on maximum likelihood estimation theory, compared to an analytical approach that incorporates the assumption of an infinite number of noise free measurements.

Abstract

Concentrations of atmospheric pollutants do occasionally reach levels below detection and it is common to deal with pollution concentration data sets which are truncated at the detection limit of the analytical instrument employed. The problem addressed in this paper is how to correct for detection limits in determining the mean and standard deviation of pollution concentration. Two methods are described and evaluated. A comprehensive comparison is made using an analytical approach that incorporates the assumption of an infinite number of noise free measurements. Additional comparisons are made using actual pollution concentration data sets, in order to estimate the magnitude of truncation errors relative to those due to finite sample size and to inaccuracies in measurement for concentrations above the detection limit. Results show that if all data points corresponding to concentration levels below the detection limit are set equal to half the detection limit an error is introduced that would probably be acceptable for most applications. If required, greater accuracy in the determination of the geometric mean and standard deviation can be obtained using a method based on maximum likelihood estimation theory.