This book is a comprehensive introduction to the statistical analysis of wordfrequency distributions intended for computational linguists corpuslinguists psycho... » 

This book is a comprehensive introduction to the statistical analysis of wordfrequency distributions intended for computational linguists corpuslinguists psycholinguists and researchers in the field of quantitativestylistics. Word frequency distributions are characterized by very largenumbers of rare words. This property leads to strange phenomena such as meanfrequencies that systematically change as the number of observations isincreased relative frequencies that even in large samples are not fullyreliable estimators of population probabilities and model parameters that varywith text or corpus size. Special statistical techniques for the analysis ofdistributions with large numbers of rare events can be found in varioustechnical journals. The aim of this book is to make these techniques moreaccessible for nonspecialists both theoretically by means of a carefulintroduction to the underlying probabilistic and statistical concepts andpractically by providing a program library implementing the main models forword frequency distributions. «