Language Evolution and Computation Bibliography

The lexicon is the central locus of association between form and meaning. The prior sections in this chapter focus on the lexicon as it figures in the cognitive systems of individuals. The lexicon can also be viewed at the level of language communities, as shared intellectual property that supports mech- anisms of information transmission amongst individuals. This viewpoint is foreshadowed by Hawkins (this volume), and sketched for linguistic systems in general in Hruschka et al. (2009). Here, I consider the relationship between the lexical systems of individuals and lexical systems at the community level. The dynamics of these systems over time, rooted in their relationship to each other, can inform our understanding of the lexicon, and of the entries and relationships that comprise it. Tackling problems in lexical dynamics, in the light of experimental findings and synchronic statistics, provides laboratory phonology both with fresh lines of empirical evidence and with fresh arenas for theoretical prediction.

Patterns of word use both reflect and influence a myriad of human activities and interactions. Like other entities that are reproduced and evolve, words rise or decline depending upon a complex interplay between their intrinsic properties and the environments in which they function. Using Internet discussion communities as model systems, we define the concept of a word niche as the relationship between the word and the characteristic features of the environments in which it is used. We develop a method to quantify two important aspects of the size of the word niche: the range of individuals using the word and the range of topics it is used to discuss. Controlling for word frequency, we show that these aspects of the word niche are strong determinants of changes in word frequency. Previous studies have already indicated that word frequency itself is a correlate of word success at historical time scales. Our analysis of changes in word frequencies over time reveals that the relative sizes of word niches are far more important than word frequencies in the dynamics of the entire vocabulary at shorter time scales, as the language adapts to new concepts and social groupings. We also distinguish endogenous versus exogenous factors as additional contributors to the fates of words, and demonstrate the force of this distinction in the rise of novel words. Our results indicate that short-term nonstationarity in word statistics is strongly driven by individual proclivities, including inclinations to provide novel information and to project a distinctive social identity.

This paper reconsiders the diphone-based word segmentation model of Cairns, Shillcock, Chater, and Levy (1997) and Hockema (2006), previously thought to be unlearnable. A statistically principled learning model is developed using Bayes theorem and reasonable assumptions about infants implicit knowledge. The ability to recover phrase-medial word boundaries is tested using phonetic corpora derived from spontaneous interactions with children and adults. The (unsupervised and semi-supervised) learning models are shown to exhibit several crucial properties. First, only a small amount of language exposure is required to achieve the model's ceiling performance, equivalent to between 1day and 1month of caregiver input. Second, the models are robust to variation, both in the free parameter and the input representation. Finally, both the learning and baseline models exhibit undersegmentation, argued to have significant ramifications for speech processing as a whole.

Zipf's discovery that word frequency distributions obey a power law established parallels between biological and physical processes, and language, laying the groundwork for a complex systems perspective on human communication. More recent research has also identified scaling regularities in the dynamics underlying the successive occurrences of events, suggesting the possibility of similar findings for language as well.

By considering frequent words in USENET discussion groups and in disparate databases where the language has different levels of formality, here we show that the distributions of distances between successive occurrences of the same word display bursty deviations from a Poisson process and are well characterized by a stretched exponential (Weibull) scaling. The extent of this deviation depends strongly on semantic type - a measure of the logicality of each word - and less strongly on frequency. We develop a generative model of this behavior that fully determines the dynamics of word usage.

Recurrence patterns of words are well described by a stretched exponential distribution of recurrence times, an empirical scaling that cannot be anticipated from Zipf's law. Because the use of words provides a uniquely precise and powerful lens on human thought and activity, our findings also have implications for other overt manifestations of collective human dynamics.

Studies of language change have begun to contribute to answering several pressing questions in cognitive sciences, including the origins of human language capacity, the social construction of cognition and the mechanisms underlying culture change in general. Here, we describe recent advances within a new emerging framework for the study of language change, one that models such change as an evolutionary process among competing linguistic variants. We argue that a crucial and unifying element of this framework is the use of probabilistic, data-driven models both to infer change and to compare competing claims about social and cognitive influences on language change.

A number of Russian verbs lack 1sg non-past forms. These paradigmatic gaps are puzzling because they seemingly contradict the highly productive nature of inflectional systems. We model the persistence and spread of Russian gaps via a multi-agent model with Bayesian learning. We ran three simulations: no grammar learning, learning with arbitrary analogical pressure, and morphophonologically conditioned learning. We compare the results to the attested historical development of the gaps. Contradicting previous accounts, we propose that the persistence of gaps can be explained in the absence of synchronic competition between forms.

Over the last decades, a considerable body of evidence has accumulated that speakers have detailed phonetic knowledge of a type which is not readily modelled using the categories and categorical rules of phonological theory. One line of evidence is systematic di-erences between languages in -ne details of pronunciation. For example, it is known that Spanish and English di-er systematically in the exact formant patterns typical of their point vowels (Bradlow 1995). Canadian French di-ers from both Canadian English and European French in the distribution of VOT times of voiced and voiceless stops (Caramazza and Yeni-Komshian, 1974). These are just two of many examples, with more reviewed in Pierrehumbert (in press) and Pierrehumbert et al. (in press); at this point, it is not possible to point to a single case in which analogous phonemes in two di-erent languages display exactly the same phonetic targets and the same pattern of phonetic variation in di-erent contexts. Exact phonetic targets and patterns of variation must accordingly be learned during the course of language acquisition. The usage-based framework readily accomodates such ndings by proposing that mental representations of phonological targets and patterns are gradually built up through experience with speech.

[From SWAP.] The most common word length in the lexicon lies in the middle of the total range. The shortest words -- light V or CV monosyllables -- are necessarily few because a cross-product of the consonantal and vocalic phonemes generates only a small number of combinations. As length increases, the number of possible forms explodes, but fewer and fewer are actually used in any given language. Experimental and computational studies relate the sparsity of long forms to the fact that the likelihood of forms as determined by a stochastic parse decreases with length. This effect occurs because long forms have more subparts than short ones, and the likelihood of any given subpart is always less than 1.0. (Coleman and Pierrehumbert 1997, Frisch et al in press). The disadvantage that long forms have in achieving a high well-formedness score is a distinct phenomenon from the tendency of individual long words to have low token frequencies (though there may of course be some deep relationship between these characteristcs). In English, a morphologically impoverished language, the most common type of word is the disyllable.

[Notes.] This is a Monte Carlo simulation on the learnability of phonological constraints, based on the idea that the phonological grammar needs to be shared even if individuals have different words in their vocabularies.