Language Evolution and Computation Bibliography

Using computational modeling techniques, this paper explores the range of conditions under which structured, language-like communication systems can emerge. In particular, we reconsider Simon Kirby's learning bottleneck model of linguistic adaptation using a different learning mechanism and different semantic domain. We demonstrate how parameters such as population size and training corpus size affect the likelihood of a population reaching consensus on a structure communication system.

This thesis explores a perspective for explaining the origins of linguistic structure that is based on considerations beyond the constraints of the language acquisition device. In contrast to the theory of Universal Grammar proposed by Chomsky, this perspective considers how the processes of language acquisition and use create a dynamical system that is capable of adapting linguistic structure to the inductive biases of learners. In this view it is possible to conceive of language adapting to aid its own survival: those languages that are more reliably and easily acquired will tend to persist for longer than their less easily learned counterparts. Thus, linguistic structures are seen as emergent, adaptive phenomena rather than preordained features of language.

The particular issue that this thesis investigates is the extent to which language adaptation can facilitate acquisition by general-purpose learners. In the Generative Grammar tradition much is made of the necessity for domain-specific constraints on the language acquisition device. (Indeed, that there must be a distinct mental com- ponent dedicated to language tasks.) This outlook is in contrast to the connectionist viewpoint, which posits far more moderately constrained, domain-general mecha- nisms. This thesis examines how language adaptation can give general-purpose, connectionist learners the appearance of being language-savvy learners.

A simulation framework is proposed in which agents attempt to communicate simple concepts to one another using sequential utterances. In earlier simulations we aim to maximise the learnability of a language for the communication task. Later simulations show how the processes of language production and acquisition, when iterated, are capable of producing such languages. In total, three series of simulations are performed.

The first series of simulations addresses the question of how linguistic structure adapts when sender and receiver disagree on the form of language that is easiest to learn. Analysis reveals that, if necessary, the structural properties of language can take on forms that compromise between the competing constraints on sender and receiver.

The second series of simulations considers the bottleneck of linguistic transmis- sion: the requirement that learners generalise from a limited set of observed utter- ances to the entire language. Results show that generalisability can be boosted in a naive, domain-general learner by allowing language to adapt to the inductive biases present in the learner.

The third and final series of simulations investigates how the dynamical charac- teristics of linguistic change depend on the properties that drive the dynamics. That is, we explore the range of conditions under which the iterated learning dynamic is suAecient to establish a learnable language throughout the population. The results of these simulations show that the iterated learning dynamic is indeed able to act as a generator of languages that general-purpose learners are capable of acquiring.

The results from these studies suggest that through the dynamics of linguistic transmission, language can adapt to the capabilities and biases of its users. Fur- thermore, that language can exploit the inductive biases of general-purpose learning mechanisms to facilitate their own acquisition, contrary to Universal Grammar's hypothesised need for an innate, domain-specific acquisition mechanism.

Traditional theories of child language acquisition center around the existence of a language acquisition device which is specifically tuned for learning a particular class of languages. More recent proposals suggest that language acquisition is assisted by the evolution of languages towards forms that are easily learnable. In this paper, we evolve combinatorial languages which can be learned by a simple recurrent network quickly and from relatively few examples. Additionally, we evolve languages for generalization in different ``worlds'', and for generalization from specific examples. We find that languages can be evolved to facilitate different forms of impressive generalization for a minimally biased learner. The results provide empirical support for the theory that the language itself, as well as the language environment of a learner, plays a substantial role in learning: that there is far more to language acquisition than the language acquisition device.

We develop a new framework for studying the biases that recurrent neural networks bring to language processing tasks. A semantic concept represented by a point in Euclidian space is translated into a symbol sequence by an encoder network. This sequence is then fed to a decoder network which attempts to translate it back to the original concept. We show how a pair of recurrent networks acting as encoder and decoder can develop their own symbolic language that is serially transmitted between them either forwards or backwards. The encoder and decoder bring different constraints to the task, and these early results indicate that the conflicting nature of these constraints may be reflected in the language that ultimately emerges, providing important clues to the structure of human languages.