Language Evolution and Computation Bibliography

Language has evolved over centuries and was gradually enriched and improved. The question, how people find assignment between meanings and referents, remains unanswered. There are many of computational models based on the statistical co-occurrence of meaning-reference pairs. Unfortunately, these mapping strategies show poor performance in an environment with a higher number of objects or noise. Therefore, we propose a more robust noise-resistant algorithm. We tested the performance of this novel algorithm with simulated and physical iCub robots. We developed a testing scenario consisting of objects with varying visual properties presented to the robot accompanied by utterances describing the given object. The results suggest that the proposed mapping procedure is robust, resistant against noise and shows better performance than one-step mapping for all levels of noise in the linguistic input, as well as slower performance degradation with increasing noise. Furthermore, the proposed procedure increases the clustering accuracy of both modalities.

Recent advances in behavioural and computational neuroscience, cognitive robotics, and in the hardware implementation of large-scale neural networks, provide the opportunity for an accelerated understanding of brain functions and for the design of interactive robotic systems based on brain-inspired control systems. This is especially the case in the domain of action and language learning, given the significant scientific and technological developments in this field. In this work we describe how a neuroanatomically grounded spiking neural network for visual attention has been extended with a word learning capability and integrated with the iCub humanoid robot to demonstrate attention-led object naming. Experiments were carried out with both a simulated and a real iCub robot platform with successful results. The iCub robot is capable of associating a label to an object with a ‘preferred’ orientation when visual and word stimuli are presented concurrently in the scene, as well as attending to said object, thus naming it. After learning is complete, the name of the object can be recalled successfully when only the visual input is present, even when the object has been moved from its original position or when other objects are present as distractors.

Recurrent neural networks have recently shown significant potential in different language applications, ranging from natural language processing to language modelling. This paper introduces a research effort to use such networks to develop and evaluate natural language acquisition on a humanoid robot. Here, the problem is twofold. First, the focus will be put on using the gesture-word combination stage observed in infants to transition from single to multi-word utterances. Secondly, research will be carried out in the domain of connecting action learning with language learning. In the former, the long-short term memory architecture will be implemented, whilst in the latter multiple time-scale recurrent neural networks will be used. This will allow for comparison between the two architectures, whilst highlighting the strengths and shortcomings of both with respect to the language learning problem. Here, the main research efforts, challenges and expected outcomes are described.

Most theories of learning would predict a gradual acquisition and refinement of skills as learning progresses, and while some highlight exponential growth, this fails to explain why natural cognitive development typically progresses in stages. Models that do span multiple developmental stages typically have parameters to "switch" between stages. We argue that by taking an embodied view, the interaction between learning mechanisms, the resulting behavior of the agent, and the opportunities for learning that the environment provides can account for the stage-wise development of cognitive abilities. We summarize work relevant to this hypothesis and suggest two simple mechanisms that account for some developmental transitions: neural readiness focuses on changes in the neural substrate resulting from ongoing learning, and perceptual readiness focuses on the perceptual requirements for learning new tasks. Previous work has demonstrated these mechanisms in replications of a wide variety of infant language experiments, spanning multiple developmental stages. Here we piece this work together as a single model of ongoing learning with no parameter changes at all. The model, an instance of the Epigenetic Robotics Architecture (Morse et al 2010) embodied on the iCub humanoid robot, exhibits ongoing multi-stage development while learning pre-linguistic and then basic language skills.

The digital age is changing our children’s lives and childhood dramatically. New technologies transform the way people interact with each other, the way stories are shared and distributed, and the way reality is presented and perceived. Parents experience that toddlers can handle tablets and apps with a level of sophistication the children’s grandparents can only envy. In Great Britain, a recent survey of preschoolers shows that a rising number of toddlers are now put to bed with a tablet instead of a bedtime story. In the USA, a telephone survey of 1,009 parents of children aged 2–24 months (Zimmerman et al., 2007a) documents that by 3 months of age, about 40% of children regularly watched television, DVDs or videos, while by 24 months the proportion rose to 90%. Moreover, with the advance and exponential use of social media, children see their parents constantly interacting with mobile devices, instead of with people around them. Still, research in the US indicates that assistive social robots seem to have a favorable effect on children’s language development (Westlund et al.). Existing theories of language acquisition emphasize the role of language input and the child’s interaction with the environment as crucial to language development. From this perspective, we need to ask: What are the consequences of this new digital reality for children’s acquisition of the most fundamental of all human skills: language and communication? Are new theories needed that can help us understand how children acquire language? Do the new digital environment and the new ways of interaction change the way languages are learned, or the quality of language acquisition? Is the use of new media beneficial or harmful to children’s language and cognitive development? Can new technologies be tailored to support child growth and, most importantly, can they be designed to enhance language learning in vulnerable children? These questions and issues can only be addressed bymeans of an interdisciplinary approach that aims at developing new methods of data collection and analysis in a longitudinal perspective. This type of research is however not yet documented.

Communicative interactions involve a kind of procedural knowledge that is used by the human brain for processing verbal and nonverbal inputs and for language production. Although considerable work has been done on modeling human language abilities, it has been difficult to bring them together to a comprehensive tabula rasa system compatible with current knowledge of how verbal information is processed in the brain. This work presents a cognitive system, entirely based on a large-scale neural architecture, which was developed to shed light on the procedural knowledge involved in language elaboration. The main component of this system is the central executive, which is a supervising system that coordinates the other components of the working memory. In our model, the central executive is a neural network that takes as input the neural activation states of the short-term memory and yields as output mental actions, which control the flow of information among the working memory components through neural gating mechanisms. The proposed system is capable of learning to communicate through natural language starting from tabula rasa, without any a priori knowledge of the structure of phrases, meaning of words, role of the different classes of words, only by interacting with a human through a text-based interface, using an open-ended incremental learning process. It is able to learn nouns, verbs, adjectives, pronouns and other word classes, and to use them in expressive language. The model was validated on a corpus of 1587 input sentences, based on literature on early language assessment, at the level of about 4-years old child, and produced 521 output sentences, expressing a broad range of language processing functionalities.

A comprehensive overview of an interdisciplinary approach to robotics that takes direct inspiration from the developmental and learning phenomena observed in children's cognitive development. Developmental robotics is a collaborative and interdisciplinary approach to robotics that is directly inspired by the developmental principles and mechanisms observed in children's cognitive development. It builds on the idea that the robot, using a set of intrinsic developmental principles regulating the real-time interaction of its body, brain, and environment, can autonomously acquire an increasingly complex set of sensorimotor and mental capabilities. This volume, drawing on insights from psychology, computer science, linguistics, neuroscience, and robotics, offers the first comprehensive overview of a rapidly growing field. After providing some essential background information on robotics and developmental psychology, the book looks in detail at how developmental robotics models and experiments have attempted to realize a range of behavioral and cognitive capabilities. The examples in these chapters were chosen because of their direct correspondence with specific issues in child psychology research; each chapter begins with a concise and accessible overview of relevant empirical and theoretical findings in developmental psychology. The chapters cover intrinsic motivation and curiosity; motor development, examining both manipulation and locomotion; perceptual development, including face recognition and perception of space; social learning, emphasizing such phenomena as joint attention and cooperation; language, from phonetic babbling to syntactic processing; and abstract knowledge, including models of number learning and reasoning strategies. Boxed text offers technical and methodological details for both psychology and robotics experiments.

The topic is characterized by a highly interdisciplinary approach to the issue of action and language integration. Such an approach, combining computational models and cognitive robotics experiments with neuroscience, psychology, philosophy, and linguistic approaches, can be a powerful means that can help researchers disentangle ambiguous issues, provide better and clearer definitions, and formulate clearer predictions on the links between action and language. In the introduction we briefly describe the papers and discuss the challenges they pose to future research. We identify four important phenomena the papers address and discuss in light of empirical and computational evidence: (a) the role played not only by sensorimotor and emotional information but also of natural language in conceptual representation; (b) the contextual dependency and high flexibility of the interaction between action, concepts, and language; (c) the involvement of the mirror neuron system in action and language processing; (d) the way in which the integration between action and language can be addressed by developmental robotics and Human-Robot Interaction.

A robotic and embodied approach to the modeling of the evolution of language addresses various aspects of language origins such as prelinguistic social coordination, signaling behavior, and the emergence of compositional lexicons. Robot language experiments typically involve tasks in which the robots must communicate about objects and entities in the environment, about their physical interaction with objects, and about their body posture. Embodied agents are multiagent systems in which a population of simulated agents live in a shared environment, can receive visual, auditory, and tactile information about the world, and can act on it. These experiments typically involve communication about spatial navigation and foraging tasks. Robotic and embodied agent models have made significant contributions to the understanding of genetic and cultural evolution dynamics in language origins, where both the semantic system and the lexicon interact and co-adapt during linguistic evolution. Robotics models have mostly focused on the emergence of shared lexicons through cultural evolution. Simulated embodied agent models have mostly investigated the genetic evolution of shared languages. Simulation models of embodied agents have been employed to model the genetic evolution of shared lexicons. The embodied modeling approach has also been employed specifically to look at the evolutionary emergence of syntax, with particular focus on compositionality. Embodied multiagent systems have also been used for modeling the cultural evolution of syntax.

This symposium includes a selection of articles on the origins and evolution of language. These are extended version of selected papers presented at ``EVOLANG6: The Sixth International Conference on the Evolution of Language'' that was held in Rome in April 2006. This selection of papers provides a multi-methodological view of different approaches to, and theoretical explanations of, the evolution of language.

In this paper we present the `grounded adaptive agent' computational framework for studying the emergence of communication and language. This modeling framework is based on simulations of population of cognitive agents that evolve linguistic capabilities by interacting with their social and physical environment (internal and external symbol grounding). These models provide an integrative vision of language where the linguistic abilities of cognitive agents strictly depend on other social, sensorimotor, neural and cognitive capabilities. Here language is not seen as an isolated and dedicated symbol processing system, but rather as a heterogeneous set of artifacts implicated in cultural and cognitive activities. The proposed modeling approach is also closely related to embodied cognition theories of the grounding of language in the organism's perceptual and motor systems.

This volume brings together studies from diverse disciplines, showing how they can inform and stimulate each other. It includes work in linguistics, psychology, neuroscience, anthropology and computer science. New empirical work is reported on both human and animal communication, using some novel techniques that have only recently become workable.

A principal theme is the importance of studies involving artificial agents, their contribution to the body of knowledge on the emergence of communication and language, and the role of simulations in exploring some of the most significant issues. A number of different synthetic systems are described, showing how communication can emerge in natural and artificial organisms. Theories on the origins of language are supported by computational and robotic experiments.

Worldwide contributors to this volume include some of the most influential figures in the field, delivering essential reading for researchers and graduates in the area, as well as providing fascinating insights for a wider readership.

Contents

Introduction

Current Work and Open Problems: A Roadmap for Research into the Emergence of Communication and Language by Chrystopher L. Nehaniv, Caroline Lyon, and Angelo Cangelosi

Section 1: Empirical Investigations on Human Language

• Evolving Meaning: The Roles of Kin Selection, Allomothering and Paternal Care in Language Evolution by W. Tecumseh Fitch
• `Needs only' Analysis in Linguistic Ontogeny and Phylogeny by Alison Wray
• Clues from Information Theory Indicating a Phased Emergence of Grammar by Caroline Lyon, Chrystopher L. Nehaniv and Bob Dickerson
• Emergence of a Communication System: International Sign by Rachel Rosenstock
• Distributed Language: Biomechanics, Functions, and the Origins of Talk by Stephen J. Cowley

Section 2: Synthesis of Communication and Language in Artificial Systems

• The Recruitment Theory of Language Origins by Luc Steels
• In silico Evolutionary Developmental Neurobiology and the Origin of Natural Language by Eors Szathmary, Zoltan Szatmary, Peter Ittzes, Gergo Orban, Istvan Zachar, Ferenc Huszar, Anna Fedor, Mate Varga, Szabolcs Szamado
• Communication in Natural and Artificial Organisms: Experiments in Evolutionary Robotics by Davide Marocco and Stefano Nolfi
• From Vocal Replication to Shared Combinatorial Speech Codes: A Small Step for Evolution, a Big Step for Language by Pierre-Yves Oudeyer
• Learning and Transition of Symbols: Towards a Dynamical Model of a Symbolic Individual by Takashi Hashimoto and Akira Masumi
• Language Change among `Memoryless Learners' Simulated in Language Dynamics Equations by Makoto Nakamura, Takashi Hashimoto and Satoshi Tojo
• The Evolution of Meaning-space Structure through Iterated Learning by Simon Kirby
• The Emergence of Language: How to Simulate It by Domenico Parisi and Marco Mirolli
• Lexical Acquisition with and without Metacommunication by Jonathan Ginzburg and Zoran Macura
• Agent Based Modelling of Communication Costs: Why Information Can be Free by Ivana Cace and Joanna Bryson
• Language Change and the Inference of Meaning by Andrew D. M. Smith
• Language, Perceptual Categories and their Interaction: Insights from Computational Modelling by Tony Belpaeme and Joris Bleys

Section 3: Insights from Animal Communication

• Emergence of Linguistic Communication: Studies on Grey Parrots by Irene M. Pepperberg
• A Possible Role for Selective Masking in the Evolution of Complex, Learned Communication Systems by Graham R. S. Ritchie and Simon Kirby
• The Natural History of Human Language: Bridging the Gaps without Magic by Bjorn Merker and Kazuo Okanoya
• Neural Substrates for String-Context Mutual Segmentation: A Path to Human Language by Kazuo Okanoya and Bjorn Merker

The original idea for this book came from the successful 2nd International Symposium on the Emergence and Evolution of Linguistic Communication (EELC '05) held in Hatfield, UK, in April 2005. Grants from the British Academy and the UK Engineering and Physical Sciences Research Council in support of this workshop are gratefully acknowledged.

The grounding of symbols in computational models of linguistic abilities is one of the fundamental properties of psychologically-plausible cognitive models. This paper presents an embodied model for the grounding of language in action based on epigenetic robots. Epigenetic robotics is one of the new cognitive modeling approaches to modeling autonomous mental development. The robot model is based on an integrative vision of language, in which linguistic abilities are strictly dependent on, and grounded in, other behaviors and skills. It uses simulated robots that learn through imitation the names of basic actions. Robots also learn higher-order action concepts through the process of grounding transfer. The simulation demonstrates how new, higher-order behavioral abilities can be autonomously built upon previously-grounded basic action categories, following linguistic interaction with human users.

Neural networks have been proposed as an ideal cognitive modeling methodology to deal with the symbol grounding problem. More recently, such neural network approaches have been incorporated in studies based on cognitive agents and robots. In this paper we present a new model of symbol grounding transfer in cognitive robots. Language learning simulations demonstrate that robots are able to acquire new action concepts via linguistic instructions. This is achieved by autonomously transferring the grounding from directly grounded action names to new higher-order composite actions. The robot's neural network controller permits such a grounding transfer. The implications for such a modeling approach in cognitive science and autonomous robotics are discussed.

The double function of language, as a social/communicative means, and as an individual/cognitive capability, derives from its fundamental property that allows us to internally re-represent the world we live in. This is possible through the mechanism of symbol grounding, i.e. the ability to associate entities and states in the external and internal world with internal categorical representations. The symbol grounding mechanism, as language, has both an individual and a social component. The individual component, called the ``Physical Symbol Grounding'', refers to the ability of each individual to create an intrinsic link between world entities and internal categorical representations. The social component, called ``Social Symbol Grounding'', refers to the collective negotiation for the selection of shared symbols (words) and their grounded meanings. The paper discusses these two aspects of symbol grounding in relation to distributed cognition, using examples from cognitive modeling research on grounded agents and robots.

In this article we present an evolutionary technique for developing a neural network based controller for an an- thropomorphic robotic arm with 4 DOF able to exhibit a reaching behaviour. Evolved neural controllers display an ability to reach targets accurately and generalize their ability to moving targets. This study demonstrates that it is possible to obtain solutions that are extremely parsimonious from the point of view of the control system. Evolutionary training techniques allow us to evolve parameters of the control system on the basis of the global effects that they produce on the dynamics arising from the interaction between the control system, the robot's body and the environment.

The paper presents a computational model of language in which linguistic abilities evolve in organisms that interact with an environment. Each individual's behavior is controlled by a neural network and we study the consequences in the network's internal functional organization of learning to process different classes of words. Agents are selected for reproduction according to their ability to manipulate objects and to understand nouns (objects' names) and verbs (manipulation tasks). The weights of the agents' neural networks are evolved using a genetic algorithm. Synthetic brain imaging techniques are then used to examine the functional organization of the neural networks. Results show that nouns produce more integrated neural activity in the sensory-processing hidden layer, while verbs produce more integrated synaptic activity in the layer where sensory information is integrated with proprioceptive input. Such findings are qualitatively compared with human brain imaging data that indicate that nouns activate more the posterior areas of the brain related to sensory and associative processing, while verbs activate more the anterior motor areas.

Evolutionary robotics is a biologically inspired approach to robotics that is advantageous to studying the evolution of communication. A new model for the emergence of communication is developed and tested through various simulation experiments. In the first simulation, the emergence of simple signalling behaviour is studied. This is used to investigate the inter-relationships between communication abilities, namely linguistic production and comprehension, and other behavioural skills. The model supports the hypothesis that the ability to form categories from direct interaction with an environment constitutes the grounds for subsequent evolution of communication and language. In the second simulation, evolutionary robots are used to study the emergence of simple syntactic categories, e.g. action names (verbs). Comparisons between the two simulations indicate that the signalling lexicon emerged in the first simulation follows the evolutionary pattern of nouns, as observed in related models on the evolution of syntactic categories. Results also support the language-origin hypothesis on the fact that nouns precede verbs in both phylogenesis and ontogenesis. Further extensions of this new evolutionary robotic model for testing hypotheses on language origins are also discussed.

Scholars studying the origins and evolution of language are also interested in the general issue of the evolution of cognition. Language is not an isolated capability of the individual, but has intrinsic relationships with many other behavioral, cognitive, and social abilities. ...

The Baldwin effect has been explicitly used by Pinker and Bloom as an explanation of the origins of language and the evolution of a language acquisition device. This article presents new simulations of an artificial life model for the evolution of compositional languages. It specifically addresses the role of cultural variation and of learning costs in the Baldwin effect for the evolution of language. Results show that when a high cost is associated with language learning, agents gradually assimilate in their genome some explicit features (e.g., lexical properties) of the specific language they are exposed to. When the structure of the language is allowed to vary through cultural transmission, Baldwinian processes cause, instead, the assimilation of a predisposition to learn, rather than any structural properties associated with a specific language. The analysis of the mechanisms underlying such a predisposition in terms of categorical perception supports Deacon's hypothesis regarding the Baldwinian inheritance of general underlying cognitive capabilities that serve language acquisition. This is in opposition to the thesis that argues for assimilation of structural properties needed for the specification of a full-blown language acquisition device.

This paper describes different types of models for the evolution of communication and language. It uses the distinction between signals, symbols, and words for the analysis of evolutionary models of language. In particular, it show how evolutionary computation techniques, such as Artificial Life, can be used to study the emergence of syntax and symbols from simple communication signals. Initially, a computational model that evolves repertoires of isolated signals is presented. This study has simulated the emer- gence of signals for naming foods in a population of foragers. This type of model studies communication systems based on simple signal-object associations. Subsequently, models that study the emergence of grounded symbols are discussed in general, including a detailed description of a work on the evolution of simple syntactic rules. This model focuses on the emergence of symbol-symbol relationships in evolved languages. Finally, computational models of syntax acquisition and evolution are discussed. These different types of computational models provide an operational definition of the signal/symbol/word distinction. The simulation and analysis of these types of models will help understanding the role of symbols and symbol acquisition in the origin of language.

This paper presents an Artificial Life and Neural Network (ALNN) model for the evolution of syntax. The simulation methodology provides a unifying approach for the study of the evolution of language and its interaction with other behavioral and neural factors. The model uses an object manipulation task to simulate the evolution of language based on a simple verb-noun rule. The analyses of results focus on the interaction between language and other non-linguistic abilities, and on the neural control of linguistic abilities. The model shows that the beneficial effects of language on non-linguistic behavior are explained by the emergence of distinct internal representation patterns for the processing of verbs and nouns.

Neural network models of categorical perception (compression of withincategory similarity and dilation of between-category differences) are applied to the symbol-grounding problem (of how to connect symbols with meanings) by connecting analogue sensorimotor projections to arbitrary symbolic representations via learned category-invariance detectors in a hybrid symbolic/non-symbolic system. Our nets are trained to categorize and name 50 2 50 pixel images (e.g. circles, ellipses, squares and rectangles) projected on to the receptive field of a 7 2 7 retina. They first learn to do prototype matching and then entry-level naming for the four kinds of stimuli, grounding their names directly in the input patterns via hidden-unit representations ('sensorimotor toil'). We show that a higher-level categorization (e.g. 'symmetric' versus 'asymmetric') can be learned in two very different ways: either (1) directly from the input, just as with the entry-level categories (i.e. by toil); or (2) indirectly, from Boolean combinations of the grounded category names in the form of propositions describing the higher-order category ('symbolic theft'). We analyse the architectures and input conditions that allow grounding (in the form of compression/ separation in internal similarity space) to be 'transferred' in this second way from directly grounded entry-level category names to higher-order category names. Such hybrid models have implications for the evolution and learning of language.

The evolution of language implies the parallel evolution of an ability to respond appropriately to signals (language understanding) and an ability to produce the appropriate signals in the appropriate circumstances (language production). When linguistic signals are produced to inform other individuals, individuals that respond appropriately to these signals may increase their reproductive chances but it is less clear what the reproductive advantage is for the language producers. We present simulations in which populations of neural networks living in an environment evolve a simple language with an informative function. Signals are produced to help other individuals categorize edible and poisonous mushrooms, in order to decide whether to approach or avoid encountered mushrooms. Language production, while not under direct evolutionary pressure, evolves as a byproduct of the independently evolving perceptual ability to categorize mushrooms.