skip to content
Evolving Culture Versus Local Minima Preview this item
ClosePreview this item
Checking...

Evolving Culture Versus Local Minima

Author: Yoshua Bengio Affiliation: CIFAR Fellow, Department of computer science and operations research, University of Montréal, Montreal, Canada
Edition/Format: Chapter Chapter : English
Database:SpringerLink
Summary:
We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concepts that cover a vast and highly-nonlinear  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

 

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

All Authors / Contributors: Yoshua Bengio Affiliation: CIFAR Fellow, Department of computer science and operations research, University of Montréal, Montreal, Canada
ISBN: 978-3-642-55336-3 978-3-642-55337-0
Publication:Kowaliw, Taras, taras@kowaliw.ca, CNRS, Institut des Systèmes Complexes - Paris Île-de-France, Paris, France; Growing Adaptive Machines : Combining Development and Learning in Artificial Neural Networks; 109-138; Berlin, Heidelberg : Springer Berlin Heidelberg : Springer
Language Note: English
Unique Identifier: 5679765656
Notes: The author would like to thank Caglar Gulcehre, Aaron Courville, Myriam Côté, and Olivier Delalleau for useful feedback, as well as NSERC, CIFAR and the Canada Research Chairs for funding.
Awards:

Abstract:

We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concepts that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best represented in brains by the composition of many levels of representation, i.e., by deep architectures; (4) a human brain can learn such high-level abstractions if guided by the signals produced by other humans, which act as hints or indirect supervision for these high-level abstractions; and (5), language and the recombination and optimization of mental concepts provide an efficient evolutionary recombination operator, and this gives rise to rapid search in the space of communicable ideas that help humans build up better high-level internal representations of their world. These hypotheses put together imply that human culture and the evolution of ideas have been crucial to counter an optimization difficulty: this optimization difficulty would otherwise make it very difficult for human brains to capture high-level knowledge of the world. The theory is grounded in experimental observations of the difficulties of training deep artificial neural networks. Plausible consequences of this theory for the efficiency of cultural evolution are sketched.

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.
Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/5679765656> # Evolving Culture Versus Local Minima
    a bgn:Chapter, schema:CreativeWork ;
    library:oclcnum "5679765656" ;
    rdfs:comment "949 $l OriginalPaper" ;
    schema:about <http://experiment.worldcat.org/entity/work/data/2504296909#Thing/artificial_intelligence_incl_robotics> ; # Artificial Intelligence (incl. Robotics)
    schema:about <http://experiment.worldcat.org/entity/work/data/2504296909#Thing/engineering> ; # Engineering
    schema:about <http://experiment.worldcat.org/entity/work/data/2504296909#Thing/computational_intelligence> ; # Computational Intelligence
    schema:creator <http://experiment.worldcat.org/entity/work/data/2504296909#Person/bengio_yoshua> ; # Yoshua Bengio
    schema:description "We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concepts that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best represented in brains by the composition of many levels of representation, i.e., by deep architectures; (4) a human brain can learn such high-level abstractions if guided by the signals produced by other humans, which act as hints or indirect supervision for these high-level abstractions; and (5), language and the recombination and optimization of mental concepts provide an efficient evolutionary recombination operator, and this gives rise to rapid search in the space of communicable ideas that help humans build up better high-level internal representations of their world. These hypotheses put together imply that human culture and the evolution of ideas have been crucial to counter an optimization difficulty: this optimization difficulty would otherwise make it very difficult for human brains to capture high-level knowledge of the world. The theory is grounded in experimental observations of the difficulties of training deep artificial neural networks. Plausible consequences of this theory for the efficiency of cultural evolution are sketched." ;
    schema:exampleOfWork <http://worldcat.org/entity/work/id/2504296909> ;
    schema:isPartOf <http://worldcat.org/issn/1860-949X> ; # Studies in Computational Intelligence
    schema:name "Evolving Culture Versus Local Minima" ;
    schema:pageEnd "138" ;
    schema:pageStart "109" ;
    schema:productID "5679765656" ;
    schema:sameAs <http://dx.doi.org/10.1007/978-3-642-55337-0_3> ;
    wdrs:describedby <http://www.worldcat.org/title/-/oclc/5679765656> ;
    .


Related Entities

<http://experiment.worldcat.org/entity/work/data/2504296909#Thing/artificial_intelligence_incl_robotics> # Artificial Intelligence (incl. Robotics)
    a schema:Thing ;
    schema:name "Artificial Intelligence (incl. Robotics)" ;
    .

<http://experiment.worldcat.org/entity/work/data/2504296909#Thing/computational_intelligence> # Computational Intelligence
    a schema:Thing ;
    schema:name "Computational Intelligence" ;
    .

<http://worldcat.org/issn/1860-949X> # Studies in Computational Intelligence
    a bgn:PublicationSeries ;
    schema:hasPart <http://www.worldcat.org/oclc/> ;
    schema:issn "1860-949X" ;
    schema:name "Studies in Computational Intelligence" ;
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.