科学工程智能.pptx

科学工程智能;;;5;6;;8;;;;;;;;;;;;Annual STC meeting, 2016;;;;;;;When and why are deep networks better than shallow networks?;28;29;30;31;32;;;Riesenhuber Poggio 1999, 2000; Serre Kouh Cadieu Knoblich Kreiman Poggio 2005; Serre Oliva Poggio 2007;;37;;;40;When and why are deep networks better than shallow networks? Why does SGD work so well for deep networks? Could unsupervised learning work as well?;How do the learning machines described by classical learning theory -- such as kernel machines -- compare with brains? One of the most obvious differences is the ability of people and animals to learn from very few examples (“poverty of stimulus” problem). A comparison with real brains offers another, related, challenge to learning theory. Classical “learning algorithms” correspond to one-layer architectures. The cortex suggests a hierarchical architecture. Thus…are hierarchical architectures with more layers the answer to the;Deep and shallow networks: universality;;can be “written” as shallow networks: the value of K corresponds to the “activity” of the “unit” for the input and the correspond to “weights”;Deep and shallow networks: universality;Deep and shallow networks;Deep and shallow networks;;;;;This is the best possible estimate (n-width result);;;56;;;The curse of dimensionality, the blessing of compositionality;Why are compositional functions important?;61;62;63;;x1 x2;;;;Images require algorithms of the compositional function type Recognition in clutter requires computations with compositional functions;;;;;;