Part II Deep models

Chapter 4 Model components

A deep model is nothing more than a complex tensorial computation that can be decomposed ultimately into standard mathematical operations from linear algebra and analysis. Over the years, the field has developed a large collection of high-level modules that have a clear semantic, and complex models combining these modules, which have proven to be effective in specific application domains.

Empirical evidence and theoretical results show that greater performance is achieved with deeper architectures, that is, long compositions of mappings. As we saw in section § 3.4, training such a model is challenging due to the vanishing gradient, and multiple important technical contributions have mitigated this problem.