In this chapter, we describe a tensor network (TN) based common language established between machine learning and many-body physics, which allows for bidirectional contributions. By showing that many-body wave functions are structurally equivalent to mappings of convolutional and recurrent networks, we bring forth quantum entanglement measures as natural quantifiers of dependencies modeled by such networks. Accordingly, we propose a novel entanglement-based deep learning design scheme that sheds light on the success of popular architectural choices made by deep learning practitioners and suggests new practical prescriptions. In the other direction, we construct TNs corresponding to deep recurrent and convolutional networks. This allows us to theoretically demonstrate that these architectures are powerful enough to represent highly entangled quantum systems polynomially more efficiently than previously employed architectures. We thus provide theoretical motivation to shift neural-network-based wave function representations closer to state-of-the-art deep learning architectures.