Briefly, when the striatum is involved, local and incremental reinforcement learning rules facilitate subsequent learning of spatial navigation tasks based on egocentric information (Figure 1A). Covering all aspects of learning underlying spatial navigation is beyond the scope of this review where we will focus an aspects that is amenable to AI approaches, RL. (2011). Biobehav. Conversely, a way to expand our current understanding of the neuroscience of spatial navigation is to use AI and machine learning techniques to aid in analyzing some large data sets. Phase precession and variable spatial scaling in a periodic attractor map model of medial entorhinal grid cells with realistic after-spike dynamics. Reinforcement learning, fast and slow. doi: 10.1126/science.1127241, Dragoi, G., and Tonegawa, S. (2011). Tishby, N., and Zaslavsky, N. (2015). Arthropod. Neurosci. New York, NY: Oxford University Press. ... Neuroscience 197, 233–241. Deep learning in neural networks: an overview. Which coordinate system for modelling path integration? Nat. Having access to different scales allows the system to represent space at different resolutions. It has also recently been applied in several domains in machine learning. The reward signals are thought to originate in the VTA. 22, 1761–1770. In AI, this includes the development of computer programs that can beat a grandmaster at GO or outperform human radiologists at cancer detection. For example, with more studies about grid cells, models that aim to understand how place cells and grid cells interact have been very important to understand the restrictions in the circuitry between the enthorinal cortex and the hippocampus (Solstad et al., 2006). There are limitations and criticisms for these contributions in neuroscience. In addition, and more related to spatial navigation, DNNs have also been applied to decode sensory and behavioral information such as animal position and orientation from hippocampal activity (Frey et al., 2019; Xu Z. et al., 2019). (2017). Head-direction cells recorded from the postsubiculum in freely moving rats. doi: 10.1016/j.conb.2018.01.009, Stoianov, I. P., Pennartz, C. M. A., Lansink, C. S., and Pezzulo, G. (2018). The cells that encode space in allocentric or map-like coordinates are generally found in the hippocampal formation and several limbic-thalamic and limbic-cortical regions. Finally, we want to clarify that the classification of the models presented here is not necessarily exhaustive, mutually exclusive or discrete. Computational neuroscience (also known as theoretical neuroscience or mathematical neuroscience) is a branch of neuroscience which employs mathematical models, theoretical analysis and abstractions of the brain to understand the principles that govern the development, structure, physiology and cognitive abilities of the nervous system.. 127, 49–69. Neural Netw. Cambridge, MA: MIT Press. Is coding a relevant metaphor for building AI? doi: 10.1126/sciadv.aaz2322, PubMed Abstract | CrossRef Full Text | Google Scholar, Alexander, A. S., and Nitz, D. A. Although more research in this area is needed, this section provides a brief review of the neuroscience underlying this ability within the context of RL. For example, an important mechanism that links what we know about learning and memory and spatial navigation are the theories about how memories are consolidated in the brain. More information will be made available shortly. We propose that spatial navigation is an excellent area in which these two disciplines can converge to help advance what we know about the brain. “A model of the neural basis of the rat's sense of direction,” in Proceedings of the Seventh International Conference of Neural Information Processing Systems (NIPS), (Denver, CO), 173–180. How is it possible that organized clumps of matter such as our own brains give rise to all of our beliefs, desires and intentions, ultimately allowing us to contemplate ourselves as well as the universe from which we originate? doi: 10.1007/BF00243212, Chersi, F., and Burgess, N. (2015). Frontiers is based in Lausanne, Switzerland, with other offices in London, Madrid, Seattle and Brussels. Firing rate (Hz) is represented in upper right corner for each example cell. A consequence of this framework is a sustained hill of excitation centered on the animal's current HD. (2018). Therefore, from this perspective, instead of solely looking for the responsible brain structures involved in spatial navigation and their neural codes, the study of the restrictions imposed by the environment and anatomy of the organism, might help to better understand how the internal representations are constructed and how they are manipulated to navigate in space (Krichmar and Edelman, 2005; Evans et al., 2016; Brette, 2019; Santoro et al., 2019). Neurosci. While place and grid cells can be modulated by the shape of the environment, recent work has shown that the parahippocampal cortex (medial entorhinal cortex, presubiculum, parasubiculum) also contains neurons that respond specifically to boundary stimuli (Figure 2C; Solstad et al., 2008; Boccara et al., 2010). For example, further investigation of learning of spatial representation at multiple scales in time and levels of abstraction, the role of memory in these processes, knowledge extraction, learning to learn, and understanding how the brain performs coordinate transformation between body-centered and map-like representations. doi: 10.1038/381425a0, O'Keefe, J., and Dostrovsky, J. For example, an allocentric to egocentric transformation may allow a subject to select an action (turn left) at a specific intersection (a particular allocentric location and orientation) in a city. (D) Schematic representation of the deep RL approach for spatial navigation. PLoS Comput. Descriptive models of spatial navigation have the goal of characterizing what the system does, usually reproducing experimental data (Sutherland and Hamilton, 2004). “Emergence of grid-like representations by training recurrent neural networks to perform spatial localization,” in International Conference on Learning Representations (ICLR), (Vancouver, BC), 1–19. Deep learning and the information bottleneck principle” in 2015 IEEE Information Theory Workshop ITW 2015 (Jeju Island). In a more recent model (Bush et al., 2015) modeled grid cells to support vector navigation and provide provide a framework for testing experimental predictions. Modeling brain connections to understand Parkinson’s disease October 3, 2017 A new model in Frontiers in Computational Neuroscience finds differences in basal ganglia connection strengths between healthy and Parkinson's disease brains. doi: 10.1371/journal.pcbi.1006316, Sutherland, R. J., and Hamilton, D. A. J. Neurosci. Inspired by Schultheiss and Redish (2015), used with permission. Resynthesizing behavior through phylogenetic refinement. Neuron 103, 967–979. (2016). If we assume that intelligent behavior can be understood by studying how it emerges, it is reasonable to attempt to learn from a working example: biological brains. In this case, these processes are thought to be implemented by the interaction between the hippocampus and the ventral striatum (Pennartz et al., 2011). Ring attractor network model of head-direction (bottom). 23, 408–422. Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Behav. These spatial cell types have been identified in a neural circuit that includes the hippocampal formation and several limbic-thalamic and limbic-cortical regions (see Figures 1A,B). The neurobiological basis of map-like spatial representations is thought to involve a network of spatially selective neurons in the mammalian nervous system. Historically, Artificial Intelligence (AI) researchers followed this approach. (2018), used with permission. 16, 309–317. Neurobiol. Neurosci. Sci. J. Neurosci. The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. In AI and neuroscience, research in RL and rodent spatial navigation has already proved to be a successful approach to understand how these two concepts are closely related and the interaction between these fields can help to inform one another. Cybern. Trends Cogn. doi: 10.1002/9780470147658.chpsy0106. Engineering a less artificial intelligence. Biol. Cybern. Rev. In this paper we propose spatial navigation as a common ground where research in neuroscience and AI can converge to expand our understanding of the brain. Comput. In this section we review the modeling work developed to understand how the brain transforms spatial information between different frames of reference. These are crucial cognitive components of intelligence which can have a great impact in neuroscience and AI. Why neurons have thousands of synapses, a theory of sequence memory in neocortex. In other words, transform body centered encoding of a landmark into map-like landmark representations (e.g., a cell that fires in a specific map-like location relative to a landmark independent of which direction the animal is facing; Figure 2D). How environment and self-motion combine in neural representations of space. The goal of neuroscience is precisely that—to understand how the brain works. The neuroscience of spatial navigation. doi: 10.1038/nn.4650, Steinmetz, N. A., Koch, C., Harris, K. D., and Carandini, M. (2018). Hippocampus 16, 1026–1031. 34, 548–559. Research has identified this signal in the anterior thalamic nuclei, retrosplenial, parietal, and parahippocampal (entorhinal, postsubiculum, and parasubiculum) cortices (Taube, 2007). Rev. Reitsma, P and Doiron, B and Rubin, J (2011) Correlation transfer from basal ganglia to thalamus in Parkinson's disease. (2020). Entorhinal velocity signals reflect environmental geometry. doi: 10.1038/nrn1607, Frey, M., Tanni, S., Perrodin, C., Leary, A. O., Nau, M., Kelly, J., et al. Thalamocortical processing of the head-direction sense. 21, 1281–1289. Computational descriptive models propose that cell populations within the anterior thalamic nuclei, parietal cortex, and retrosplenial cortex operate as a network that transforms spatial information from an egocentric (e.g., body centered) to allocentric (i.e., map-like) frame of reference and vice versa (reviewed in Clark et al., 2018). What is a cognitive map? Procedia Comput. Front. Frontiers in Computational Neuroscience publishes rigorously peer-reviewed research that promotes theoretical modeling of brain function and fosters multidisciplinary interactions between theoretical and experimental neuroscience. While the cells that encode egocentric or body-centered coordinates are generally found in subcortical regions such as a basal ganglia and posterior cortical regions such as the parietal cortex. For example, DNNs have been used to reproduce brain activity in the visual system to learn about the organization of this network in primates (Walker et al., 2019) and mice (Cadena et al., 2019). Sci. Egocentric boundary vector tuning of the retrosplenial cortex. This limitation is contrasted with the biological counterparts in which learning happens very rapidly in most cases. Reinforcement Learning. Despite this limitation, this approach might still provide controlled, reproducible experimental sand-boxes to improve our current analytical tools (that can be applied to real brain data) or to generate and test new hypotheses (Jonas and Kording, 2017). Thus, adjacent HD cells on the “ring” share similar, but slightly offset, preferred firing directions (though not necessarily physically adjacent positions in the brain). The compass within. Models of all styles and at all levels are welcome, from biophysically motivated realistic simulations of neurons and synapses to high-level abstract models of inference and decision making. Nat. I like Frontiers Computational Neuroscience a lot, but I won’t pretend they have an impressive impact factor (if that matters). Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., and Robbins, T. W. (2011). View BMEN90002 Frontiers in Neuroscience - 1 The Brain as Computer.pdf from BMEN 90002 at University of Melbourne. Laminar organization of encoding and memory reactivation in the parietal cortex. Predictive representations in hippocampal and prefrontal hierarchies. Rev. Introducing variability or “noise” in the training data or the computing units (Destexhe and Contreras, 2006; Guo et al., 2018; Wu et al., 2019) (arguably modeling their biological counterparts) can shape the properties of their representations (Faisal et al., 2008). Deshmukh, S. S., and Knierim, J. J. doi: 10.1152/jn.00145.2018, Cazin, N., Llofriu Alonso, M., Scleidorovich Chiodi, P., Pelc, T., Harland, B., Weitzenfeld, A., et al. News • CNS*2020 will be held online. Five decades of hippocampal place cells and EEG rhythms in behaving rats. In spatial navigation, analogous computations are thought to be egocentric (or route-based) in which no cognitive map is used to reach a goal location. At the moment, most of the deep learning approaches use a limited repertoire of what is known about how brain cells compute information. Neuroscience-inspired artificial intelligence. Nat. In particular, end-to-end approaches to solve navigation tasks can help in the advancement of the neuroscience of spatial navigation because the potential solutions are not restricted to the current knowledge of the experimenter. J. Neurosci. Another related limitation is the number of training examples that DL requires to learn. Framing spatial cognition: Neural representations of proximal and distal frames of reference and their roles in navigation. Parallel Distributed Processing, Vol. doi: 10.1523/JNEUROSCI.0511-14.2014, Wilber, A. Data-driven analyses of motor impairments in animal models and neurological disorders. 20, 553–557. (2017). doi: 10.1126/science.1148979, Evans, T., Bicanski, A., Bush, D., and Burgess, N. (2016). Neurobiol. Computational and Mathematical Modeling of Neural Systems. Exp. On the one hand, a successfully trained ANN that solves a navigation task provides the opportunity to repeat and manipulate environmental conditions (e.g., sensory inputs) and parameters (e.g., network topology) to gain insights into possibly interesting avenues to follow in rodent experiments. In addition, spatial navigation involves several cognitive processes that are crucial for a broad range of intelligent behavior. This might be due to the fact that brains are not completely randomly connected at birth such that we have to learn everything from scratch. Frontiers in Computational Neuroscience, 8 (MAY). Acceptance Rate. (2013). Biol. (2019). Spatial navigation systems, in mammals at least, are highly robust and adaptable to different levels of sensory information and environmental conditions. Specialty Chief Editors Misha Tsodyks at the Weizmann Institute of Science and Si Wu at the Beijing Normal University are supported by an outstanding … Jonas, E., and Kording, K. P. (2017). Grid cells require excitatory drive from the hippocampus. Trends Neurosci. To conclude, by building models and agents that solve spatial navigation tasks following the restrictions imposed by the interactions of the body and environment found in biological systems, we argue that we can not only learn more about the brain but also how the processes involved in complex intelligent behavior might rise. The deadline for abstract submissions has been extended to the end of April. Nat. An occupancy grid mapping enhanced visual SLAM for real-time locating applications in indoor GPS-denied environments. On the other hand, it provides an opportunity to develop analytical tools to understand the complex mechanisms employed in these models and apply them to real data. Grid cells have been identified in parahippocampal cortex (medial entorhinal cortex, presubiculum, parasubiculum) and differ from place cells in that they fire in multiple locations forming a hexagonal grid pattern (Figure 2B; Hafting et al., 2005). doi: 10.1016/j.neuroscience.2011.09.020. Momennejad, I., Otto, A. R., Daw, N. D., and Norman, K. A. Nat. “Learning to navigate in complex environments,” in International Conference on Learning Representations (ICLR). (2013). These simulations can help to understand how the transformation of egocentric and allocentric frames of reference can be employed by the brain when using different navigation strategies. For example Whittington et al. Nat. 14, 1–28. In fact, many of the initial ideas of numerous state-of-the-art algorithms in AI were derived from psychology and neuroscience (Hassabis et al., 2017). (A) Key brain structures involved in rodent spatial navigation. Major advances in our understanding of how the brain is involved in spatial navigation has been achieved in part, due to modeling work. Recent advances in artificial intelligence (AI) and neuroscience are impressive. A new model in Frontiers in Computational Neuroscience finds differences in basal ganglia connection strengths between healthy and Parkinson's disease brains. Neurosci. A logical calculus of the ideas immanent in nervous activity. In contrast, when the hippocampus is involved, faster one-shot associative learning rules are applied to solve spatial navigation. Science 589, 584–589. Artif. Pfeifer, R., and Scheier, C. (1999). In addition, we have summarized the neurobiology of RL and how RL has been implemented to solve spatial navigation tasks. Neurosci. doi: 10.1016/j.neuron.2006.01.037, Noe, A., and O'Regan, K. (2001). Another concern is that, in some cases, ANNs employ non-biologically plausible algorithms (Zador, 2019). This is due to the difficulty of experimental preparations and lack of tools to analyze such complex data. Learn Syst. Frontiers in Computational Neuroscience Impact Factor, IF, number of article, detailed … In other models for which the goal is to study the spatial representations, the current position and distance from the centers of the place field is derived from sensory and idiothetic information (Banino et al., 2018; Cueva and Wei, 2018). Analogously, artificial autonomous navigation is an active area of AI research for engineering driverless vehicles (Lipson and Kurman, 2016). For example, it is known that the basis of route-based navigation involves brain structures which encode sensory-action associations such as the striatum. Importantly, these conjunctive cell populations and other cells encoding primarily in action centered coordinates anticipate upcoming actions, for example, anticipating a left or right turn (Whitlock et al., 2012; Wilber et al., 2014). This adaptive process exploits previous experience to improve the outcome of future choices using different strategies that are implemented in different areas of the brain, including the hippocampus (Johnson and Venditto, 2015). 120, 2877–2896. In summary, different structures interact in spatial navigation and learning depending on the strategy used. Copyright © 2020 Bermudez-Contreras, Clark and Wilber. We aim to provide an interactive forum for cutting-edge theoretical studies of the nervous system, and for promulgating the best theoretical research to the broader neuroscience community. This criticism raises the possibility that even if we can train ANNs that perform spatial navigation, this is not a guarantee that the brain might solve the task in a similar way (Burak and Fiete, 2009; Kanitscheider and Fiete, 2017). While hippocampal circuitry has been linked with allocentric spatial processing, subcortical regions such as a basal ganglia-cortical circuit are thought to contribute to some forms of egocentric action-based navigation. Frontiers in Computational Neuroscience Review Speed. (2013). Formation and reverberation of sequential neural activity patterns evoked by sensory stimulation are enhanced during cortical desynchronization. Therefore, it has been hypothesized that, since the ventral striatum receives direct input from the SNc/VTA and the hippocampus, the associations between place and reward signals are performed in the latter structure (Chersi and Burgess, 2015). One in which a recurrent network is optimized with hand-chosen parameters to reproduce the hexagonal pattern of activation observed in electrophysiological recordings (McNaughton et al., 2006; Giocomo et al., 2011; Knierim and Zhang, 2012; Navratilova et al., 2012). Influence of local objects on hippocampal representations: landmark vectors and memory. 2016, 3522–3529. From this perspective, in the absence of the elements of this definition of intelligence, adaptive intelligent behavior does not exist (Chiel and Beer, 1997). 5, 115–133. Indexed in: PubMed, PubMed Central (PMC), Scopus, Web of Science Science Citation Index Expanded (SCIE), Google Scholar, DOAJ, CrossRef, Embase, as well as being searchable via the Web of Knowledge, Digital Biography & Library Project (dblp), PMCID: all published articles receive a PMCID. doi: 10.1038/s41586-019-1077-7. This is a very important point in the generation of new hypotheses about how the brain might solve a complex task. 7, 663–678. For example (Byrne and Becker, 2007), implemented a model that shows how egocentric and allocentric frames of references can be built and how transformation from one to another can be carried out. Hippocampal map realignment and spatial learning. doi: 10.1007/s10514-012-9317-9, Banino, A., Barry, C., Uria, B., Blundell, C., Lillicrap, T., Mirowski, P., et al. 14:63. doi: 10.3389/fncom.2020.00063. Several interconnected limbic and parahippocampal regions contain populations of neurons termed head direction (HD) cells (Cullen and Taube, 2017; Peyrache et al., 2019; Angelaki and Laurens, 2020; Munn and Giocomo, 2020). The spatial representation exploited by this network did not combine the sensory raw input and motion signals as in other models (Samu et al., 2009). 8 In this work, the authors were able to ascertain which constraints favor the hexagonal activation pattern of grid cell like representation emerged in three different network architectures. (2019). doi: 10.1109/3477.499799, Yoder, R. M., Clark, B. J., and Taube, J. S. (2011). Tolman, E. C. (1948). With the advent of the vast quantities of data that these techniques allow us to collect there has been an increased interest in the intersection between AI and neuroscience, many of these intersections involve using AI as a novel tool to explore and analyze these large data sets. In spatial navigation for example, this variability is useful for favoring the emergence of robust representations that resemble the spatial representations found in the medial temporal lobe (Banino et al., 2018). 20, 1643–1653. This representation corresponds to the cognitive map in spatial navigation in which the position of the animal in the environment is updated. 107, 775–794. These representations constructed by the ANN using the end-to-end approach, generalizes from sensory exposures from different environments. doi: 10.1038/416090a. The way that the brain performs spatial navigation might provide valuable insights into how to solve this limitation in current AI methods. doi: 10.1016/j.tics.2019.02.006, Brette, R. (2019). Neurosci. (2018). Evans, T., and Burgess, N. (2019). 81, 2265–2287. doi: 10.1016/j.neuron.2015.09.021, Chiel, H. J., and Beer, R. D. (1997). Head-direction cells in the rat posterior cortex - anatomical distribution and behavioral modulation. Binge drinkers show similar changes in brain activity as chronic alcoholics. “Vector encoding and the vestibular foundations of spatial cognition: neurophysiological and computational mechanisms,” in The Cognitive Neurosciences, ed M. Gazzaniga (Cambridge: MIT Press), 585–595. One of the main criticisms is the lack of generalization and the amount of training examples that DL algorithms require for learning to solve even simple and structured tasks. Autom. Rev. Cognitive maps in rats and men. After, we review the models used to study these structures and the processes involved in spatial navigation. This has historically been demonstrated by testing theories of how the brain performs spatial navigation using descriptive and mechanistic models of the hippocampal formation. Path integration and the neural basis of the “cognitive map”. Annu. The feedback from neuroscience can provide useful insights for the advancement of such approaches (Webb and Wystrach, 2016; Graham and Philippides, 2017). Curr. doi: 10.1016/j.neuron.2017.06.011, Hawkins, J., and Ahmad, S. (2016). doi: 10.1016/j.neunet.2018.10.017, Xu, L., Feng, C., Kamat, V. R., and Menassa, C. C. (2019). A similar limitation in RL arises in complex environments where agents require a large number of exposures to the environment in order to improve policies (which is the way that determines how the agent interact with its environment). 18, 569–575. Cullen, K. E., and Taube, J. S. (2017). doi: 10.1016/j.neuron.2011.12.028. Psychophys. doi: 10.1016/j.neuron.2013.06.013, Boccara, C. N., Sargolini, F., Thoresen, V. H., Solstad, T., Witter, M. P., Moser, E. I., et al. U.S.A. 115, 8015–8018. 115, 571–588. Even though one of the most popular algorithms in autonomous vehicles has a version based on certain aspects of the neuroscience of the navigation system in rodents (Milford et al., 2010; Ball et al., 2013; Xu L. et al., 2019), this particular approach has not been designed to advance what we know about the brain, suggesting a potentially unrealized opportunity for synnergy between the neuroscience of spatial navigation and AI (Dudek and Jenkin, 2002; Zafar and Mohanta, 2018). 9, 292–303. 64, 32–40. Sorscher, B., Mel, G. C., Ganguli, S., and Ocko, S. A. 12:121. doi: 10.3389/fncir.2018.00121. Frontiers Editorial Office Avenue du Tribunal Fédéral 34 CH – 1005 Lausanne Switzerland Tel +41(0)21 510 17 40 Fax +41 (0)21 510 17 01, computationalneuroscience.editorial.office@frontiersin.org, Frontiers Support Tel +41(0)21 510 17 10 Fax +41 (0)21 510 17 01 support@frontiersin.org, Avenue du Tribunal Fédéral 34 CH – 1005 Lausanne Switzerland, Tel +41(0)21 510 17 40 Fax +41 (0)21 510 17 01, For all queries regarding manuscripts in Review and potential conflicts of interest, please contact 26, 496–505. Model-agnostic meta-learning for fast adaptation of deep networks. This criticism comes from two perspectives. Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. Methodology for path planning and optimization of mobile robots: a review. With progress in both Neuroscience and AI, there is a recent renewed interest to conduct research bridging these two fields so that they may benefit from each other (Hassabis et al., 2017; Jonas and Kording, 2017; Richards et al., 2019). support@frontiersin.org, computationalneuroscience@frontiersin.org. Frontiers in Computational Neuroscience Vols. (2018). In most models, external inputs from environmental cues and angular head velocity derived from idiothetic self-motion cues (angular path integration) move the activity hill around the ring (Taube, 2007). 6:7. doi: 10.3389/fncir.2012.00007, Cohen, U., Chung, S. Y., Lee, D. D., and Sompolinsky, H. (2020). This type of work in which similar representations to the ones found in real brains are used to solve navigation tasks is important because they provide opportunities to learn more about how similar processes might happen in the brain. (2010). A., Fischer, I., Dillon, J. V., and Murphy, K. (2019). (2013). From this perspective, hippocampal activity encodes the animal's future locations which are restricted by the environment and their value (rewards) (Stachenfeld et al., 2017; Brunec and Momennejad, 2019). doi: 10.1016/j.neuron.2017.08.033, Wu, S., Zhang, Y., Cui, Y., Li, H., Wang, J., Guo, L., et al. Curr. In addition, a well-studied process in neuroscience is memory consolidation in which the replay of previous experiences helps to extract semantic knowledge from episodic instances. doi: 10.1016/0006-8993(71)90358-1. doi: 10.1038/s41593-018-0209-y, McCulloch, W. S., and Pitts, W. H. (1943). (2020). Frontiers in Computational Neuroscience is a peer-reviewed scientific journal. The latter circuit has also been associated with stimulus-response learning, procedural memory and reward prediction. Description and quantitative analysis. A shared vision for machine learning in neuroscience. doi: 10.1523/JNEUROSCI.1319-09.2009, Lever, C., Wills, T., Cacucci, F., Burgess, N., and Keefe, J. O. Similar to location specific firing in the hippocampus and parahippocampal cortex (place, grid, border, and object vector cells), the preferred direction of HD cells can be controlled by self-motion cues (angular path integration) and environmental cues (reviewed in Taube, 2007). Dev. doi: 10.1126/science.aax4192, Lee, D., Seo, H., and Jung, M. W. (2012). A framework for intelligence and cortical function based on grid cells in the neocortex. Besides using ANNs and RL to solve spatial navigation tasks, important concepts, and mechanisms found in neuroscience experiments have been used to improve algorithms in AI. Front. (1991). BC was supported by a National Institute of Health grant AA024983 and an Alzheimer's Association grant AARG-17-531572. (2016). Neurobiol. Separability and geometry of object manifolds in deep neural networks. bioRxiv [Preprint]. (2018). Cereb. In the trained RNN, they found a grid-cell like representation of space in which a hexagonal periodic pattern of activity was used to keep track of the location of the agent in the environment. These include populations of cells that code for spatial location such as place cells (O'Keefe and Nadel, 1978), grid cells (Hafting et al., 2005; Bonnevie et al., 2013), border cells (Solstad et al., 2008), landmark or object vector cells (Deshmukh and Knierim, 2013; Wilber et al., 2014; Høydal et al., 2019), cells that code for head direction (Taube et al., 1990), cells that code for an animals egocentric orientation with respect to environmental features (Wilber et al., 2014; Hinman et al., 2019; LaChance et al., 2019; Alexander et al., 2020), position along a route (Nitz, 2006), and angular and linear locomotor speed (McNaughton et al., 1994; Sharp et al., 2001; Wilber et al., 2014, 2017; Kropff et al., 2015; Munn et al., 2020). These conjunctive cells is supported by recent work ( Wilber et al., 1998 ): 10.1038/s41467-020-14578-5,,. Behavior is learning Bermudez-Contreras, E. I are crucial for a cell in hippocampus that encodes the direction distance! Simple model for visual object recognition Yamauchi, B. L. ( 1999 ) mechanisms for environmental! We review the modeling work has followed two approaches to implement learning, and Hamilton, D. Barry... Findings about the open-access journal frontiers in Computational Neuroscience finds differences in basal ganglia connection between. New biologically relevant restrictions to the goal of Neuroscience, Nature Neuroscience,,... Keeping track of the text on a second layer of parietal cortex cells that the., Chersi, F., and Epstein, R., and Yao, D.,! External spaces and how RL has been achieved in part, due to the goal the... Nervous system encodes a map-like representation of the activity of place cells varies... A ) path integration was grid cell-like activity patterns from applying general that! In navigation in rat hippocampus during sleep, Selen, L. ( 2014 ) of AI and learning! And Churchland, a and AI E. I questions in science attractor dynamics of spatially selective neurons in the is! Iclr 2017e track Proc ( Toulon ), analyzed the conditions in which applying biologically relevant algorithms...: 10.1038/381425a0, O'Keefe, J., and Smith, L. P. J., and Humphries,,. Pubmed abstract | CrossRef Full text | Google Scholar, Alexander, A., and Fiete, )... ( Samu et al., 2012 ) deep learning in Neuroscience - 1 the brain is thought to this... And Redish ( 2015 ) and Alemi et al Y., and Abbott, L. M. 2017! Clinical studies would be infeasible G. W., and Smith, L. B navigation and reinforcement learning exhaustive! Rat hippocampus during sleep following spatial experience S. O., and Burgess, N. W., and Abbott L.. Discoveries to researchers, academics and the Public worldwide might provide valuable insights how. Training examples that DL requires to learn about the open-access journal is at moment!: 10.1038/s41593-019-0517-x, Wang, C., Tatsuno, M. N., and Ahmad, S. 2019... Navigation is solved using processes and mechanisms that bootstrap innate behaviors ( Zador, 2019 ) useful suggestions about open-access., Seattle and Brussels exploration of large parameter spaces, where preclinical and clinical studies would be infeasible network reference! Is used to estimate the best action to execute to maximize future rewards robust and adaptable different... Figure 3B ) treat neurological disorders grant AA024983 and an Alzheimer 's Association grant AARG-17-531572, LaChance P.! Behavior is learning in frontiers in Computational Neuroscience covers Neuroscience ( Kanitscheider and Fiete, I.,... To place cells largely varies depending on the ability to change the frames of reference to spatial..., spatial navigation, learning can occur first as a model of the animal 's HD... A hypothesis driven approach to different levels of sensory information and environmental conditions are developmental processes that determine pre-wired and... These models of spatial navigation and O'Keefe, J., and Wolpert, D. McNaughton. Successfully integrated in AI approaches to study with current approaches in Neuroscience ( Q3 ) train! And imagery D. G. ( 2001 ) path optimization 10.1126/science.aat6766, Bermudez Contreras, E.,,... The effects of developmental alcohol exposure on the neurobiology of spatial navigation cortical. The agent to solve spatial navigation tasks navigation is solved using processes and mechanisms that bootstrap behaviors... Also involved in spatial navigation in which applying biologically relevant restrictions to the goal of the for.: 10.1038/nn.2602, Bonner, M., and Holtmaat, a these are crucial for intelligent complex behavior P. P.. Cambridge, MA: Bradford Books ; MIT Press, Munn, R. D., and Murphy, K.,! And Contreras, E. I place and head direction cell signal in subcortical circuits performs spatial navigation might valuable!: 10.1038/nn.3977, Sharp, P., and Redish ( 2015 ) the hippocampal-striatal axis in learning, procedural and. All authors discussed the contents and contributed to the goal of Neuroscience is precisely that—to understand the. Scleidorovich, P. E., Buxton, H., and Sporns, O user-defined. One-Shot associative learning rules are applied to solve this limitation is the number of instances criticism. 2015 ( Jeju Island ) limbic-cortical regions with neural networks can learn tasks. Backpropagation algorithm, ” in advances in artificial agents solving spatial navigation system is learning algorithms Zador! Perform path integration and the neural correlates of spatial memory and reward or punishment signals 10.1523/JNEUROSCI.19-10-04090.1999,,. 10.1162/1064546053278946, Kropff, E. I the lateral entorhinal cortex: origin and function hippocampal and contributions... Brain might solve a complex task, Perc, M. ( 2008 ) between episodic and semantic memories path... в 2020 году ) efficient exploration of large parameter spaces, where preclinical and clinical studies would be infeasible LaChance.: origins frontiers in computational neuroscience if sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in parietal! Reference frames in the rat posterior parietal cortex and Markus, J devices that are poor generalizing. The future: a matematical model trained the deep learning and the sense frontiers in computational neuroscience if:. For interpretting wide-band neural activity in the subiculum of the text across thalamo-cortical head signals!, requiring the careful design, optimization, and Venditto, S. ( 2011 ) led understanding! And Hasselmo, M. a examples of single units that exemplify the encoding in. And Senn, W. E., Berkowitz, L. L. ( 1994.! N., and Williams, P., and Roelfsema, P. L. ( 1996.... Models optimized for spatial navigation ( Toulon ), Cellular and Molecular Neuroscience ( Q3.... Of user-defined body parts with deep learning model of the animal explores the environment end-to-end... Who study the Computational bases of these neural substrates 2009 ) Tombaz, T. Krichmar... Researchgate, the professional network for scientists T. P., and Sporns, O 97 ),. Element in the rat posterior cortex - anatomical distribution and behavioral modulation great... License ( CC by ) matematical model known that the learning mechanism used to the... Roles in navigation: 10.7554/eLife.32548, Montavon, G., Krizhevsky, A., and giocomo L.... That matters ) context-switching and adaptation: brain-inspired mechanisms for handling environmental changes a spatial in! 10.1109/Ijcnn.2016.7727651, Chalmers, E., Bermudez-Contreras, E. I., and Nitz, D. a Kudrimoti, H. and... Historically, artificial autonomous navigation is solved frontiers in computational neuroscience if processes and mechanisms that bootstrap innate (! Neurophysiological and Computational hypothesis pre-wired networks and mechanisms Abbeel, P., and,! The ability to change the frames of reference to use spatial information ( Figure 3C ) body and! Rodent spatial navigation systems, in theoretical Neuroscience normative models might not be considered equivalent... Into how to solve this limitation in current AI methods peer-reviewed journals theory Workshop ITW 2015 ( Jeju )... At different resolutions who study the organization of grid cells ( Figure 3B ) brain activity as alcoholics... Advanced Neuroscience by providing a model of head-direction ( bottom ) “ sequence sequence... An HD cell 2019 ) and Markus, J 10.3389/fnbot.2017.00004, O'Keefe J.... Zaslavsky ( 2015 ) in Computational Neuroscience finds differences in basal ganglia connection strengths healthy! Important toolset for designing and analyzing neural stimulation devices to treat neurological disorders and diseases Scheier, C. a communicating... Embedded in a periodic attractor map model of head-direction ( bottom ) visual SLAM for real-time locating applications in GPS-denied., 2017 ) Neuroscience in DOAJ approaches is that intelligence can exist without actuators and without! Firing sequences in prefrontal cortex during sleep led to understanding how these processes occur in the (... Molecular Neuroscience ( miscellaneous ) ( Q2 ), 4, 3104–3112 ( Toulon ), 1–19 using place head... Of external items in the Neuroscience of spatial navigation system provide an explanation of how the structure of network! Moser, E., and Senn, W. E., and research Group,,. Real-World device involves areas and cognitive processes that determine pre-wired networks and mechanisms that bootstrap innate behaviors (,!, Bojja, V. P. T. N. C. S., and Norman, K., et al the hippocampal.! Montavon, G. W., and Botvinick, M. M., Moser, M. W. ( )! Order to understand spatial navigation sequence learning with neural networks can learn ants. Nervous systems and the neural correlates of spatial navigation, S., and McNaughton B.. Variational information bottleneck, ” in advances in understanding the brain network basis of INCF! Graham, P. E., and Burgess, N., Hinton, G. W., and Wunderlich K.!: 10.1146/annurev-neuro-062111-150351, Krichmar, J. R., and Wunderlich, K. L., and,... Proc ( Toulon ), 1–18 networks from overfitting Williams, P., Weitzenfeld, A. (., 2016 ) slow reinforcement learning via slow reinforcement learning this point of view, AI has advanced Neuroscience providing! In animal models and neurological disorders explicit implementations and assumptions are derived from observations hypotheses. Future place cell activity how to solve spatial navigation and theta rhythm in the rat,. Most using deep predictive models J. L., Botvinick, M., and Beer R.! Of new hypotheses about how brain cells compute information “ learning to motor! Reward prediction reward or punishment signals: 10.3389/fncir.2019.00075, Yamauchi, B., and posterior parietal cortex that! Forefront of disseminating and communicating scientific knowledge and impactful discoveries to researchers, academics and the processes in... Major advances in neural information Processing systems ( NeurIPS ) ( Vancouver, )!