Self-Communicating Deep Reinforcement Learning Agents Develop External Number Representations

Abstract

Symbolic numbers are a remarkable product ofhuman cultural development. The developmentalprocess involved the creation and progressive re-finement of material representational tools, suchas notched tallies, knotted strings, and countingboards. In this paper, we introduce a computa-tional framework that allows the investigation ofhow material representations might support num-ber processing in a deep reinforcement learning sce-nario. In this framework, agents can use an exter-nal, discrete state to communicate information tosolve a simple numerical estimation task. We findthat different perceptual and processing constraintsresult in different emergent representations, whosespecific characteristics can facilitate the learningand communication of numbers.