A fundamental feature of the human brain is its capacity to learn novel motor skills. This capacity requires the formation of vastly different visuomotor mappings. In this work, we ask how these associations are formed de novo, hypothesizing that under specific training regimes generalizable mappings are more readily formed, while in others, local state-actions associations are favored. To test this, we studied learning in a simple navigation task where participants attempted to move a cursor between various start-target locations by pressing three keyboard keys. Importantly, the mapping between the keys and the direction of cursor movement was unknown to the participants. Experiments 1 and 2 show that participants who were trained to move between multiple start-target pairs had significantly greater generalization than participants trained to move between a single pair. Whereas Experiment 1 found significant generalization when start-targets were distal, Experiment 2 found similar generalization for proximal targets, which suggests that generalization differences are due to knowledge of the visuomotor mapping itself and not simply due to planning. To gain insight into the potential computational mechanisms underlying this capacity, we explored how a visuomotor mapping could be formed through a set of models that afforded construction of a generalizable mappings (model-based), local state-action associations (model-free), or a hybrid of both. Our modeling work suggested that without continued variability between start-target pairs during training, model-based processes eventually gave way to model-free processes. In Experiment 3, we sought to further test this shift in learning processes by exposing participants to initially high variability before settling into a condition of no variability over a long-period of training. We found that generalization performance remained intact after a prolonged period of no variability suggesting that the formation of visuomotor mappings might occur at an early stage of learning. Finally, in Experiment 4 we show that adding stochasticity in the mapping can also promote model-based learning of a visuomotor mapping, suggesting that the learning may unfold implicitly. Overall, these studies shed light on how humans could acquire visuomotor mappings in their lives through exposure to variability in their feedback.