The pattern of generalization following motor learning can provide a probe on the neural mechanisms underlying learning. For example, the breadth of generalization to untrained regions of space after visuomotor adaptation to targets in a restricted region of space has been attributed to the directional tuning properties of neurons in the motor system. Building on this idea, the effect of different types of perturbations on generalization (e.g., rotation vs. visual translation) have been attributed to the selection of differentially tuned populations. Overlooked in this discussion is consideration of how the context of the training environment may constrain generalization. Here, we explore the role of context by having participants learn a visuomotor rotation or a translational shift in two different contexts, one in which the array of targets were presented in a circular arrangement and the other in which they were presented in a rectilinear arrangement. The perturbation and environments were either consistent (e.g., rotation with circular arrangement) or inconsistent (e.g., rotation with rectilinear arrangement). The pattern of generalization across the workspace was much more dependent on the context of the environment than on the perturbation, with broad generalization for the rectilinear arrangement for both types of perturbations. Moreover, the generalization pattern for this context was evident, even when the perturbation was introduced in a gradual manner, precluding the use of an explicit strategy. We describe how current models of generalization might be modified to incorporate these results, building on the idea that context provides a strong bias for how the motor system infers the nature of the visuomotor perturbation and, in turn, how this information influences the pattern of generalization.