According to the Body-Specificity Hypothesis, humans preferentially associate positive features with their dominant side with which they interact more fluently, and negatives features with their non-dominant side with which they act more clumsily. The current research investigated implicit space-valence mappings in two clinical populations, namely, patients with unilateral stroke and individuals who suffered the loss of a limb. Across the two studies, our findings offered general support for the Body-Specificity Hypothesis with important caveats. Specifically, the body-specificity effect was more detectable in the two groups of right-handed individuals with motor fluency impairment on their left side than healthy controls. This is possibly because the former can gain more fluent experience with their dominant hand since they exclusively use the intact hand for self-sufficiency, which may increase their bias toward “right-is-good” pattern. Taken together, the results provided converging evidence that individual differences in action fluency may moderate the body-specificity effect.