Sign languages like British Sign Language (BSL) include partially lexicalized constructions depicting object handling or manipulation – handling constructions. Object sizes gradiently vary, yet it is unclear whether handling handshapes depict handled objects categorically or gradiently. This study investigates the influence of sign language experience on perception of handling handshapes. Deaf signers and hearing non-signers completed perceptual handshape identification and discrimination tasks. We examined whether deaf BSL signers perceived handshape continua categorically or continuously compared with hearing non-signers, and whether reaction times were modulated by linguistic representations. The results revealed similar binary categorization of dynamically presented handling handshapes as deaf and hearing perceivers displayed higher discrimination accuracy on category boundaries, and lower, but above chance, within-category discrimination, suggesting that perceptual categorization was not uniquely mediated by linguistic experience. However, RTs revealed critical differences between groups in processing times; deaf signers’ RTs reflected stronger category bias and increased sensitivity to boundaries, suggesting underlying linguistic representations. Further, handshape variability within categories influenced deaf signers’ discrimination RTs in a manner that suggested graded category organization, with handshape prototype grounding the category. These findings provide an insight into the internal organization of handling handshapes and highlight the complex relationship between sign language, cognition, and gesture.