The impact toughness and related microstructure of the dissimilar 9Cr/NiCrMoV welded joint fabricated by narrow-gap submerged arc welding were systematically investigated in the paper. Results indicated that the fracture appearance transition temperature (50% FATT) for weld metal was −11 °C, while low and scattered absorbed energies determined by different crack growth paths for the heat affected zone of 9Cr were gained which could not satisfy the requirement of service. However, a dramatically enhanced impact toughness was obtained by optimizing the post-weld heat treatment (PWHT) process. Microstructure characterization revealed that the microstructure evolution from martensitic laths in the previous PWHT metals to a softer ferrite matrix with the supersaturated carbon precipitating from the matrix led to higher toughness in the optimized PWHT materials. In addition, the observation of the fracture morphology found that the fractography varied from brittle fracture to a fracture mode with both brittle and ductile fracture feature with the change of crack growth paths in 9Cr-HAZ.