Dissimilar joints of advanced 9Cr/CrMoV have been successfully welded by narrow gap submerged arc welding using multi-layer and multi-pass techniques. The objective of our study is to establish the correlation between impact toughness and microstructural characteristics of the welded joints. Impact toughness tests were conducted in a wide range of temperature from −60 °C to 80 °C for different regions in the dissimilar joints. The fracture appearance transition temperature of base metal of 9Cr, CrMoV and weld metal were tested as 23 °C, −9 °C and −2 °C respectively, which all satisfied the service requirement. Optical microscope and scanning electron microscope revealed that weld metal and base metal of CrMoV comprised martensite and bainite while 9Cr was composed of lath martensite. The low toughness in the latter region arose from large grains with excessive carbide precipitates. Nonuniform microstructure in the heat-affected zone of 9Cr side caused different crack propagation paths and subsequently led to large variations of absorbed energy. When crack propagates along carbon-enriched zone in heat affected zone, the absorbed energy was 48 J. With crack deviating far from carbon-enriched zone, the absorbed energy increased to 147 J. Examination on fracture surfaces revealed the typical brittle fracture appearance in 9Cr and inter-granular fracture mode in heat-affected zone of 9Cr side when crack propagated along carbon-enriched zone.