4H-silicon carbide (SiC) wafers were annealed at 1300 and 1600 °C for 30 min and 60 min in a conventional and purified Ar atmosphere. The surface roughness before and after annealing was evaluated by atomic force microscopy. The surface roughness before annealing was approximately 2.37 nm in root mean square. The roughness, after annealing for 30 min at 1300 and 1600 °C in a conventional Ar furnace, was increased to 4.53 and 14.9 nm, respectively. The roughness, after annealing for 60 min, was 5.01 and 19.1 nm, respectively. In this study, the G3 grade Ar gas (99.999%) was supplied in the conventional furnace tube. When the Ar gas was purified to an impurity concentration of less than 1 ppb, and it was supplied in the leak-tight furnace tube, the roughness after 30-min annealing improved 4.27 and 6.93 nm at 1300 and 1600 °C, respectively. The roughness after 60-min annealing was also reduced to 3.54 and 9.28 nm, respectively. We assume that a significant reduction of H2O concentration in the annealing atmosphere might play an important role in suppressing surface roughening of SiC during high-temperature annealing.