Transit observations indicate a large dispersion in the internal structure among the known gas giants. This is a big challenge to the conventional sequential planetary formation scenario because the diversity is inconsistent with the expectation of some well defined critical condition for the onset of gas accretion in this scenario. We suggest that giant impacts may lead to the merger of planets or the accretion of planetary embryos and cause the diversity of the core mass. By using an SPH scheme, we show that direct parabolic collisions generally lead to the total coalescence of impinging gas giants whereas, during glancing collisions, the efficiency of core retention is much larger than that of the envelope. We also examine the adjustment of the gaseous envelope with a 1D Lagrangian hydrodynamic scheme. In the proximity of their host stars, the expansion of the planets' envelopes, shortly after sufficiently catastrophic impacts, can lead to a substantial loss of gas through Roche-lobe overflow. We are going to examine the possibility that the accretion of several Earth-mass objects can significantly enlarge the planets' photosphere and elevate the tidal dissipation rate over the time scale of 100 Myr.