We will introduce a regularization for $p$-adic multiple zeta values and show that the generalized double shuffle relations hold. This settles a question raised by Deligne, given as a project in the Arizona Winter School 2002. Our approach is to use the theory of Coleman functions on the moduli space of genus zero curves with marked points and its compactification. The main ingredients are the analytic continuation of Coleman functions to the normal bundle of divisors at infinity and definition of a special tangential base point on the moduli space.
