We develop an unconditionally energy stable immersed boundary method, and apply it to simulate 2D vesicle dynamics. We adopt a semi-implicit boundary forcing approach, where the stretching factor used in the forcing term can be computed from the derived evolutional equation. By using the projection method to solve the fluid equations, the pressure is decoupled and we have a symmetric positive definite system that can be solved efficiently. The method can be shown to be unconditionally stable, in the sense that the total energy is decreasing. A resulting modification benefits from this improved numerical stability, as the time step size can be significantly increased (the severe time step restriction in an explicit boundary forcing scheme is avoided). As an application, we use our scheme to simulate vesicle dynamics in Navier-Stokes flow.