This paper proposes a novel control framework for a single-master/multi-slave teleoperation system to grasp and handle an object, considering nonlinearity and uncertainty in the dynamics of the slaves and time-varying delay in the communication channel. Based on passive decomposition approach, the dynamics of the slaves are decomposed into two decoupled systems, and then two higher order sliding mode controllers are designed to control them. Also, a synchronization control methodology for the master is developed. Stability is fully studied using the passivity property and Lyapunov theorem. Finally, simulation and practical results confirm that the control system works well against the conditions.