We report on follow-up observations of 20 short-duration gamma-ray bursts(T90 < 2s) performed ing′r′i′z′JHKswith the Gamma-Ray Burst Optical Near-Infrared Detector (GROND) between mid-2007 and theend of 2010. This is the most homogeneous and comprehensive data set on GRB afterglowobservations of short bursts. In three cases, GROND was on target within less than 10 minafter the trigger, leading to the discovery of the afterglow of GRB 081226A and its faintunderlying host galaxy. In addition, GROND was able to image the optical afterglow andfollow the light curve evolution in five further cases: GRBs 090305, 090426, 090510,090927, and 100117A. Three of the aforementioned six bursts with optical light curves showa break: GRBs 090426 and 090510 as well as GRB 090305. For GRB 090927, no break is seen inthe optical/X-ray light curve until about 150 ks/600 ks after the burst. A decay slope ofthe optical afterglow of GRB 100117A could be measured. Using these data supplemented byabout ten events taken from the literature, we compare the jet half-opening angles of longand short bursts. We find a tentative evidence that short bursts have wider opening anglesthan long bursts. However, the statistics are still very poor and follow-up observationsof these events are therefore very important to gain as much observational data aspossible.