Jan Pieter Hendrik van Gilse was born in Rotterdam on 11 May 1881, the youngest son of Jan Albert van Gilse and Maria Auguste Hockelmann. He had two brothers and two sisters. His father, a lawyer and journalist, was involved in political activism and from 1897 to 1901 was a member of the Liberal Union government.

From an early age, van Gilse knew that he wanted to become a conductor and composer and at sixteen he enrolled at the Conservatoire in Cologne. His teachers were Franz Wüllner for conducting and composition, and Max van de Sandt2 for piano. He had his first successes while still a student, with his Concert Overture (1900) and First Symphony (1901). He continued his studies for some years with Engelbert Humperdinck in Berlin.

In 1905, van Gilse was given his first job, as the rehearsal conductor at the Stadt Theater in Bremen. In the same year he completed and conducted the performances of his Third Symphony, Erhebung, in Bremen and Amsterdam (his Second Symphony had been completed in 1903). The composition received a prestigious German music award, enabling him to work in Rome for two years, but he did not enjoy his time there, feeling too homesick to work. Much to his relief, by May 1911 he and his wealthy wife, Ada Hooijer, had settled in Munich, where their sons were born, Janric in 1912 and Maarten in 1916. In 1913, he wrote Frau Helga von Stavern, an opera which has still never been performed, and completed his song-cycle (with orchestra) to texts from Rabindranath Tagore's Gitanjali in 1915.

Conditions in Munich were deteriorating by the day because of the First World War, and so the family returned to the Netherlands. Now van Gilse could not find employment to his liking. In the past, he had applied for conducting posts with the Residentie Orchestra in The Hague and the Concertgebouw Orchestra in Amsterdam, but he was rejected. A job application in Arnhem was also unsuccessful. Finally, in 1917 he was appointed as a conductor at the Utrecht Municipal Orchestra, replacing its conductor, Wouter Hutschenruyter, who after decades of dedicated service, became the director of the music school in Rotterdam.