Johor, despite being “longest and most intimately associated with the British”, was the last of the Malay states to bow to pressure to accept a British adviser with wide-ranging political powers (Emerson 1937/70, p. 198). It was in fact only on 12 May 1914, on the very eve of World War I in Europe, that Sultan Ibrahim signed an agreement with the British allowing the General Adviser — a position created in Johor only four years earlier — to have the same powers as those enjoyed by British Residents in the rest of the Malay states. This fact testifies to the slow pace of colonization that the state of Johor went through during the nineteenth century, and to the late realization of direct British power there. Sultan Ibrahim's father, the energetic Sultan Abu Bakar, had encouraged migrant workers to develop estates along Johor's many rivers, and had thus directed economic growth towards commercial agriculture (Winstedt 2003, pp. 134–35). He was also responsible for turning Johor into a constitutional monarchy in April 1895. Apparently, his budgetary sense was not the best. When he died, his son inherited “a working regime, a written constitution and an empty Treasury”. What Johor managed to develop at the edge of British influence and pressure was a Malay bureaucracy “offering careers to local men which were much esteemed, both in terms of financial reward and high status in community” (Gullick 1992, pp. 110, 118). The relative independence of the royal family held great significance for the political thinking of Johor's elite, for the educational paths its upper class tended to take, and for the close ties that came to exist between the important families and the ethnic groups involved, and indeed between Johor and Singapore (Winstedt 2003, pp. 134–35).

Ismail was born within this setting on 4 November 1915 into an already illustrious family. His father was Abdul Rahman bin Yassin, “a stoutly built and dark man, almost morose in demeanour” (Robert Kuok, interview 10 February 2006). Abdul Rahman's sister, Anima, had died young, leaving him the only surviving child of Mohamed Yassin bin Ahat, a government officer and son of Orang Kaya Ahat of Padang Muar. Mohamed Yassin's wife was a daughter of Chinese convert Haji Mohamed Salleh bin Abdullah from Singapore, who was Johor State Treasurer, and his Malay wife from Mersing.