The Horn of Africa is the most conflict-ridden region in the African continent. Both inter-and intra-state conflicts have dominated the region. In a bid to check intra-state conflicts and accommodate ethno-national and religious diversity, federal or federal like models of governance have been proposed, discussed, and, in some cases, adopted across the region. Focusing on Ethiopia, Somalia, Sudan and South Sudan, this article discusses the origin, reasons, and prospects of the federal idea in the Horn. The article argues that the major rationale for the federal idea in the Horn is the containment of communal tensions. Yet, the track record of federalism in alleviating communal tensions has not been encouraging. This is partly related to design issues that have undermined the efforts to use federalism to address communal tensions. More importantly, however, the commitment to genuinely implement the federal idea has largely been absent.