When, at the end of almost three months of agitation over the Fugitive Slave Law, it appeared that the country was no closer to reconciliation, Elisha Whittlesey, the first comptroller of the Treasury in the Fillmore administration and the person responsible for covering the cost of the rendition of fugitive slaves, suggested a possible solution to what seemed like an increasingly intractable problem. He knew of what he spoke. His was a conservative Whig solution. It included a declaration of opposition to slavery; a condemnation of abolitionist agitation; recognition that, in the end, slavery's future depended entirely on those who owned slaves; and, finally, the removal of free blacks and freed slaves from the country. Although he was opposed to slavery, it answered no “good purpose,” as he quaintly put it to a friend, to interfere in “the domestic relations” of slaveholders. There were thousands, if not “tens of thousands” of slave owners, he estimated, who would “liberate their slaves” if people in the Free States would cease “interfering with the institution of slavery.” The fact that abolitionists did little to help escaping slaves to settle permanently among them struck Whittlesey as utterly perverse. Under the circumstances, the only option available, the one that would ensure the ultimate emancipation of the slaves and the peace of the country, was to cease agitation, allow slaveholders the time and freedom to do with their slaves as they saw fit and, when they did decide on a policy of manumission, provide them with a place where they could send the emancipated. The obvious location was Africa and the colony of Liberia, now thirty years old, a place where, because of its climate, whites could not live, a place set aside by nature and Providence for the black man. The void created by the removal of former slaves would be filled, he predicted, by the starving “lower classes” of Europe who would be invited to come to America as laborers. In that way, over time, Whittlesey reasoned, America would rid itself of slavery and become both prosperous and wholly white.

At the time, Whittlesey was riding a wave of resurgent colonizationism, which peaked in the years after Liberian independence in 1847 and the Compromise of 1850, with its promise of sectional reconciliation.