The family from which Sir Thomas Fowell Buxton was descended, resided, about the middle of the 16th century, at Sudbury in Suffolk, and subsequently at Coggeshall in Essex. At the latter place, William Buxton, his lineal ancestor, died in 1624. Thomas, the son of William Buxton, claimed and received from the Heralds' College, in 1634, the arms borne by the family of the same name, settled before 1478 at Tybenham in Norfolk, and now represented by Sir Robert Buxton, Bart.

Isaac Buxton, a merchant, and the fifth in direct descent from William, married Sarah Fowell, an heiress; connected with the family of the Fowells, of Fowels-combe in Devonshire. From her was derived the name of Fowell, first borne by her eldest son, who married Anna, daughter of Osgood Hanbury, Esquire, of Holfield Grange in Essex. The first Thomas Fowell Buxton lived at Earl's Colne in the same county, but was residing at Castle Hedingham when his eldest son, Thomas Fowell, the subject of this memoir, was born, on the first of April, 1786.

Mr. Buxton was a man of a gentle and kindly disposition, devoted to field sports, and highly popular in his neighbourhood, where he exercised hospitality on a liberal scale. Having been appointed High Sheriff of the county, he availed himself of the authority of his office to relieve the miseries of the prisoners under his superintendence, visiting them sedulously, notwithstanding the prevalence of the jail fever. He died at Earl's Colne in 1792, leaving his widow with three sons and two daughters.