Henry James was born on 15 April 1843 at 21 Washington Place, New York, of Irish and Scots-Irish descent. His grandfather William James had been an immigrant, self-made man and multi-millionaire; his father, Henry James senior, was an unworldly amateur philosopher and Swedenborgian whose ever-changing educational theories led to the young James and his brothers and sister spending their formative years in Europe. The family returned to America in 1858.

When the Civil War broke out in 1861, the question of whether he should volunteer was settled for James when he injured his back while helping to put out a fire, an injury he suffered from intermittently for much of the rest of his life. His two younger brothers, Wilky and Robertson, fought on the Union side, though his elder brother William, later to become the famous philosopher and psychologist, went to study at Lawrence Scientific School at Harvard instead.

In 1869 James returned to Europe and spent a year travelling in England, France and Italy. It was while he was abroad that he heard of the death from tuberculosis of his young cousin Minny Temple, whose fate was later to be the inspiration for two of his greatest novels, The Portrait of a Lady and The Wings of the Dove. Whether or not he was in love with Minny, James certainly loved her – he was to write many years later that her death felt like the end of youth both for his brother William and himself.