Many Victorian clergy left a lasting built legacy in their parishes, changing the appearance and layout of their churches and building new rectories and schools. In this Armstrong was no exception. In his first couple of years in Dereham he visited the restored neighbouring churches in Yaxham, Billingford, Necton, Reymerstone and Cranworth. He was able to write, ‘How wonderful is the move towards restoration. Not long hence the unrestored church will be an exception’, and to comment, ‘Those not renovated just make me feel sad’ (31/7/52).

The maintenance of their churches was a duty taken seriously by the majority of clergymen, but their perception of what this should involve changed over the period in question. As antiquarians acquired an increasing understanding of the development of medieval architecture, so a more sympathetic ‘restoration’ rather than ‘alteration’ dominated. To encourage true ‘restoration’, the Cambridge Camden Society (later the Ecclesiological Society) and the Oxford Society for Promoting the Study of Gothic Architecture were both founded in 1839. The mentor of the movement was Augustus Pugin, actively involved in ecclesiastical work in the 1840s. It was not so much the size of the church which was significant, rather its form, so that it suited the rituals to be performed within it. Importance was placed on the chancel, with an ornamental reredos behind the altar, a bell tower and, most significantly, stained glass. The promotion of restoration was articulated in Ruskin's The Seven Lamps of Architecture, written in 1849, in which he deplored the practice of gutting churches and renewing what could in fact have been repaired. It was the high church rectors in particular who were responsible for the Gothic revival and who wanted to restore church buildings, and also much of the ceremonial, to their pre-Reformation glory. Galleries, box pews and with them pew rents were seen as eighteenth-century additions to be removed. Emphasis on the pulpit, often a three-decker, with its clerk's reading desk, highlighting the role of the priest as preacher and teacher, was to be replaced by a focus on the altar and the priest as mediator between God and man.