CARLTON HALL STOOD IN A LARGE PARK BETWEEN SAXMUNDHAM AND KELSALE, facing across a low valley to Carlton Parish Church. The main block was destroyed by fire on Christmas Day 1941, the service wing being converted into a house after the war. At the beginning of the twenty-first century the remaining wing and its environs were developed with new housing including a new mansion.

The original Carlton Hall appears to have been on a different site from the building burnt down in 1941. Its owner in 1655 bore the name Osborne, described as ‘a Kentish man his estate between £200 and £300 p. annum’. This hall may have been the seat of the lord of the manor of Carlton but the manor belonged to the Bence family of Thorington and was held separately from the estate from the early eighteenth century.

In the middle of that century Carlton was owned by Osborn Fuller of Tottenham High Cross in Middlesex who acquired it in 1753 from the mortgagees of William Fuller of Saxmundham and his wife. Whether Osborn and William Fuller were related and whether Osborn's name reflects relationship with the Osborne who owned Carlton Hall in the previous century are matters for speculation. Osborn Fuller died in 1794, having increased the size of the estate by a number of purchases both in his own name and that of his son Edward, who was born about 1782. Edward Fuller also made a number of purchases to increase the size of the estate, but it was subject to mortgages and various parts were sold off in 1837 to Lionel Dove. In 1850 the estate was purchased by Edward Walter Bonham, who lived in France, and was transferred to the marriage settlement of his son Henry, who married Augusta Musgrave. Following Henry Bonham's death in 1856 his widow, who in the following year married the second Earl of Stradbroke, became the effective owner of Carlton until it formed part of the marriage settlement of her son, Henry Walter Musgrave Bonham, in 1875.