Sir John accumulated many properties during his lifetime and, through his court connections, was granted possession of numerous lands and castles across Wales by royal charter. Through his marriage to Hawys, Charlton also acquired Powys Castle which he rebuilt with the distinctive red sandstone towers that still characterise the world-famous property today.
If only in name, it seems Charlton Castle continued to be used as a residence of the Lords of Powys until the 16th Century and, by 1820, all that remained of the formerly high-status property was a solitary corner tower. Sir John's former home is still enclosed by a well-preserved, if dry, moat and an entrance causeway which leads to some well preserved foundations and walling. The remnants of a large fishpond that once served the Charlton estate and a rabbit warren, for which Sir John was granted free use in 1307, are located nearby.