The Old Gaol was built in 1748 by Richard Grenville Temple, Lord Viscount of Cobham.  It was enlarged by Sir George Gilbert Scott in 1839.

Prisoners were incarcerated in damp cells with no heating, lights or toilets and fed on bread, water, soup and gruel. They exercised in the courtyard and often spent time in the stocks for various misdemeanours.

3 people died in the gaol, one prisoner escaped over the wall and another walked out when the gaoler went home and forgot to lock the door.  It is reputed to be haunted and paranormal investigators are regular visitors.

The building has been a police station, a fire station, an air-raid shelter and an ammunition store.  In 1984, following threat of demolition, the building was brought by a charitable trust and saved for the  town.  It is now run and maintained by volunteers.