Set around a pleasant green, the parish has seen a massive decline in population during the 20th century. Today under 70 residents remain. The redundant church is now owned by the village on a peppercorn rent and is used for village services at least 3 times a year.
The village has a number of impressive, moated old houses.
Southolt appears on John Speed's 1610 map as "Southold"