Curson Lodge is believed to date from the early 16th century. It was built either as an inn or a lodging house. During the 17th century, the building was used as a naval hospital.

It seems most probable that it was originally a lodging-house for Lord Curson's Elephant & Castle on the opposite corner of Silent Street. There seems to be some confusion in the historic record between the two buildings, both being said to have been used as a naval hospital and inn.

One theory about the origins of Silent Street's name is that when Curson Lodge was being used as a hospital for soldiers wounded in the Dutch wars of the 1650s, the noise of horses and carts in the street kept disturbing them, so straw was lain in the road to make it silent.

It's said to have operated as an inn during most of the 18th century and also for a period during the 19th (from about 1838 onwards).

Although often said to have been associated with Cardinal Wolsey, it seems unlikely that this was in fact the case. However, a blue plaque on the wall notes that his parents were born in a (long-demolished) building across the road. Wolsey himself is believed to have been born in the Black Horse.

The building was damaged in February 2013 when a car ploughed into the cornerpost. Fortunately recent structural alterations meant that this post was no longer load-bearing.