The Listed Buildings Register says this building is the "former Castle Inn". At some (uncertain) time, the license and name moved up the street to another Castle Inn.

The building dates from the 17th century.

It isn't marked on any available OS map (at least as early as 1881) so presumably closed before that date.