It's thought to have been just to the north of the town hall.

Haverhill: The Three Goats
High Street
Compact brick built and tiled Beer House
On the First Floor: 3 Rooms and 4 Attics, approached by three staircases.
On the Ground Floor: Bar, Tap Room, Parlour, Kitchen and Scullery, also Cellar in the basement.
In the rear, large Workshop, Pin Shed and Loft over.
Let to Mr. Samuel Suckling.Details of pub premises belonging to William Ward & Son sold, along with the brewery by auction, January 23rd 1894***. Taken from the original source of documentation at the Suffolk Record Office

The 1916 Kelly's Directory lists Mrs Mary Eley as a blacksmith at 36 High Street.