The Old Drovers Tavern at Nelson was originally known as The Sickle and Sheath Drover’s Tavern. Located at 3307 US 20, the northeast corner of the Nelson four corners, the building is eligible for the National Register of Historic Places.
The building was constructed c.1804 serving as a tavern for cattle drovers on the Cherry Valley Turnpike (US 20) that had opened just a few years earlier. The tavern went through multiple owners during much of the mid-19th century. In May 1874, the Cazenovia Republican reported that John James & Son had acquired the property and would make extensive repairs before opening a store there. It is probable that repairs included a reconstruction of the hotel facade to include two of the present-day storefronts and canopy. The James store sold hardware and tinware fabricated by John James' son William T. James. The building also served as the village post office. After William James’ death in 1917, his son Bryan, with his wife Anna as bookkeeper, partnered with F. J. Hamilton to open a feed store in one half of the building. After the post office closed in 1934, Hamilton opened an antiques store in the other half. Bryan and Anna James continued to sell antiques and lived in the building until Bryan's death in 1972, at which time Bill Magee bought it. The building continued to be an antiques store and auction house until recently.

The building retains a high degree of architectural integrity to its period of construction. It is a six-bay, side-gabled, clapboard-sided building with multiple windows at different levels on the second story. This suggests more than one period of construction. Some historic double-hung sash remain, while others have been replaced. There are cornerboards and a delicate frieze board and cornice returns, suggesting its Federal-period construction date. The first story includes a full-length canopy and three historic storefronts, two of which have center doors and flanking multi-light windows. This appears to have been constructed after a historic photo of the building was taken around 1920. 

The building's setting and orientation at a major intersection along the historic Cherry Valley Turnpike contribute to its significance.