The original part of the home, a four room clapboard cottage with a central fireplace, was built in the 1840's by Barrington King for H.W. Proudfoot and his wife, Euphemia. Mr. King, son of Roswell's founder, employed Mr. Proudfoot as a bookkeeper in his newly constructed Roswell Mills, which would later become famous in its own right for its production of Roswell Grey cloth used in Confederate uniforms. Reportedly, in anticipation of Sherman's march, huge supplies of Confederate uniforms were taken from the mill and secretly stored at Naylor Hall. In the summer of 1864, the Proudfoot's home was heavily damaged by Federal troops. After occupation, Proudfoot began to rebuild. He remained with the mill, and in his home, until his death in 1871. In the late 1930's Colonel Harrison Broadwell purchased the property, naming it Naylor Hall in honor of his wife's family. He also added the columns, the handcrafted woodwork, and the portico encompassing the original structure.