Plans had called for redeveloping the property into housing, reusing six buildings including the administration building that was heavily damaged.
A developer has owned the site since 2015, but the project has stalled over the past decade.
Preservationists say they are now trying to assess what remains following the fire.
They note the buildings have sat vacant for years, with reports of people repeatedly entering the structures despite efforts to secure them.
“I don’t know what’s next for this site,” said Yvonne Laube of the Town of Poughkeepsie Historic Preservation Board.
“We’ll take it from there and see what’s left, inventory what is left and see what can be saved and what can be reused,” she said.
The cause of the fire has not been determined.
Officials and preservation groups say the next steps will depend on how much of the site can still be salvaged.