The former HBC Engineering site in Power Station Road, Halfway, closed in December 2009 and went into the hands of administrators Grant Thornton.
Developer Citygrove Securities later purchased the site and had a proposal to build a Sainsbury’s there approved in January 2012.
The former HBC Engineering site in Power Station Road, Halfway which is being demolished
However, Communities Secretary Eric Pickles intervened and prevented full permission from being granted.
Sainsbury’s then pulled out in March.
Citygrove resubmitted the same application for a food store a year ago but it was turned down unanimously by Swale council’s planning committee due to its potential impact on Sheerness town centre.
Ownership then reverted back to Grant Thornton and in the last week property investment company TB Sheerness Limited bought the site.
Notices about the demolition of the building have been put up around the area and windows on the building have already been taken out.
Director Paul Graham said: “The buildings are going to be removed and the site will be cleared down to slab.
“There is demolition in process for the redundant buildings as they are not fit for purpose.
“We have looked at this site over quite a long period and it’s been on the market for a while.”
Mr Graham said the complex was dilapidated and had attracted squatters.
He went on to say plans about what will be built in its place are yet to be decided as it is still, “very early stages”.
The firm will start considering what will replace the factory once the demolition is completed, which it anticipates it will take contractors a maximum of 12 weeks.
David Mosdell, who runs Danley Marshes Riding Centre, in nearby said he is concerned that the demolition work will interfere with his business.
He said: “We are really, really flat out on Saturdays from 9am to 4pm but they [the contractors] seem to think it won’t bother us.”