The owner of The Grand is considering legal action against HM Revenue and Customs after he was arrested and his business raided.
Michael Stainer was held in custody for nine hours following the dawn bust on the hotel on Thursday.
Officials took computers and accounting records from the premises on The Leas, Folkestone, and also arrested Mr Stainer’s wife Doris.
It is understood HMRC may have been looking for cash which could have been used to pay The Grand’s tax liabilities.
Witnesses reported seeing officials raid the property and take away a number of items in sealed plastic bags.
Three were arrested and released on bail, a spokesman for HMRC said today after raids in Folkestone and Barham.
Mr Stainer, a chartered accountant, appeared at the High Court in June in a case brought against him by HMRC over who was responsible for paying PAYE and national insurance. A decision is yet to be officially made.
"I don’t know what they were expecting to find. Nothing has come of it. We are considering taking action because they came on a fishing expedition." Michael Stainer
He said: “They didn’t have any proper evidence for a warrant.
“They knew there was unpaid PAYE because of the figures we gave them during the tribunal but presumably someone told them something else was wrong.
“I don’t know what they were expecting to find. Nothing has come of it. We are considering taking action because they came on a fishing expedition.
“They are supposed to have evidence to get a warrant and they did not have any.”
Mr Stainer marked 40 years of owning The Grand at the beginning of this month.
He bought the hotel in 1975 when it was under threat of demolition and conversion into modern flats.
He has since restored many rooms and lets 44 long-lease flats in the building, with another 18 flats available, which have 46 rooms between them.