A councillor who owns an Airbnb insists they have "positive impacts", after a survey showed 87% of residents are concerned about the huge increase.
Residents have long-raised concerns about Whitstable being jam-packed with second homes and holiday lets - with neighbours complaining about the lack of community.
But Airbnb owners have defended their properties, saying they are "vital" to the local economy.
The online survey, carried out by the Canterbury Green Party, reveals 87.6% of residents have concerns about short-term holiday lets or Airbnbs in Whitstable, out of almost 170 responses.
Those who responded reported concern about the impact on their mental health, according to the Party.
But it was also revealed 66.2% of residents agree there are "positive impacts" of Airbnbs for the town, out of 142 responses.
Chris Cornell, a Labour town councillor and also an Airbnb owner, says they are a "vital part of our tourist economy" - but supports them having more regulation.
He says they have an "important role in our economy" as they are usually cleaned and serviced by local people.
And he argues that if the town did not have them, there would be "a lot of jobs in local businesses that wouldn't be there".
"I pay business rates and have commercial cleaning contracts so I’m not your average Airbnb owner as I pay more than I’m forced to," he says.
"But like many people, it was an investment and a way of sharing the town we love with other people.
"It gives also gives us flexibility to allow friends and family to come and stay.
"Like most Airbnb owners, it’s not big business and we don’t own thousands of them or taking billions of pounds in.
"Most of us are people trying to share the town we love and support local business."
Whitstable is ranked as the eighth most popular town in the UK for second-home owners, with Albert Street thought to have among the highest concentration.
Its appeal is clear. The pretty seaside town is perfect for those looking for a staycation - offering multiple pubs, seafood restaurants, a beautiful pebble beach and a picturesque working harbour.
But for many who call the area home, the influx of tourists is pushing locals out and eroding the sense of community they felt many years ago.
Earlier this year, KentOnline reported how many who call the area home think the influx of tourists is pushing locals out and eroding the sense of community they felt many years ago.
Cllr Cornell believes there are "valid concerns from residents" regarding the proliferation of the holiday lets in the area.
But he added: "I think you need to be careful about whether they are to do with Airbnbs.
"There are valid concerns about waste collection - Airbnbs have people that put waste out at all times of the day.
"Business owners should have commercial contracts but the council should make that easier.
"There are valid concerns about people who play loud music at night but you can have that same problem with neighbours.
"Airbnb owners want to make sure their neighbours are well looked after."
Whitstable residents are not alone in wanting more control over numbers.
People in Whitby voted overwhelmingly in favour of preventing new properties from becoming holiday lets or second homes.
A parish poll in the North Yorkshire tourist hotspot earlier this year saw 93% of those taking part agree to restrict new development to "full-time local occupation". But the vote has no power to bind policy-makers.
Campaigners in Whitby say something needs to happen to prevent young people leaving the old fishing port and make it easier to attract teachers, doctors and other key workers who have been priced out of the town.
Cllr Cornell says he doesn't mind a referendum, but argues it should ask what the town should do about them, rather than whether Airbnbs are a problem.
"I would like to see Airbnbs treated as businesses," he added.
"They should pay business rates, not be able to use residential waste collection contracts, they should have valid gas and electricity checks, fire evacuation routes and plans in the same way a hotel should.
"There should be a register for the council about where they are so they can contact them about issues such as fly-tipping and have a proactive relationship with owners of Airbnbs."
"The council needs to do much more to deal with the real concerns of residents about parking..."
The survey also found 80.7% of residents, out of 171 responses, support the introduction of regulations on Airbnbs.
The Green Party says respondents want a better balance between short-term lets and residential properties and to ensure the council receives contributions it needs to provide essential services.
Residents also want owners to be accountable for the upkeep of properties to a good standard and for the behaviour of guests.
Meanwhile, the Canterbury district's only Green Party councillor, Clare Turnbull, says "there are obviously economic benefits from holiday lets".
"But the council needs to do much more to deal with the real concerns of residents about parking, anti-social behaviour, rubbish collection and the availability of rental housing for locals, which all affects the community," she said.
"Overall, it is fair to say the majority of Whitstable residents who responded felt that there are too many short-term lets in Whitstable.
"The balance has shifted in a way that means it is no longer a lovely place to live with a strong sense of community in its narrow streets."
Cllr Turnbull says she intends to make a formal motion to Canterbury City Council next month so that residents' concerns can be properly addressed
"We support sensible regulation but want it to be proportionate and supportive," she added.