Kent County Council has launched legal proceedings against a group of travellers after an illegal camp was set up near the Cyclopark in Gravesend.
The authority confirmed it was seeking a magistrates' court hearing as soon as possible to secure an order to remove the unauthorised encampment close to the sporting facility.
Travellers were spotted moving on to the site last week and have refused to leave despite being issued with a "direction to leave" order on Thursday.
A Cyclopark spokeswoman said: "Today, Cyclopark has been advised by Kent County Council, which owns the land east of the park, that it is seeking a magistrates’ court hearing as soon as possible to secure an order to remove the unauthorised encampment of about 25 caravans.
"KCC, which issued a “direction to leave” order on the travellers last Thursday, is working very closely with police to continuously monitor the situation and public concerns."
Walkers and cyclists have told Kent Online some people are refusing to go back to the area following reports of fly-tipping and abuse.
Dog walker Alison Edwards said: "It does feel intimidating. People have seen cars hurtling through the fields.
"There has been little respect for the area with several trees chopped down. People have stopped from going up there."
The public are advised to call the KCC Contact Centre on 03000 414141 to report any incidents. Information is available on the KCC website: www.kent.gov.uk/unauthorisedsite