Lee Anderson has doubled down on his criticism of Sadiq Khan and refused to apologise for remarks that sparked an Islamophobia row, as Rishi Sunak came under pressure to call out his words as Islamophobic.

The former Tory deputy chairman on Monday admitted his original remarks were “clumsy”, but said saying sorry “would be a sign of weakness”.

It came after the Prime Minister broke his silence on the matter to denounce Mr Anderson’s comments that cost him the Conservative whip as “wrong”.

The Ashfield MP was suspended over the weekend after he claimed “Islamists” had “got control” of Mr Khan and London.

On Monday, Mr Anderson told GB News: “When you think you are right you should never apologise because to do so would be a sign of weakness.”

In a statement to the channel, where he presents a weekly show, he said he had made comments “that some people thought were divisive”.

“Politics is divisive and I am just incredibly frustrated about the abject failures of the Mayor of London, Sadiq Khan,” Mr Anderson said.

“My words may have been clumsy but my words were borne out of sheer frustration at what is happening to our beautiful capital city.”

In a fresh attack on Mr Khan, the now-independent MP said: “Hundreds of people had been arrested for racist abuse on these marches and we barely hear a peep from the mayor. If these marches were about something less fashionable, Sadiq Khan would have been the first to call for them to be cancelled. It’s double standards for political benefit.”

Mr Sunak said Mr Anderson had been deprived of the Tory whip because his “choice of words wasn’t acceptable, it was wrong”.

But he refused to describe the MP’s comments as Islamophobic when pressed repeatedly, telling broadcasters in Yorkshire: “I think the most important thing is that the words were wrong, they were ill-judged, they were unacceptable…

“It’s important that everybody, but particularly elected politicians, are careful with their words and do not inflame tensions.”

Sir Keir Starmer said the Tory leader “lacks the backbone to call this out for what it is”.

Speaking to reporters in Shrewsbury, the Labour leader said: “I think this is straightforward. It’s Islamophobia and the Prime Minister should call it out for what it is.

“The reason he won’t is because he is so weak.”

And Mr Khan said the Prime Minister’s refusal to call Mr Anderson’s remarks Islamophobic “speaks volumes”.

The Labour Mayor wrote in the Evening Standard: “It shouldn’t be hard to call out comments that are so unambiguously ignorant, prejudiced and racist. Yet those at the top of the Conservative Government are stubbornly refusing to do so.

“It’s a tacit endorsement of anti-Muslim hatred and can only lead to the conclusion that anti-Muslim bigotry and racism are not taken seriously.”

The Prime Minister also declined to criticise former home secretary Suella Braverman, who claimed “the Islamists, the extremists and the antisemites are in charge now”.

“I think that those comments were not about an individual in particular,” he said, adding that “it’s important that we call out” the kind of “unacceptable” scenes “we have been witnessing on our streets in recent times”.

Asked whether his party has an Islamophobia problem, Mr Sunak told local BBC radio stations in the morning: “No, of course it doesn’t”.

Amid a wider spat in the party about language used, West Midlands mayor Andy Street lambasted fellow Conservative Paul Scully for claiming that a part of Birmingham with a large Muslim community was a “no-go area”.

“The idea that Birmingham has a ‘no-go’ zone is news to me, and I suspect the good people of Sparkhill,” Mr Street said on X, formerly Twitter.

“It really is time for those in Westminster to stop the nonsense slurs and experience the real world.”

No 10 said Mr Sunak did not agree with Mr Scully’s assessment.

Asked whether the Prime Minister concurred that there are “no-go” areas in Birmingham, his official spokesman said: “No, and the PM has talked before about the value of the very diverse communities and societies that we have in the UK.”

A Conservative party source had defended Mr Anderson’s comments on Friday night, before he was stripped of party support on Saturday amid mounting condemnation from across the political divide.

Mr Anderson, a standard bearer for the Tory right, will now sit as an independent unless he defects to another party that chooses to offer him its backing.

Reform UK leader Richard Tice did not rule out opening the door to Mr Anderson after his suspension.

“Lee Anderson may have been clumsy in his precise choice of words, but his sentiments are supported by millions of British citizens, including myself,” he said in a statement.

“I do not and will not give a running commentary on any discussions I have with any MPs, but those MPs have my number.”

We have always been clear that this Government does not and will not tolerate anti-Muslim hatred

Mr Anderson’s comments have put a spotlight on the ongoing dispute over the classification of Islamophobia.

Downing Street reiterated the Government’s position of refusing to back the all-party parliamentary group on British Muslims’ 2019 definition.

The Prime Minister’s official spokesman told reporters: “As the Government has stated previously, there are issues in relation to the APPG’s definition of Islamophobia, which conflates race with religion, does not address sectarianism within Islam, and may unintentionally undermine freedom of speech.

“But as I’ve said, we have always been clear that this Government does not and will not tolerate anti-Muslim hatred.”

Tory peer Baroness Warsi criticised Business Secretary Kemi Badenoch after she said the definition supported by Labour and the Liberal Democrats “creates a blasphemy law via the back door if adopted”.

“This is absolute nonsense Kemi,” Baroness Warsi, who was a cabinet minister in Lord David Cameron’s government, tweeted.

“The government has dragged its heels on any work to tackle this form of racism…

“No new initiatives, no engagement with communities, no definition. Instead culture wars, othering and blatant anti-Muslim racism.”