Boris Johnson is facing calls from senior Tories to stand down as prime minister after he admitted attending a drinks party during lockdown.
The PM apologised for the way he had handled the event in the Downing Street garden in 2020 and said he understood the public's "rage" over it.
Cabinet members including deputy PM Dominic Raab rallied round Johnson.
But Scottish Tory leader Douglas Ross and senior backbenchers William Wragg and Caroline Nokes called on him to go.
Ross, an MP and an MSP, said he had a "difficult conversation" with Johnson after the PM's apology to MPs earlier on Wednesday.
He said he would write to the 1922 Committee - which organises Tory leadership contests - to register his lack of confidence in the PM.
"He is the prime minister, it is his government that put these rules in place, and he has to be held to account for his actions," he said.
If 54 backbench Conservative MPs send letters to the 1922 committee it will trigger a leadership challenge.
Ministers have urged MPs to wait for the outcome of an investigation into alleged Covid-rule breaking at Downing Street parties by senior civil servant Sue Gray, which they say will be published shortly.
But another backbencher William Wragg, who chairs an influential select committee, said the prime minister's position was "untenable".
"I don't think it should be left to the findings of a civil servant to determine the future of the prime minister and indeed who governs this country," he told BBC Radio 4's PM programme.
And fellow Tory Caroline Nokes, who chairs another Commons committee, said the prime minister should resign now as he was "damaging the entire Conservative brand".
The former minister, who has previously been critical of Johnson's leadership, told ITV's Robert Peston: "Regretfully, he looks like a liability.
"And I think he either goes now, or he goes in three years' time at a general election."
At Prime Minister's Questions, Johnson sought to calm anger about reports he had attended a "bring your own booze" party at the height of the first lockdown.
He admitted he was at the 20 May 2020 gathering for about 25 minutes, so that he could "thank groups of staff" for their hard work.
He said: "I believed implicitly that this was a work event."
But he added: "With hindsight I should have sent everyone back inside.
"I should have found some other way to thank them, and I should have recognised that - even if it could have been said technically to fall within the guidance - there would be millions and millions of people who simply would not see it that way."
Labour leader Sir Keir Starmer said the PM's defence was "so ridiculous that it's actually offensive to the British public" and he called on Johnson "to do the decent thing and resign".
The SNP's leader at Westminster Ian Blackford called on Tory MPs to force the PM out - and Lib Dem leader Sir Ed Davey said he had to go.
Sir Ed has written to the Metropolitan Police to ask them to investigate Johnson's attendance at the 20 May party.
The PM's statement was met with a mixed reaction from Tory MPs, with some saying he had sounded genuinely contrite.
Dan Poulter, who also works as a psychiatrist, said: "I am pleased that the prime minister has apologised, but it's not much consolation to those of us who cared for patients on the frontline of the NHS and saw them die of Covid".
Dominic Raab said the PM had "given a very clear account" of what had happened and had apologised for the "perception" of rule breaking.
He said detailed claims about the 20 May party were being investigated by Sue Gray and the prime minister would update the Commons once she had finished her work.
Levelling-Up Secretary Michael Gove also backed the PM at a meeting of the 1922 committee.
Speaking to the BBC before the meeting of Tory backbenchers, Gove said it was "right for the PM to apologise" but it was also "important we all wait for Sue Gray's report to conclude".
Chancellor Rishi Sunak, who has been visiting a pharmaceutical firm in the West Country, has not been available for interview.
But on Wednesday evening, he tweeted that Johnson was "right to apologise", and he supported the PM's call for "patience" whilst Gray completed her investigation.