“This election result reflects the very real anger felt by the electorate towards Downing Street,” Sir Roger wrote on Twitter on Friday morning after the Lib Dems’ Helen Morgan won a historically safe Tory seat by 5,925 votes.
“Nobody wants a leadership challenge in the middle of the pandemic but one more strike and he’s out.”
Sir Roger said Boris Johnson now has the Christmas recess to “re-group”.
He, however, also admitted he had already written a letter to the chair of the Tory backbench 1922 Committee expressing no confidence in Mr Johnson.
He also said that he should resign if it is proved that he deliberately misled Parliament by insisting that no Covid laws were broken when staff held Christmas gatherings at Downing Street last year.
Sir Roger said on Sky News: “The House has got time to think about this, and the Party has got time to think about this. He has got about three weeks now to regroup and he is going to have to do that very fast indeed. We have had two strikes; first of all the Conservative Party in the House of Commons earlier this week, now this result, one more strike and he’s out.
“This is not a happy time for the Prime Minister I am afraid.
“I have written to Sir Graham Brady but I did that after the Barnard Castle business because I felt then that there was a lack of the right leadership and I made that view known and I have been making it known since.
“I can’t tell you what the rest of the party is saying but I have an idea that there will be a lot of people who are very concerned this morning.”
Another Tory MP, Charles Walker, said now was not the time for a leadership challenge: “Of course it’s disappointing, of course it’s a really bad morning, of course we would have liked to have won it.
“But it doesn’t mean the end, and it certainly doesn’t mean leadership challenges. The Conservative Party is not going to have a leadership challenge as we are heading into further potential restrictions around Covid…that would be completely self indulgent.”
The polling expert Professor John Curtice, from the University of Strathclyde, said the result was a political earthquake.
“I think this is certainly at least an 8.5 on a scale out of 10 ,” Prof Curtice told BBC Radio 4’s Today programme. “It’s not quite unprecedented but the precedents are not very comfortable for the Conservatives.
“We are looking at a 3 4 per cent swing from Conservatives to Liberal Democrats since 2019, the only previous occasions when a swing has been bigger in a Conservative defended by-election with the Liberal Democrats was 5 3per cent in Christchurch.”
Asked if this was an earthquake, Prof Curtice said: “Yes..Not least of course because in a sense two weeks ago nobody would have seen this coming.
“This was not a constituency first of all that looked like at all promising territory for the Liberal Democrats, it voted 60 per cent to Leave [the EU], it was very different from Chesham and Amersham where the Liberal Democrats were always in with a good chance.
“We are seeing Conservative voters and Leave voters in a true blue constituency taking the opportunity to protest pretty spectacularly about what they think about this current Government.”
However, Conservative Party chairman Oliver Dowden insisted that Mr Johnson was still an “electoral asset” despite admitting his party had been given a “kicking” in North Shropshire.
He told Sky News: “I think the Prime Minister is still an electoral asset and I tell you why. Just take the three really big calls we have faced: get Brexit done, we have delivered the fastest booster programme in the world twice over and we have seen the economy through this terrible period.
“Actually on the big calls the Prime Minister has the vision and the direction to get us through this difficult period.
“I have no idea whether letters are being written but I have every confidence tat the Prime Minister has the leadership to see us through this period.”