The business secretary has suggested parliament’s standards commissioner should consider quitting, after the government voted to tear up the independent system for combating sleaze among MPs.

Kwasi Kwarteng said he “doesn’t feel shame at all” and insisted the move would restore probity in public servants, in the face of a huge backlash over the move to rescue the former cabinet minister Owen Paterson from a 30-day Commons suspension and possible byelection.

Paterson was found to have committed an “egregious” breach of lobbying rules for repeated contact with ministers and officials while being paid more than GBP100,000 on top of his taxpayer-funded salary. But Boris Johnson whipped Tory MPs to support the creation of a new committee chaired by a Conservative backbencher to review the entire standards process, which opposition parties have already vowed to boycott.

Those who voted for the amendment included 22 Tories who were either already under investigation or had been ruled against by the standards commissioner.

Kwarteng said it was “the express will of parliament”, although the vote was narrowly won by a majority of just 18, with dozens of Tory MPs choosing to abstain – one parliamentary private secretary who did so, Angela Richardson, lost her job.

“I don’t feel shame at all,” Kwarteng told Sky News on Thursday. “It was an independent process, an independent vote at parliament. And I’m really interested in seeing the system reformed and made more just and fair by allowing people a right of appeal.”

He insisted the vote, taken on the day that Paterson’s suspension was due to be confirmed, was not “about the rights and wrongs of what Owen Paterson said or did or how he was paid”.

Instead, Kwarteng said it was “about getting a system of fairness back into almost what might be a kind of employment tribunal” and that the right to appeal was something Paterson should have had.

He also suggested the standards commissioner, Kathryn Stone, who led the nearly two-year investigation into Paterson, should consider resigning. Kwarteng said: “I think it’s difficult to see what the future of the commissioner is, given the fact that we’re reviewing the process and we’re overturning and trying to reform this whole process. But it’s up to the commissioner to decide her position.”

Tory whips had rung round MPs on Tuesday night and asked if they would support the Leadsom amendment, and the chief whip, Mark Spencer, is said to have phoned Johnson that same evening to convince him it was the right course of action, with the prime minister agreeing.

However, discontent is at fever pitch among some Conservatives. One who voted for the amendment said: “I really regret it.” Others lambasted the “politically insane” decision and Paterson and the government’s refusal to back an amendment that would have still sanctioned him, but reduced the suspension to below 10 days, meaning he would have escaped a recall petition.

Labour said the call by Kwarteng for Stone to consider quitting was just the latest example of ministers trying to avoid public scrutiny. Thangam Debbonaire, the shadow leader of the Commons, said: “Having already ripped up the rules policing MPs’ behaviour to protect one of their own, it is appalling that this corrupt government is now trying to bully the standards commissioner out of her job.

“Johnson must immediately distance himself from these latest attempts to poison British politics. And all decent people of all political beliefs must stand against these naked attempts by Tory MPs to avoid scrutiny of their behaviour.”