Boris Johnson is continuing to reshuffle his government, sacking longstanding ministers in moves expected to make way for ambitious younger MPs.

The media minister, John Whittingdale, and the veteran schools minister, Nick Gibb, first appointed to the education department by David Cameron, have both been removed.

Whittingdale tweeted that he was sorry to be stepping down, and sad to say goodbye to officials.

Neither man’s replacement has yet been named, but Downing Street sources have made clear the prime minister is keen to refresh the government with MPs who have arrived in the House of Commons more recently.

Junior ministerial appointments continue to be announced, as the reshuffle rolls into its second day. Penny Mordaunt – a staunch Brexiter, popular among backbench colleagues – has been moved from the Treasury to be minister of state at the Department for International Trade, under its new boss Anne-Marie Trevelyan.

In a wide-ranging reshuffle of the cabinet on Wednesday, Gavin Williamson was fired as education secretary after his handling of the exams fiasco during the coronavirus crisis, while Robert Buckland lost his job as justice secretary.

Buckland was replaced by Dominic Raab, who was demoted from foreign secretary after widespread criticism of his handling of the Afghanistan crisis, during which he was on holiday in Crete while Kabul was falling to the Taliban.

Liz Truss succeeded him as foreign secretary, the first time a woman has held the post in more than a decade.

Oliver Dowden was replaced as culture secretary by Nadine Dorries, and he instead was made the Tory party co-chairman before quickly readying Conservative staff for the next general election which is expected to take place in 2024, or potentially earlier.

“You can’t fatten a pig on market day,” he was understood to have said. “It’s time to go to our offices and prepare for the next election.”

Michael Gove succeeded a sacked Robert Jenrick as housing secretary and was entrusted with a further key position in the post-coronavirus agenda by taking responsibility for “levelling up”.

On Thursday the defence secretary, Ben Wallace, who remained in his role in the reshuffle, has said Boris Johnson did not sack or demote MPs from the cabinet because of their incompetence.

Wallace said characterisations of Williamson, who was sacked as education secretary, have been “unfair” and said Raab was not demoted from foreign secretary because he was holidaying while Kabul was falling to Afghanistan.

Wallace told BBC Breakfast the prime minister had removed people from government “not because they’re incompetent, not because they weren’t loyal enough” but because he had to “refresh his team”.

Wallace said of Raab: “Dominic is by trade a lawyer, he started his life in the Foreign Office as a human rights lawyer and he’s gone to the Ministry of Justice, which is actually a very, very important role and a role he desperately understands.”

Wallace also said that improving women’s representation in the cabinet was key in Johnson’s thinking. Speaking on Sky News, he said: “The prime minister wanted to bring forward a number of women MPs, he’s determined to both level up not only in the country but also in my party’s representation around the cabinet table.”

He defended the appointment of Nadine Dorries as secretary of state for digital, culture, media and sport, saying: “I think Nadine Dorries is actually a bestselling author … She’s sold thousands and thousands of books and now if that isn’t part of culture, media and sport, I don’t know.

“What’s great about Nadine Dorries is she produces culture that people buy and actually want to see rather than some of the more crackpot schemes we’ve seen being funded in the past by taxpayers’ money.”

Meanwhile, the former head of Ofsted, Sir Michael Wilshaw, has said the new education secretary, Nadhim Zahawi, has a “big job on his hands”.

Wilshaw told Radio 4’s Today programme: “I think it’s really important we shouldn’t underestimate the damage and disruption that Covid caused to schools, teachers and students.

“The new secretary of state has a big job on his hands to stabilise the education system and restore confidence amongst head teachers and teaching staff, which has been badly damaged.

“He’s got to be a powerful voice in cabinet. When I was a head teacher, I was inspired by strong secretaries of state.”

He said the government should have listened to Sir Kevan Collins, who was appointed as an “education recovery” tsar but quit in June after a row over funding.