Joe Biden could not have been clearer: a Taliban takeover of Afghanistan was “not inevitable”, the US president said on 8 July. Boris Johnson, the UK prime minister, was equally confident – “there is no military path to victory for the Taliban” – he told MPs earlier that day, five weeks ago.

The president said he trusted “the capacity of the Afghan military” who were better trained, better equipped and “more competent in terms of conducting war”. And the prime minister agreed: “I do not believe that the Taliban are guaranteed the kind of victory that we sometimes read about.”

Yet the high-speed collapse of the Afghan government and armed forces, the fall of Kabul, a city of more than 4 million people with barely a shot fired, demonstrates how badly wrong these assessments were. Nor were they just the over-optimistic statements of politicians seeking to justify an exit made for domestic political reasons.

They were echoed by military and intelligence planners, even as the Taliban were making rapid advances across the Afghan countryside, in preparation for the well-telegraphed US-led withdrawal.

“It is unlikely that the Taliban would ever get to full authority if it chose to fight to the end over the whole of Afghanistan,” Gen Sir Nick Carter, head of the British armed forces, said on the same day, highlighting a range of other possible scenarios, including the survival of the Kabul government – whose president fled over the weekend – or a negotiated deal between it and the Taliban.

It was an opinion to which the leading British general stuck, even as the provincial capitals began to fall. Just over a week ago he argued in the Times that the ousted government’s “military strategy is to achieve a stalemate” and that the key was to hold cities such as Herat and Kandahar, both of which fell within days. “There are increasing signs that moderate Afghans in support of the government and its security forces are beginning to show the sort of defiance that’s needed,” the chief of defence staff added.

US intelligence sources were not quite as sure, but even their judgments were still more optimistic than what transpired.

An assessment, leaked to the Wall Street Journal in late June, concluded that the Kabul government led by Ashraf Ghani could collapse in six months after the US withdrawal was completed. Yet, even then, it was already being noted that Afghan security forces were frequently surrendering without a fight in the rural areas, abandoning western equipment as they did so.

Clearly, it was not supposed to end like this. For the British, once combat operations ended in 2014, the focus was on training the Afghan army, with initiatives such as the “Sandhurst in the sand” officer training academy founded with GBP75m of taxpayers’ money. But the reality was, as military analysts observe, that the Afghan army was not effective without residual US air and ground support.

Nick Reynolds, a land warfare analyst with the thinktank Rusi, said the collapse of the Afghan security forces was predictable, exacerbated by corruption and a lack of local legitimacy. “Even basic issues such as ensuring that soldiers received regular pay and were properly supplied and equipped were not entirely resolved,” he said.

Such insights either eluded politicians or were set aside by the US and the UK, which were ready after 20 years of military engagement to exit with haste. The result is a collapse that has left scenes of chaos at the international airport as desperate Afghans try in vain to cling to US air transport planes – and extraordinary propaganda value for those who want to argue western military power is limited when, if 2,500 US troops had stayed on, the Taliban takeover would not be happening.