The Taliban have 80,000 troops in comparison with a nominal 300,699 serving the Afghan government, yet the whole country has been effectively overrun in a matter of weeks as military commanders surrendered without a fight in a matter of hours.

It is a tale of two armies, one poorly equipped but highly motivated ideologically, and the other nominally well-equipped, but dependent on Nato support, poorly led and riddled with corruption.

The US aid spending watchdog for Afghanistan warned last month that the US military had little or no means of knowing the capability of the Afghan National Defense and Security forces (ANDSF) when required to operate independently of the US forces, despite spending $88.3bn on security-related reconstruction in Afghanistan up to March 2021.

It found the US military to be persistently overoptimistic about Afghan military capability, even though it had no reliable evidence to make that assessment, and said the departure of thousands of US contractors, agreed by the US with the Taliban in 2020, “could significantly impact the sustainability of the ANDSF, in particular their ability to maintain aircraft and vehicles”.

The watchdog had, it said, repeatedly warned about “the corrosive effects of corruption” within the force. With its reliance on advanced equipment, and with widespread illiteracy in its ranks, the force could not reliably maintain its strength and combat readiness.

Of the $88.3bn spent, the watchdog said: “The question of whether that money was well spent will ultimately be answered by the outcome of the fighting on the ground, perhaps the purest monitoring and evaluation exercise.”

The report’s clear warnings are likely to be reviewed by US Congress as it seeks to understand why such vast spending on training the Afghan military has led to a collapse to the Taliban in a matter of weeks, leaving western politicians shocked and bemused.

It also raises the question of why the Biden administration ever thought it was safe to leave Afghan forces on their own after a decades of dependence on the US for key skills, including air cover, logistics, maintenance, and training support for ANDSF ground vehicles and aircraft; security; base support; and transportation services. The US president said as recently as 8 July that there was no likelihood of Afghanistan being overrun.

At the same time, the level of Taliban attacks were increasing. In each three-month period since 29 February 2020, the date of the US-Taliban agreement, there were significantly more enemy-initiated attacks than in their corresponding quarters the previous year.

Yet a week before Biden said Afghanistan would not be overrun, it was reported by the independent Afghan Analysts Network that the Taliban had captured 127 of the 420 district centres, about 25% of the total, and by 21 July, Gen Mark Milley, the chairman of the US joint chiefs of staff, said the figure was more than half. In June, he said, it had been only 81. It was noticeable that some of the districts falling were in traditionally anti-Taliban areas.

The additional problem was a central government facing a severe fiscal crisis precipitated by the loss of customs revenues and declining aid flows. Many officials complained that they had not been paid for months.

Fear was a further factor. As momentum swung towards the Taliban, fostered by Taliban social media, the speed of events became fuelled by fear of revenge and personal scores being settled under cover of a takeover, particularly in a large city like Kabul.

The Afghan government provided no counter-narrative.

By then the US retreat was well under way and near irreversible; its withdrawal was more than 90% complete by 5 July. The process included 984 C-17 aircraft loads being transported out of Afghanistan, more than 17,000 pieces of equipment being turned over, and 10 facilities, including Bagram airfield, being handed over to Afghanistan’s Ministry of Defence.

Sigar said $88.3bn had been allocated up to March for security-related reconstruction, compared with $36bn for governance and development, yet it discovered the Pentagon had always always found it “extremely challenging” to evaluate the fighting and administrative capacity of the ANDSF.

It started its multibillion-dollar training of Afghan forces in 2002 and three years later took control of training both the police and military, so US military trainers have had nearly two decades to ready the Afghan forces for a Taliban insurgency.

At the outset, the US began transforming the Afghan National Army from a light-infantry force to a combined-arms service with army, air force, and special forces element.

The Sigar report found that from 2005 the US military had been seeking to evaluate the battle-readiness of the troops they had been training, but by 2010 acknowledged that its monitoring and evaluation procedures “failed to measure more intangible readiness factors, such as leadership, corruption and motivation – all factors that could affect a unit’s ability to put its staffing and equipment to use during actual war-fighting”.

The assessment mechanism changed again in 2013, but in 2014, with few signs of progress emerging, it was decided that the assessment reports should become classified. The focus shifted from battalions to command headquarters.

The report also found a disjunction between what generals told Congress and what lower level officers reported. Sigar reports, for instance, that in March 2011 testimony to Congress, Gen David Petraeus – then the commander of the International Security Assistance Force – claimed that “investments in leader development, literacy and institutions have yielded significant dividends” for the ANDSF, that Afghan forces were taking on significant combat roles against the Taliban, and that Afghan local police units were increasingly limiting the Taliban’s ability to intimidate communities”.

Many other US generals made similarly optimistic claims. But other reports indicated “the absence of success on virtually every level”.

In a 2012 Armed Forces Journal article, Lt Col Daniel Davis, who spent a year in Afghanistan speaking with US troops and their Afghan counterparts, wrote that his observations “bore no resemblance to rosy official statements by US military leaders about conditions on the ground”.

The Sigar report also lambasted the tendency for politicians and senior military to look for good news. It says there is a “natural desire for good news to pass on up the chain of command”.

“In the words of one former senior military official: ‘As intelligence makes its way up higher, it gets consolidated and watered down; it gets politicised. It gets politicked because once policymakers get their hands on it, and frankly, once operational commanders get their hands on it, they put their twist on it. Operational commanders, state department policymakers and Department of Defense policymakers are going to be inherently rosy in their assessments. They will be unaccepting of hard-hitting intelligence.”