The Hammers have finally signed a contract for a 99-year lease on the stadium and will move for the start of the 2016/17 season but Gold has turned his ire on the original decision-makers including London 2012 chairman Sebastian Coe, ex-Olympics minister Tessa Jowell and former mayor Ken Livingstone to rule out football in 2007.
Gold said: "I'm angry from a taxpayer point of view, not a West Ham point of view. It has impacted on West Ham because the London Legacy Development Corporation has had to negotiate a tougher deal with us. It was arrogance. It wasn't even foolhardy, it was a form of arrogance."
He went on: "A child would know that the main issue after building the stadium for the Olympics was what was going to happen in the future.
"Go back over 24 years, you've got six stadiums and five of them are obsolete. What were they thinking?"
The conversion work, which will start in October, will cost between £150million and £190m and former sports minister Richard Caborn, who had argued unsuccessfully for football in 2007, branded the deal as "the biggest mistake of the London Olympics".
Caborn told the Press Association: "West Ham are basically getting a stadium costing more than £600million for just £15million and a small amount in annual rent [£2million].
"This is the biggest mistake of the Olympics and lessons should be learned from this."
The London 2012 centrepiece will now be transformed into a 54,000-seater venue and London mayor Boris Johnson defended the cost to the public purse.
Johnson, who is also chair of the LLDC, said: "I think when you look at the deal the income, which is going to come in from rent, hospitality and naming rights, will be very, very substantial. That means there will be no more subsidy from the taxpayer to keep the whole thing going."