The prescription drug Stilnox was banned by the Australian Olympic Committee just before the 2012 Games and the athletes now face sanctions from the governing body for breaching their Olympic team membership agreement.
"We stand here today to admit that we took Stilnox following a day of bonding for the relay team at the training camp in Manchester," the six swimmers said in a statement read out at a news conference in Sydney.
The Olympic silver medallist and his team-mates Matt Targett, Eamon Sullivan, James Roberts, Cameron McEvoy and Tommaso D'Orsogna apologised for their actions, which they said followed a "tradition" in the Australian relay team.
The statement said the drug had been prescribed before the AOC ban and the swimmers had then indulged in "childish" and "stupid" pranks such as knocking on the doors of other athletes.
"I think one of the reasons I agreed to go along with it was all the pressure I was under," said Magnussen. "Completely inappropriate in hindsight."
Misuse of prescription drugs along with allegations of drunkenness, breaching curfews, deceit and bullying were highlighted in a review published on Tuesday, which said slack management had allowed a "toxic" culture to develop in the team.
The relay squad arrived in London confident of winning gold but ended up fourth in the final as the swimming team produced Australia's worst results in the Olympic pool for two decades.
"If I thought it would affect our performance, there's no way we would have taken it," said Sullivan. "I believe there's no way it affected our performance."