Attorney general Dominic Grieve applied for the quashing of the original findings, saying there was a "good" case for setting them aside and holding new hearings into the deaths of the 96 Liverpool fans killed in the 1989 disaster at Sheffield Wednesday's Hillsborough stadium.
That will lead to a re-examination of the roles of the police, Sheffield council, Sheffield Wednesday and emergency services in the events of April 15, 1989 - and there is a possibility that verdicts of 'unlawful killing' could be reached.
The move comes after a damning report into the tragedy - which occurred in the early stages of an FA Cup semi-final against Nottingham Forest - revealed a cover-up which attempted to shift the blame on to the victims.
The lord chief justice said it was "inevitable... desirable and reasonable for a fresh inquest to be heard.
"However distressing or unpalatable, the truth will be brought to light. In this way, the families of those who died in the disaster will be properly respected. Our earnest wish is the new inquest will not be delayed for a moment longer than necessary."
Medical evidence presented by the Hillsborough Independent Panel revealed that 58 of the victims "definitely or probably" could have survived beyond 3.15pm, the cut-off point imposed by the original coroner.
"The new medical evidence presented by the panel's report leads to the conclusion that justice has not been done," Grieve said, while the lord chief justice said there was "ample evidence to suggest that the 3.15pm cut-off was seriously flawed" and also referred to the 116 amendments to police statements - which sought to portray them in a more positive light - and safety of the stadium.
Former Durham Chief Constable Jon Stoddart will lead the new investigation.
"The findings of the Hillsborough Independent Panel were truly shocking, but while the families have now been given the truth, they have not yet received justice," Home Secretary May said.
"I am determined to see a swift and thorough response to the findings of the Hillsborough Panel to deliver justice for the 96 football fans who died and the families who have fought so hard on their behalf.
"Jon Stoddart is a skilled and dedicated investigator who will bring a huge amount of policing experience to this demanding job."
She added that the inquiry will focus on the fans' deaths and cooperate with a separate police watchdog investigation into the conduct of the police in the aftermath of the disaster.
Some of the families of victims, who have campaigned to have verdicts overturned, attended the hearing.
The Independent Police Complaints Commission announced on December 12 that it had begun consulting families on its provisional terms of reference for its investigation into the aftermath of the tragedy.
Liverpool Walton MP Steve Rotheram, one of a number of MPs at the proceedings, said beforehand: "The High Court hearing is the day that the families have fought almost a quarter of a century for. The opportunity to quash the original inquest verdicts of accidental death seemed like an impossible task for 23 years.
"It is the moment they have waited over two decades for. My hope is that the overwhelming evidence that was uncovered in the Hillsborough Independent Panel report will be enough to emphatically prove that Hillsborough was not an accident.
"The wheels of justice turn slowly in Britain but they are beginning to gather momentum. This is just the beginning of a process that will see one of the greatest injustices in the last century put right and those really responsible for Hillsborough held to account."