Murray, whose life support at a Brisbane hospital was turned off on Sunday morning, was a former State of Origin coach for New South Wales and guided the Rhinos to their last Challenge Cup final triumph in 1999.
Murray played at the top level with Parramatta and South Sydney before embarking on a colourful coaching career that included a stint in charge of Fiji for the 1995 World Cup.
He coached North Queensland and the Sydney Roosters in the NRL and took Hunter Mariners to the final of the 1997 World Club Championship before joining Leeds in November of that year.
In addition to masterminding the Rhinos' Wembley triumph, he took them to the inaugural Super League Grand Final in 1998 when they lost to Wigan.
Murray coached New South Wales in 2006 and 2007 and was due to take charge of Wynnum-Manly Seagulls in the Queensland Cup this year and Australia's women at the recent World Cup in England before standing down due to ill health.
He was hospitalised in Brisbane after a suspected heart attack earlier this month and spent a week in a coma after a heart attack in March.
Leeds chief executive Gary Hetherington said: "It is devastating news and so sad to have lost Graham, who will be fondly remembered by all our fans, players and staff who knew him from his time with us.
"He was very popular and so professional in everything he did. He was an outstanding coach and mentor and he cared about all aspects of rugby league.
"The game has lost a great servant and personality and our thoughts are with (Murray's wife) Amanda and (daughter) Kara."
Former Leeds player Barrie McDermott, who was a member of the Rhinos' cup-winning team, said: "Graham Murray had a huge influence on Leeds Rhinos in the late '90s but his legacy is everywhere. So many of the players he influenced are still here and contributing to the philosophy he cultivated."
Leeds will wear black armbands as a mark of respect when they face London Broncos on Thursday at the Twickenham Stoop.