"CFR Cluj's club management and Ioan Andone have reached an agreement to end relations," the club said in a statement.
"We are proud of everything we've accomplished together. Andone has and will always have a special place in the hearts of our supporters."
The Railwaymen, playing for more than an hour with 10 men, drew 1-1 at Galatasaray in a lively Champions League Group H match on Tuesday to stay second behind Manchester United having won one, drawn one and lost one of their opening three matches.
United have nine points, Cluj have four, followed by Braga (3) and Galatasaray (1).
However, local media and fans criticised Andone for CFR's domestic performances. They are 10th in the Romanian top flight with 16 points from 12 matches, 13 points behind leaders Steaua Bucharest and only six points above the relegation zone.
Andone, who led Cluj to a domestic double in 2008, began his second spell as coach in April after replacing Portuguese Jorge Costa and guided the team to their third league title.
According to local media reports, former Napoli and Lazio coach Edoardo Reja is on a shortlist to replace Andone, who could rejoin Rapid Bucharest - the club he coached n 2010.