Italian media said Milan, languishing 11th in Serie A, were poised to gamble on their former midfielder Clarence Seedorf as his replacement even though the Dutchman has no previous coaching experience.
Reports said Seedorf, currently playing for Brazilian club Botafogo, would arrive in Italy on Thursday although the player himself told the newspaper he did not want to comment.
"AC Milan wish to communicate that Massimiliano Allegri and his staff have been relieved of their duties with immediate effect," the Serie A club said in a statement as they confirmed Allegri's downfall.
"AC Milan would like to thank Mr Allegri and his staff for their time with the club and wish them the best for the future. Coach Mauro Tassotti will take charge of the team in the meantime."
Allegri was dismissed less than 24 hours after his team squandered a two-goal lead to lose 4-3 at promoted Sassuolo, conceding four times in just over half an hour.
The defeat left Milan with 22 points from 19 games, 20 adrift of Napoli who occupy the Champions League playoff spot in third position.
Milan have won only five league games all season.
Allegri said he thanked "the owners and the club for the three-and-a-half seasons on the red and black bench. It has been a gratifying professional experience, with significant successes," he told the ANSA news agency.
Chief executive Adriano Galliani said he was sorry to see Allegri go.
"I feel very sorry for Allegri, especially for the way the situation developed on a human level, but that's football."
The phlegmatic 46-year-old, in his fourth year at the club, has faced constant media speculation over his future in the last few weeks and said during the Christmas break that he would be leaving the club at the end of the season.
The writing was on the wall for Allegri when club director Barbara Berlusconi, daughter of club president Silvio, told ANSA that Sunday's performance was unacceptable.
"It was a disappointing evening that confirms it's necessary to change," she said. "It's not tolerable for our fans to watch unacceptable performances such as this."
Allegri took charge in 2010 and led Milan to the Serie A title in his first season and second place the year after. His fortunes changed dramatically when the club sold several top players in the middle of 2012 to balance the books, saying they were putting the emphasis on youth.
Milan finished third last season but, plagued by injuries, have struggled domestically this term.
They are, however, the only Italian survivors in this season's Champions League and face Atletico Madrid in the last 16 next month.