Haye, 32, faces Fury in a domestic super-fight in Manchester on September 28.
The Londoner, a former world heavyweight champion, says he once considered signing Fury to his promotional company but was left underwhelmed by his 2009 bout with John McDermott.
"I watched him live when he fought John McDermott the first time in 2009 and I watched him live as an amateur before that," Haye told the Daily Star. "I remember being really unimpressed.
"Me and my trainer Adam Booth were looking at signing young fighters. We looked at George Groves, Michael Maguire and Tyson Fury.
"They were the three standouts and we signed up Maguire and George. We gave Fury a swerve. I walked out after watching him for a couple of rounds.
"I could have been his promoter. It would be a bit presumptuous to say he would have signed, but I wasn't interested."
The unbeaten Fury, meanwhile, responded by claiming Haye was rude to him when they first met – and that the Hayemaker "obviously made a mistake" by writing him off.
"I was just a bit disappointed at the reception I got when I first met him, from a fan to a champion," the 25-year-old Fury said.
"I went to shake his hand afterwards and say: 'Hello, David.' He just snubbed me and walked away.
"He didn't say 'good fight', or 'how are you?' – nothing. I'm nice. But obviously he wasn't in the best mood. But it doesn't really bother me.
"If a kid comes over to me, especially an amateur fighter, and says 'hello Tyson', I don't just go 'you what?'
"I go 'hello, how are you doing, do you want a photo?'
"Obviously they made a mistake by not signing me. They signed David Price, used him as a sparring partner for a little bit, busted him up and then gave him his contract back.
"I wouldn't have (gone) with Hayemaker anyway. I wouldn't have somebody as my promoter who is in the same division as me.
"It would have been a bit weird fighting the boss but we all know who will be boss come September 28."