On Thursday, Brown announced he would retire from management at the end of the season but denied his decision had anything to do with Aberdeen's ninth-place position in the Clydesdale Bank Premier League.
Brown insists the Dons can still avoid a fourth consecutive bottom-six finish with three games left until the split, beginning with Saturday's trip to Celtic Park.
But he previously denied suggestions that another season in the bottom half would constitute failure to a club that was accustomed to league titles and European trophies under Sir Alex Ferguson in the 1980s and cup success under Alex Smith soon after.
Brown, who joined the Dons from Motherwell in December 2010, said: "The expectation is great here and there is a justification historically for that but there is not a justification in the immediate past.
"When I was at my previous club, we didn't think Aberdeen was a threat. In fact we came up here the last time and won 3-0 with a young team from Motherwell.
"Motherwell are not doing that now. They are coming up here and delighted with a draw now, and they are the second best team in the country. When we came up three years ago with Motherwell, we came expecting three points, let's be honest.
"I'm hoping that's changed. Motherwell people will tell you that's changed and I think that has changed in other clubs. So to say it's a poisoned chalice is a bit harsh.
"Aberdeen had the status everyone thinks when the two Alexes had the job. Jimmy Calderwood did very well here too. Jimmy could buy players though, I couldn't buy a player. We were in a selling situation.
"Please believe me, the other clubs don't have the perception of Aberdeen that people have here. They don't. I was at other clubs."