Fraser, who won two Clydesdale Bank Premier League young player of the month awards this season but has not yet played 90 minutes, signed for the npower League One club on Friday after the Dons accepted an undisclosed bid in midweek.
Fraser rejected a new contract offer last month, and Aberdeen felt they had no option but to accept the bid rather than wait for a compensation tribunal in the summer.
Brown, whose side face Inverness in the Clydesdale Bank Premier League on Saturday, said: "I'm disappointed that we're losing him but he seemed anxious to go and he was advised his future was elsewhere. We had resigned ourselves to losing him."
Brown had hoped there might have been a rethink after Fraser returned from a six-week injury absence in the December 29 win at Dundee
"He got a good response from the fans when I put him on at Dundee," Brown said.
"I thought that maybe because the fans were so good to him that he might have reconsidered the decision but he didn't.
"We are used to losing players and you just have to shrug your shoulders. I think he will flourish. I will not be churlish and wish him anything other than the very best.
"I'm sure he will flourish, unfortunately it won't be in the red jersey we have got here.
"It's a club on the up and up. They are in the third tier of English football but it's a good level of football and they are an ambitious club."