Green was criticised for publicly revealing that he calls business partner and former Rangers director Imran Ahmad "my Paki friend".
The Yorkshireman responded by insisting that he "deplores" racism but did not apologise for the comments, which he made during a Sunday newspaper interview.
Show Racism the Red Card chief executive Ged Grebby said: "The comments of Charles Green are very ill-advised and someone in his public position should really know better than to use this kind of racist and offensive language.
"The use of the term 'Paki' is highly offensive and it is something that we at Show Racism the Red Card are trying to educate young people against. For the CEO of Rangers to use the term in any context is not helpful as it sends out the wrong message to young people."
In a statement to Press Association Sport, Green said: "It is unfortunate that Show Racism the Red Card has fallen into the trap of taking something out of context and giving a knee-jerk reaction. In the interview, I referred to the fact we are now living in a politically correct world where words and phrases that were once used are no longer acceptable.
"My use of the word 'Paki' was when I was referring to how I sometimes warmly address my colleague and friend Imran Ahmad when we meet, something we both know is light-hearted. I deplore racism and sectarianism and this football club does a huge amount of work in this area to help eradicate these problems."
Green's outspoken nature has put himself at the centre of a number of controversies since taking over at Ibrox last summer. The racism row came days after Green found himself in the spotlight over his failure to explain the extent of his dealings with former Rangers owner Craig Whyte ahead of the purchase of assets in June.
But on Monday night he admitted he might have gone too far after repeating his view that this was the worst Rangers team of all time, during the same interview as his comments on Ahmad.
Green told STV: "Perhaps I shouldn't have said this was the worst Rangers team ever. Perhaps that wasn't a good thing to say. Was it true? Was it misleading? It wasn't said to win fans over. It wasn't said to win anyone over. The press say Charles Green says this to keep the fans onside. I don't."