Ritchie, a batsman who played for Australia between 1982 and 1987, made the remarks when speaking at a lunch held by the Gabba Members' Club on the first day of the first Test on Friday.
CA contacted Ritchie, who admitted making the comments, and he was reminded about the organisation's anti-racism policy.
CA has also stated it does not think it appropriate for Ritchie to speak at any of its international venues.
The governing body has also been given assurances by Queensland Cricket and the Gabba Members' Club that Ritchie is not due to speak at the venue for the remainder of the Test.
A CA spokesman said: "We are very disappointed at comments Greg Ritchie has made at the luncheon. A senior CA official has contacted Greg Ritchie, who has admitted making the comments, and CA has made it clear to Mr Ritchie that it finds his comments totally unacceptable.
"QC and the Gabba has confirmed that Mr Ritchie is not booked to speak at the venue for the rest of this Test. We have spoken to QC and the Gabba Members' Club and reminded them of the obligations under the ICC Anti-Racism Policy for International Cricket and of the Australian community expectations of speakers that speak publicly within our venues."
CA is also writing to remind all of its member states and venues about its support of anti-racism initiatives.
The spokesman continued: "Cricket Australia believes there is no place for racism in sport, on or off the field. We fully support the ICC Anti-Racism Policy for International Cricket.
"We regularly carry messages as per the (policy) on our big screen at matches, so all our spectators clearly understand their obligations under it. CA believes that it is not appropriate for Mr Ritchie to be speaking at our international cricket venues at present."
- Sports & Recreation
- Greg Ritchie