Chelsea defender Terry, 31, is accused of calling QPR player Anton Ferdinand a "f****** black c***" during a Premier League game on October 23 last year.
He denies a racially aggravated public order offence, saying that he was sarcastically repeating the slur that Ferdinand mistakenly thought he had used.
Cole, also 31, told Westminster Magistrates Court: "I think we shouldn't be sitting here."
He said that while racism should never be tolerated, repeating what you thought someone said was 'completely different'.
Cole said: "If I repeated something that I thought you said, that's totally different than if someone just says something."
Cole described Terry as "a close friend", with whom he had played at Chelsea and for England for six or seven years. But he said there was a "conflict" because he is also friends with Ferdinand and his brother Rio.
He was asked by George Carter-Stephenson QC, for Terry: "Did you want to get involved in the situation?" "No."
The barrister asked about a statement Cole made after the match. Mr Carter-Stephenson said: "When you made the statement what was in your mind?" Cole replied: "The FA summoned me to do it."
Mr Carter-Stephenson continued: "Did you feel there was any conflict?" Cole said: "Yeah. JT is my team-mate and friend; Anton and Rio and his family, I've known for a long time."
Terry had said that he had been "keen" to speak to police about alleged racist abuse because "I knew there was nothing out there that would show that I had done anything wrong.
"I was keen to go forward with my police statement, my FA statement. If I had anything to hide I wouldn't have done that."
He said he made a statement before seeing any film footage of the incident.
"If there was ever any doubt in my mind, I could have held back, soaked it up and thought 'Let's wait until tomorrow, let's see what kind of footage is out there."
Earlier, prosecutor Duncan Penny put it to Terry that he had made a remark about having sex with Ferdinand's girlfriend, in response to taunts about his alleged affair with a team-mate's ex-partner.
Terry denied this.