Media reports said two fans were understood to have been waving an Albanian flag as Kolarov, who came on as a substitute in the game, warmed up on the side of the pitch in the first half at St James' Park.
Local police said in a statement, without naming any player, that they had received "a report of racist language having been used" at the stadium during the match.
"Northumbria Police take such allegations extremely seriously and inquiries are ongoing into the incident," it added.
Majority-Albanian Kosovo declared independence from Serbia, of which it was formerly a province, in 2008 after the two Balkan sides fought a war in 1998-99. Serbia rejects the secession. (Reporting by Alan Baldwin, editing by Ed Osmond)