Khachanov, 17, did not face a single break point as he picked apart the Serbian - who appeared to struggle with an injury - to earn a 6-4 6-4 win in just 77 minutes.
The little-known Russian was ranked 808th in the world prior to the tournament with his only title of note coming at the European Junior Championships in July when he dropped just one set in six matches.
The world junior number 19 reeled off 14 aces against the Serb, who was ranked eight in the world last season but has suffered from illness and injury and is currently ranked 27th.
"I heard Tipsarevic is injured, and I saw it at the beginning of the match," Khachanov said afterwards. "And the rhythm of the play was absent, so it was hard.
"He was trying to get winners from the first ball. My serve saved me. Of course, he wasn’t in his best shape, so this is why I wasn’t too happy at the end.
"After the match I asked him if he remembered me as we hit together during the Kremlin Cup two years ago, when he won it. He answered: 'Yes, of course. Congratulations, you’re playing so well'."
Eurosport Russia commentator Alexander Kalivod described Khachanov as "a Russian Del Potro, with great serves and forehand".
While Khachanov does not have overly impressive junior results apart from his aforementioned continental title, he beat Romanian star Victor Hanescu in the first round at St Petersburg last month.
Defending champion Andreas Seppi – the number two seed – strolled past Paolo Lorenzi in an all-Italian affair 6-3 6-2 to set up a quarter-final match with Edouard Roger-Vasselin who beat Sergiy Stakhovsky 7-5 7-5.
Qualifier Andrey Golubev was the last man to book a last eight spot on Wednesday, overcoming sixth seed Horacio Zeballos 6-3 6-4.