"I've worn the leader's jersey in the Giro (d'Italia) and Vuelta a Espana, I'm just missing the yellow jersey," the Manxman told a news conference on Thursday.
Cavendish, who won his first British road race title last weekend, is targeting the green jersey for the points classification on the Tour but with the opening stage being designed for sprinters, he is a natural candidate for a brief spell in yellow.
"For sure, I won't be wearing it in Paris but I'd just be wearing it," the OmegaPharma-Quick Step rider said two days before the peloton tackle the 213 kilometres from Porto Vecchio to Bastia on the island of Corsica.
Cavendish will battle it out with Slovakian Peter Sagan for the green jersey but said a spell in yellow would be a dream come true.
"The yellow jersey is not just one of the most iconic symbols in cycling, it's one of the most iconic symbols in the world of sport," said Cavendish, who has won 23 Tour de France stages.
"I think to be able to wear that for a day in your life, it's a thing to make any rider's career, it's a thing you dream about as a child; it would be a beautiful thing."