Goss outsprinted rivals Juan Jose Haedo (Saxo Bank) and Tyler Farrar (Garmin-Barracuda) to take the historic win for his newly formed Australian national team - but the finale was marred by a nasty crash involving Britain's Cavendish (Team Sky) which almost brought about the withdrawal of race leader Taylor Phinney.
Launching his attack from deep, Cavendish saw his front wheel swiped by Roberto Ferrari (Androni Giocattoli) after the Italian suddenly veered to the right at top speed.
In the ensuing pile-up, American rider Phinney, winner of the race's opening time trial on Saturday, went down hard and looked to be on the verge of pulling out of the race.
Twenty-one-year-old Phinney entered an ambulance on the finish straight to receive treatment for an ankle injury before emerging a few minutes later to cross the line and take to the podium.
Cavendish did complete the stage but was forced to walk across the line with his bike over his shoulder and his red points jersey in tatters. The 26-year-old Manxman landed heavily on his left shoulder and was smashed into by two Farnese Vini riders as he slid across the road.
It was a dramatic end to an otherwise quiet day on the Giro's third, and final, stage in Denmark.
The day started with a poignant ceremony in memory of Wouter Weylandt, the Belgian rider who was killed on stage three of last year's Giro.
Once the 190km stage in Eastern Jutland got under way, a six-man group broke clear to build up a maximum lead of 3:35 on the road. One of the six escapees, Alfredo Balloni (Farnese Vini), consolidated his lead in the mountains standings by taking the sprint over the day's only fourth category climb midway through the stage.
Local rider Mads Christensen (Saxo Bank) soloed off the front of the leading group as the peloton reined in the escapees. Christensen led the race for the first of three 14.2km laps of Horsens before he too was swallowed up by the bunch.
The Sky team of Cavendish controlled the front of the race but allowed another Danish rider, Lars Bak of Lotto Belisol, his moment in the limelight for the second time in as many days. Having attacked on Sunday, Bak rode ahead of the bunch for the second lap of Horsens before being reeled in ahead of the business end of the stage.
Led out by Geraint Thomas and Peter Kennaugh, Cavendish was in a strong position approaching the final kilometre.
But the world champion soon found himself a little boxed in as the race hit the finishing straight. Spotting a gap, Cavendish launched his bid to pick up a second successive win on Danish soil.
A sudden change of direction from Ferrari, however, sent Cavendish sprawling into the tarmac - and almost ended youngster Phinney's hopes of taking the pink jersey into Italy, where the race will resume with a team time trial in Verona on Wednesday following an air transfer from Denmark.
With Cavendish out of the picture, Saturday's runner-up Goss coasted to a straightforward win to open GreenEdge's Giro account and seize the race's red points jersey.
Argentine Haedo pipped American Farrar for second, while the man whose dangerous sprinting will surely provide much debate for Tuesday's rest day, Ferrari, took fourth - although he was later relegated to last place by the race jury.
Welshman Thomas - second on the GC, nine seconds behind Phinney - led Team Sky's complaints over Ferrari's conduct, asking for the Italian to be expelled from the entire race "for crashing 10 guys at 60 km/h".
For Phinney and Cavendish - both battered and bruised - Tuesday's rest day has come at just the right moment.