Bayern Munich's David Alaba put Austria ahead with a penalty before Janko struck in a devastating six-minute first-half spell for the hosts in their European Group C match where the two sides are battling for second place behind probable winners Germany.
Johan Elmander for Sweden replied with eight minutes to go as the visitors produced a late rally.
Zlatan Ibrahimovic was largely ineffective for Sweden as they suffered another blow following their goalless draw at home to Ireland in March.
Austria, who last qualified in 1998, have 11 points from six matches, three more than Sweden who have played a game less. Germany lead with 16.
Austria's opener began with a long kick upfield by goalkeeper Robert Almer which bounced off the back of a Sweden player and into the path of Martin Harnik who raced past a defender and was upended by Andreas Isaksson.
Alaba converted the penalty with a low shot which sent Isaksson the wrong way in the 26th minute.
Harnik was involved in the second goal when he floated over a cross from the right and Janko dived in front of his marker and met the ball just before it hit the ground with a header which rocketed into the corner of the net.
Alaba nearly added another with a 25-metre shot early in the second half before Austria handed the initiative to the visitors.
Ibrahomivic sprang into action as he wasted a free kick on the edge of the area by firing straight into the defensive wall, then had header saved at point-blank range by Almer.
With eight minutes to go, he flicked the ball into the path of Elmander who lifted it past Almer. Sweden had a penalty appeal turned down as they pressed for an equaliser in a dramatic finale.