Hamdi Salihi netted the only goal of the game on 67 minutes at a freezing Ullevaal Stadion and despite ending the game with 10 men following the sending off of Andi Lila in the final minute, the unfancied visitors held on for a famous win.
Albania sit second in the table, with nine points from five games played. Defeat for Norway dented their chances of reaching the finals in Brazil and they lie in fourth on seven points.
In the night's other game in the group, Iceland downed Slovenia 2-1 Ljubljana.
Slovenia took the lead on 34 minutes through Milivoje Novakovic but a second half brace from Tottenham midfielder Gylfi Sigurdsson saw Iceland overturn the deficit.
Iceland are level on points with Albania while Slovenia are bottom, level on three points with Cyprus, who have played a game less.