Rovers snatched the lead after 72 minutes through Colin Kazim-Richards' scrappy effort after Wojciech Szczesny had parried Martin Olsson's shot.
Arsenal pressed for an equaliser and goalkeeper Jake Kean somehow saved a close-range header from Theo Walcott before Olivier Giroud's effort in stoppage-time was blocked as Rovers clung on for a famous win.
Appleton - who started the season in charge at Portsmouth and then had a short spell with Blackpool - paid tribute to the efforts of his side and said: "To come to Arsenal and get through a cup tie is one thing but to keep a clean sheet and defend as well as we did is another and that is testament to our players.
"We had a feeling they might rest a few of their players because they have a big game on Tuesday in the Champions League. However, regardless of what side they were going to put out, we knew we would concede a lot of possession at times, but we are happy with that.
"We have a mentality now where we are hard to beat and are resilient. It has been almost getting into the heads of the players to take pride in keeping clean sheets and enjoy defending, which people do not like doing at times, we also have a bit of quality as well."
Kazim-Richards was once on the books at Arsenal as a youngster before beginning his professional career at Bury, with spells at Brighton, Sheffield United, Galatasaray, Toulouse and Olympiacos following before his latest loan move to Rovers.
Appleton said: "Kaz has been fantastic for the last five games and has played in a couple of different roles. I am delighted he got the reward for all of his work, it was not his greatest of strikes but I think he was just trying not to welly it into Row Z and got the bit of luck he deserved."
Appleton admitted it was some achievement to have enjoyed three cracks at the FA Cup this season with as many clubs.
"Everyone knows the trouble which came with the job at Pompey and I had a brief stint at Blackpool when I ended up with a 1-1 draw at Craven Cottage in the last round, then was lucky enough to win at Derby, but this is the icing at the top of the cake," he said.