Junior Stanislas marked his first start since November by scoring the only goal of the game with a brilliant strike into the top corner from outside the box with nine minutes to go.
The Clarets survived a late scare as Lee Grant got down brilliantly to turn the ball away after Jason Shackell had diverted Jonathan Parr's cross towards his own goal and Mile Jedinak hit the crossbar in stoppage time, but Dyche's side held on.
He said: "I thought our football was excellent but it was a tight game and we did well to win it.
"We asked the players to give a lot in their performances and they did that once again.
"I am sure Ian Holloway will argue that Palace should have got something from the game and if you isolate the second half maybe he is right, but over the 90 minutes I think we deserved to win.
"We scored a fantastic goal and I am delighted for Junior, but the whole team worked hard for it.
"Football is swings and roundabouts and I think we have been hard done by at times but when we apply ourselves then we don't have to worry about silly decisions and we can challenge anyone in the league.
"We need to improve our consistency now and see if we can get more wins on the board. If we can do that then who knows where it could take us."