Rovers outscored their opponents five tries to four, with Travis Burns, Kris Welham, David Hodgson, Josh Hodgson and Craig Hall all crossing for the hosts.
Kirk Dixon, Jake Webster, Richard Owen and Daryl Clark were the try-scorers for Castleford, who came up just short despite holding a 16-14 lead at the interval.
Burns opened the scoring for Rovers in the fifth minute thanks to a bit of luck that has been missing from their season so far. His initial kick bounced off Rangi Chase back into his own hands and, although he still had work to do, he managed to ground the ball.
Welham then extended their lead when a turn of speed took him behind the Castleford defence, although Burns missed the conversion.
It took a piece of opportunism from Dixon to register Castleford's first score of the game. Dave Hodgson made a mess clearing Jordan Tansey's kick to give Dixon the chance to score and then add the conversion to make it 10-6 after 22 minutes.
Rovers' response was impressive. On the back of three successive penalties, the home side spread play from middle to left for Dave Hodgson to score in the corner. Burns was off target with the conversion but Rovers still held a 14-6 lead.
However, the Tigers ended the half on top thanks to two glaring errors from Greg Eden inside the Rovers' full-back own half. The first was punished by former Robins centre Webster and Owen put the visitors in front for the first time with his touchdown coming after Eden dropped a high bomb. Dixon added the conversion to send the Tigers into the break ahead.
However, Rovers regained their lead seven minutes after the break, Josh Hodgson with Burns adding the extras, and they extended their advantage in the 53rd minute when Tansey tried to gather Burns' kick through but succeeded only in scooping the ball up for Hall to score near the posts. Burns' conversion left Castleford trailing 26-16.
Another Eden error allowed Castleford back into the game, though. The full-back allowed Tansey's kick to bounce and Clark capitalised by collecting the ball and diving over near the posts. Dixon's conversion made it 26-22 with 22 minutes to play, but neither side could add to their tally.