Jamie Ellis kicked two penalty goals in the last six minutes as the Tigers hit back from 12-10 down to snatch their first victory over Leeds for three years. Castleford led 10-6 at half-time after Jordan Tansey and Jake Webster scored first-half tries but Ryan Hall's 49th-minute score looked set to give champions Leeds the edge until Ellis's boot decided the outcome.
"I am really happy for the players. They have worked really hard and the new guys that have come in have been really good. I thought it was a real shot in the arm for the effort the players have put in," said Millward.
"I thought we were very impressive in the wet conditions in the first 30 minutes. I thought our halves Rangi Chase and Jamie Ellis were very good in the first half and I thought Jordan Tansey was excellent. I felt it could have been a lot closer last week in our defeat at Warrington but I was really impressed that we had a bit of backbone about us that we didn't have last year."
Leeds coach Brian McDermott blamed "unfortunate errors" for his side's first defeat of the season. "Well done to Castleford. Both teams put in a really big effort," he said.
"I haven't really got too much negative to say about us but I think I will be showing the players in terms of what we need to do better. I think the game came down to some unfortunate errors. I don't think in general Castleford handled the conditions better than us - it was just unfortunate mistakes towards the back end that cost us.
The Rhinos picked up injuries to Ryan Bailey (knee), Brett Delaney (hamstring) and Zak Hardaker (concussion). Hardaker was taken off in the 64th minute after being flattened by Castleford's Rangi Chase.
"I am not sure how long they will be out for," added McDermott. "I don't think Delaney will be less than a week and we are not sure what the tweak is with Bailey's knee, whether it's a little nip of a cartilage or a bit of strain.
"Hopefully Zak will recover from his collision. I'd be wary of commenting on that incident but in general shoulder barges are all right - that has what this competition has ruled - but if it's head contact, it's head contact. I'm not saying it is with Rangi Chase - I'll have to look at it - but he knocked the kid out clean."