Toulon, who were aiming for a Heineken Cup-Top 14 double, lacked creativity, and Jonny Wilkinson's nine points were not enough as South African scrumhalf Rory Kockott scored a try and kicked eight points and Remi Tales added two late drop goals for Castres.
Delon Armitage scored a consolation try for Toulon.
"I can't find the words. It was David against Goliath today," Kockott told France 2 television, praising Castres' team spirit.
"We are a team in which there is no star, noboby is better than the others," he added.
Wilkinson and Kockott traded penalties early on in an uninspiring first half.
The only spark came when Kockott caught Toulon's defence off guard as he scooted between the posts before converting the try himself to give Castres a seven-point lead on the stroke of halftime.
Wilkinson kept Toulon in the match with two more penalties but would have been ruing his misfortune after watching two of his attempts hit the posts.
Tales's 72nd-minute drop goal pushed Castres four points clear and there was more breathing space when he repeated the dose under pressure just minutes later.
Kockott put the result beyond doubt with a 40-metre penalty with time almost up.
"It's hard to take in, we did not play the way we wanted," England's 2003 World Cup winner Wilkinson said of Toulon's performance.
"We need to keep the same spirit next year and learn from the lessons from today. We were not good enough to win."