The DR Congo international attracted unwanted headlines last weekend for his red card against West Ham after kicking out at midfielder Gary O'Neil. However, Clarke knows the value to the Baggies of the former Paris St Germain player, who joined Albion four years ago.
Mulumbu has 15 months remaining on his current deal although West Brom have a further year's option in their favour. Albion have already started negotiations with striker Shane Long and defender Gareth McAuley about extending their spells at the Hawthorns. Clarke said: "We are speaking to Youssouf, Shane and Gareth about new contracts."
He added: "There are one or two others behind them as well (to hold talks with) but we can't speak to everyone at the same time.
"Youssouf has been excellent for us since I have been here and is one of the players we want to keep."
Clarke is also confident the players he wants to stay at West Brom will still be at the club next season.
He said: "I think come the summer the ones that we have to keep - that I want to keep - will still be at the club."
Albion have had three players red-carded this season in Peter Odemwingie, Goran Popov and Mulumbu, and Clarke believes all three sending-offs should have been avoided.
He said: "We have had three red cards this year. All three were avoidable so it is something we have to address. Peter at Fulham, Goran against Tottenham and Youssouf against West Ham - all three are the fault of our player.
"Everyone makes mistakes. If you don't make them, then you don't do anything. They all apologised straightaway. You try to move and we try to make sure it doesn't happen again in the future."