Hamilton was quickest going into the closing stages of Q3, while Rosberg had been fastest of all in Q2.
They ended up fourth and sixth on the grid respectively.
While polesitter Sebastian Vettel pitted for fresh intermediates for a final shot at the top spot at the end, the Mercedes joined the majority in staying out through Q3 on one set of Pirellis.
"Putting new tyres on was definitely the way to go," said Hamilton.
"Our strategy was to stay out, we thought it would dry up more and it work in our favour, but it didn't.
"I want to be on pole so initially when you get out of the car you don't think you're happy.
"But there are lots of positives to take out of yesterday and today so I'm excited about the race."
Rosberg was certain he had the pace to take pole had it stayed dry.
I was feeling really comfortable in the dry," he said.
"I was the fastest guy out there and had the best car. Then unfortunately the rain came, but still I thought 'OK, no problem, I can still go fast'.
"I just messed up the strategy, staying out there when others came in for a new set and I had damaged tyres."
The German is still convinced he can challenge for victory on Sunday.
"I have a good car, especially once it's in the race, and as we saw in Melbourne, seventh place won the race, so it's still possible," Rosberg said.
AUTOSPORT Malaysian GP coverage:
FP1 FP2 FP3 Qualifying Race Gallery Live commentary Official quotes