Apart from a second quarter where they scored 35 points to go in level at half-time, the Celtics struggled to contain Miami, particularly the key pair of LeBron James and Dwyane Wade.
"When we took our time in the second quarter we got everything we wanted and then we went back to the way we started the game in the third quarter," said Rivers.
"On the road you just can't have two quarters of lulls. You just can't afford it and I thought we did that."
The Boston coach said it was no surprise that Wade (22 points) and James (32 points) dominated but was annoyed with the way his team handled them.
"You're not going to take everything away from them. They have two sensational players but in my opinion we let Wade, we let LeBron play in extreme comfort and we gave the other guys everything they wanted as well.
"So we'll go back and see what we can do better and we'll fix it.
"They're cat quick ...but as much as you can watch it and prepare for it, I do think you need to face it once to see it and get used to it.
"I thought their speed at times overwhelmed us for sure."
Rivers was unhappy with the way his team were out-rebounded 48-33 despite Miami only shooting at 50 per cent from the field.
"They shot 50 per cent from the field and they still out- rebounded us on the offensive end," Rivers added.
"If a team shoots 50 per cent they can't also have more offensive rebounds than you."
Regarding the Celtic's offense, Rivers said he wanted his point guard Rajon Rondo to play a less calculated game.
"I thought he was reading a lot instead of playing on his instincts. I think sometimes his IQ hurts him. He's trying to read the defense.
"You can't read and play at speed at the same time and we go through it a lot. I told him 'just trust your instincts, your speed has to be part of it. Your instincts will take over and you'll make the right decision."