The Rebels charged out to a 21-0 lead by the 22nd minute at home but trailed 29-27 with eight minutes to play as the Force ratcheted up the pressure with two second-half tries.
Mark Gerrard slotted home a penalty in the 73rd minute to snatch back the lead and the Rebels defended grimly to notch up their first win of the season after being denied the previous week by an 81st-minute try against the Cheetahs.
"It's a great feeling," said Rebels captain Gareth Delve. "It was a bit of a hairy moment right there at the end, a bit of deja vu... To get the win means a lot to us and we can kick on from here."
The Rebels stunned the Perth-based Force with three early tries, the first from outside centre Lachlan Mitchell who bustled over by the right post in the eighth minute after receiving a sharp, flat pass from fly-half Cipriani a few metres from the line.
Inside centre Lloyd Johansson got the Rebels' second seven minutes later, bursting through the line with a deft sidestep after linking up with fullback Julian Huxley.
Gerrard received after a turnover deep in defence and burned down the right touchline to set up the Rebels' third, the winger giving two defenders the slip before steering a neat pass inside to the charging Cipriani who bolted 40 metres to cross beneath the posts.
Cipriani appeared to injure a hamstring during his dash over the line and came off soon after, stalling the Rebels' momentum.
Force captain David Pocock capitalised on a howling mistake by winger Cooper Vuna, who toe-poked a loose ball near the try-line that the flanker simply scooped up and crossed for the softest of tries.
Winston Stanley dived over near the left corner after taking a long, lofted pass hurled out of a ruck by scrumhalf Brett Sheehan in the 33rd minute to score his first Super Rugby try and bring the Force within 12 points.
Winger Alfie Mafi thundered through a gaping hole in the Rebels defence to cross under the posts four minutes after the break and fullback David Harvey slotted home a penalty as the Force clawed back to trail 24-22.
Huxley steadied the Rebels' ship with a 56th-minute penalty, but Nick Cummins eluded a clutch of defenders seven minutes later with a withering run from outside the 22-metre line to score the Force's fourth try and secure a bonus point.
Harvey's conversion gave the Force a 29-27 lead and Rebels fans groaned as they contemplated a seventh straight loss on home soil, but Gerrard kicked truly to take back the lead and booted the ball into touch after the whistle to seal a nervous win.