Defender Brad McKay notched the decisive spot-kick after Ross Callachan saw his effort saved by substitute goalkeeper Mark Ridgers.
But Jamie Hamill netted from the spot 12 minutes later and was involved in the incident that led to Raith's Dougie Hill being red-carded in the 74th minute, before the high drama of penalties settled the issue.
Following a scrappy start, Hearts midfielder Jason Holt registered a shot on target after six minutes when his low drive from 25 yards forced David McGurn into a smart save.
Raith striker Elliot then had appeals for a penalty waved away by referee Brian Colvin after tumbling in the box with Hamill in hot pursuit.
Kevin McHattie picked up a yellow card for a late challenge on Grant Anderson, and from a Callum Booth set-piece defender Douglas Hill nodded over the bar.
Jamie MacDonald got down well to a left-footed Joe Cardle effort at the near post after 22 minutes but it proved to be the Hearts goalkeeper's last piece of action. The shot-stopper limped off moments later and was replaced by Ridgers.
The substitute goalkeeper was called into action moments before the break to make a diving save to palm away a rising Elliot drive from 30 yards.
Raith were on the front foot at the start of the second half and went ahead through Fox.
A dangerous Elliot run into the box caused panic and Ridgers produced a great save to block a Gordon Smith shot from close range but Fox found a gap in a congested box to fire a left-footed effort into the net.
Hearts equalised from the penalty spot in the 62nd minute after Jason Thomson brought down Billy King inside the area.
Hamill stepped up and sent McGurn the wrong way from 12 yards with a low shot.
The Hearts midfielder was involved again as Raith were reduced to 10 men in the 74th minute after Hill was shown a straight red card for appearing to heat-butt Hamill.
Extra-time came and went without either team managing to break the deadlock. Hearts twice wasted the chance to settle the tie on penalties when Holt and David Smith failed to convert but McKay finally won it in sudden death.
- Sports & Recreation
- Jamie Hamill
- Mark Ridgers
- Liam Fox
- Douglas Hill
- Calum Elliot