The best-of-seven series between the two neighbouring foes has now become the best-of-three and is living up to all the fiery expectations, with the teams and coaching staff trading insults and blows on and off the ice.
Three players, Mike Rupp and Stu Bickel from the top-ranked Rangers, and New Jersey's Ryan Carter, were each given 10-minute misconduct penalties after a brawl in the third period that began when Rupp knocked Devils goalie Martin Brodeur on his backside with a punch to the chest.
"I was minding my own business," Brodeur said.
"I think it's a good sign when they take liberties on players because it means they're off their game a little bit."
The Rangers' Ryan McDonagh and Devils' Adam Henrique were given five minute majors when they dropped the gloves in the opening period and squared off for the first fist-fight of the series.
New York captain Ryan Callahan and New Jersey's Russian winger Jersey's Ilya Kovalchuk continued to taunt each other from their opposing penalty boxes after they came to blows.
A total of 18 penalties, 12 against the Rangers and six against New Jersey, were issued during the hostile game, triggering an ugly spat between the opposing coaches, the Devils' Peter DeBoer and the Rangers' John Tortorella, who began shouting at each other during the final period.
The pair had already been bad-mouthing each other in the lead-up after New York forward Brandon Prust was suspended for elbowing New Jersey Devils defenseman Anton Volchenkov in Game Three on Saturday, but neither wanted to talk about the latest incident.
"This isn't about John and I, this is about the guys on the ice," said DeBoer. "We're trying to win a series and get to the Stanley Cup, we don't need any extra fuel.
"You've got two competitive teams, there's a lot at stake, they're battling to the buzzer, that's the way I saw it."
Unlike Saturday's third game, when sixth-seeded New Jersey failed to score despite firing 36 shots at the New York net, the Devils took their chances this time.
They finally found a way past the impenetrable defense of New York goalie Henrik Lundqvist, who recorded shutouts in two of the previous three games, when Brian Salvador scored with a wrist shot then Travis Zajac netted a slap shot less than four minutes later in the opening period.
New Jersey captain Zach Parise added two more goals in the final period, the first off a power play and the latter into an empty net when Lundqvist was pulled after the Rangers had scored a consolation goal through Ruslan Fedotenko five minutes from the end.
"It was a struggle for a number of our guys but we have to move on and have a short-term memory," said Tortorella.
"I'm truly confident that we'll respond the right way because we've done it all through the play-offs."
There is never any love lost between the teams but with a place in the Stanley Cup final at stake, the animosity has been steadily building with each game, and there are no signs of it letting up with the sides heading back to the cauldron-like Madison Square Garden on Wednesday for Game Five.
"Throughout a seven game series, both teams are going to get frustrated at certain things," said Parise.
"I think the intensity is only going to rise from here, there's a lot at stake, and we're up for the challenge."