Substitute Odejayi coolly picked his spot from 10 yards after Steve Mildenhall had parried Alex Revell's 60th-minute drive.
And before the home side could recover former Bristol City player Odejayi set up Frecklington to smash a volley into the roof of the net from 12 yards.
Joe Anyinsah replied for Rovers with a swerving right-footed shot from the corner of the box with nine minutes left.
A scrappy first half had seen Frecklington waste the best chance when lobbing Mildenhall and clearing the crossbar.
John-Joe O'Toole shot wide for Rovers and Tom Parkes headed over from a corner, but the visitors edged the opening 45 minutes.
After the Rotherham goals Rovers sub David Clarkson forced a fingertip save from Andy Warrington.
