The on-loan Yeovil forward stretched to convert Lewis Alessandra's cross from the left for a close-range 78th minute winner.
Lavery latched on to Luke Young's through ball down the left before cutting inside and cleverly clipping the ball past Bury keeper Brian Jensen.
Jensen denied Lavery minutes earlier by palming over the striker's point-blank goal-bound header following a corner.
Home keeper Luke McCormick made an equally good save to deny Danny Mayor as he latched on to a return pass from Sedgwick in the 33rd minute.
Bury began the second half well with Daniel Nardiello cutting inside and beating McCormick with a shot that came back off the inside of the post.
In the next attack Sedgwick scored with a brilliant measured lob from the right after spotting McCormick off his line but it proved to be just a consolation.