The Lions took the lead in the 52nd minute when Worner gathered a low return pass but released the ball fearing his momentum would take him outside the box, allowing Keogh to slot home into an empty net.
Manager Steve Lomas made seven changes to the side that lost to Yeovil at the weekend and it was one of those in Jermaine Easter that should have broken the deadlock in the first-half.
In the 28th minute the striker picked the ball up on the edge of Wimbledon's box and unleashed a curling effort heading for the top corner but Worner was equal to it.
Easter went close again after 34 minutes when Nicky Bailey, making his full debut, found him with his back to goal in the area but his low effort was easily blocked.
Early in the second-half James Henry fired over from six yards out and after gifting Keogh the opener, Worner denied Martyn Woolford with a smart save with is legs.
In the 77th minute though Woolford fired home a deflected Easter cross as Millwall were cruising through, but substitute Luke Moore had other ideas and nipped in at the near post in the 89th minute to slot past debutant Stephen Bywater.
But Bywater was then equal to a stoppage time Jack Midson effort to ensure Millwall's path into round two.