The 21-year-old was Holloway's first piece of business in the transfer window and it paid early dividends as he grabbed the winner in the 40th minute.
The right-back, who only joined the Lions on a three-month loan deal on Friday, saw his over-hit cross float over the head of Ipswich goalkeeper Dean Gerken and go into the top corner.
Martin was a nuisance early on as he jinked past Luke Chambers on the left, but Scott McDonald failed to get his cross under control and Ipswich defender Smith hacked clear.
Liam Trotter almost curled the Lions ahead on 17 minutes when his shot from the edge of area sailed just wide of Gerken's left-post.
The midfielder then released Scott Malone on 25 minutes but, with three Millwall attackers piling into the area, his cross was too high.
A minute later Martin, an Ipswich player last season, wasted a great opportunity for the Lions when he fired tamely at Gerken, having outpaced Chambers to a loose ball.
But a minute later the Millwall keeper had to be more alert to keep out Daryl Murphy's fierce strike after the forward had collected McGoldrick's neat flick.
Ipswich were beginning to settle and Aaron Cresswell was next to test Forde when he met Carlos Edwards' drilled cross with a bouncing volley.
Millwall defender Shane Lowry took aim from 40 yards on 37 minutes, but it was easy for Gerken to hold on to his looping drive.
But, as the clock hit 40 minutes, Gerken was helpless to prevent Fredericks from scoring on his debut.
McDonald fired wide a minute later as the hosts ended the half on the up.
Early in the second half Smith met Ryan Tunnicliffe's corner with a powerful header, but Martin was well placed to nod the Ipswich defender's effort to safety.
Danny Shittu got in an important block five minutes later after Cresswell put through Murphy and a Martin free-kick drifted just wide of Gerken's post on 63 minutes.
A minute later Malone fizzed the ball across the face of goal and, with 20 minutes to go, Martin should have made the points safe.
However he smashed over from 10 yards, having cut inside of Christophe Berra, with Ipswich not troubling Millwall thereafter as the Lions recorded their first win since early December.
- Sports & Recreation
- Dean Gerken