Neil Lennon's under-strength side - without suspended captain Scott Brown and the injured Kris Commons - proved to be too strong on the night as a high energy policy managed to grind down a technically slick visiting side in a match of few chances for either side.
With Celtic tiring and down to 10 men, Lasse Schöne drove the ball into the corner of the net with the last kick of the ball, but there was barely time to restart the match before the final whistle sounded.
Barcelona lead the section at the halfway stage of the scheduled fixtures. Barca have seven points with AC Milan on five after the top two drew 1-1 in Milan. Celtic are next on three points with Ajax bottom of the table on one before they host Celtic at the Amsterdam ArenA on November 6.
45+1' - FORREST FIRES CELTIC AHEAD: Forrest slides the penalty high into the net. Superb penalty from Forrest. Right in the corner. Jasper Cillessend guessed right by diving to his left, but no chance of saving that. Celtic lead 1-0.
53' - FORSTER HEROICS: Brilliant stop by Forster. Had to make that save as he came off his line to stop from Thulani Serero. Superb move from Ajax.
54' - KAYAL FINDS CELTIC'S SECOND: Celtic move 2-0 clear. Kayal sees a shot deflect off Denswil before wrong footing Jasper Cillessen in the Ajax goal.
57' - AJAX BLOW BIG CHANCE: Kolbeinn Sigthorsson knocks the ball wide from inside the home area after a shot had broken to him kindly. Should really have hit the target there.
66' - KAYAL ALMOST ADDS A THIRD: Kayal with a surging run for Celtic. Had options, but goes for goal. Straight into the arms of Jasper Cillessen. Easy enough take in the end for the goalkeeper.
78' - CELTIC LIVE DANGEROUSLY: Another chance for Ajax. Ball breaks to Thulani Serero, but he lifts the ball over the bar from 12 yards. Should really have made that count. Slack by Celtic in defence.
86' - BITON SEES RED: Biton has been sent off for a two-footed late challenge on Serero, who is forced to walk off for treatment. Celtic down to 10 men.
89' - SCHONE SHOT: Schöne unloads a shot from distance. Forster does superbly well to get down and make the latest of his fine stops.
90+4' - AJAX GRAB GOAL: Schone wallops a shot into the corner of the net. Ajax pull one goal back, but it's too late.
All 32 teams in the Champions League this season have now conceded, Barcelona & Schalke were the last to succumb.
Celtic have only suffered four defeats in 26 Champions League home games.
Celtic had lost their last four Champions League games without scoring a single goal before scoring here.
Celtic have ended a run of 413 minutes without a goal in the Champions League thanks to James Forrest's penalty against Ajax.
TALKING POINT: Can Celtic win next time out in Amsterdam? Tough ask, but where there is life there is hope. Ajax are technically very efficient, but do leave space at the back. Celtic will need to produce more going forward as sitting back will only invite trouble.
Celtic: Forster 8, Izaguirre 7, Ambrose 6, van Dijk 8, Mulgrew 8, Lustig 7, Kayal 8, Samaras 7, Stokes 7, Pukki 7, Forrest 8..subs: Ledley 6, Biton 5, Baldé 6
Ajax: Cillessen 7, Van Rhijn 7, Veltman 7, Blind 7, Denswil 6, Poulsen 6, De Jong 7, Serero 5, Fischer 7, Sigthórsson 7, Andersen 7..subs: Boilesen 6, Schöne 7, De Sa 6
MAN OF THE MATCH: Fraser Forster (Celtic): Didn't have as much to do as against Barcelona, but produced the goods when needed. Just gets the nod ahead of Charlie Mulgrew, who can play almost any position well.
THE MATCH IN A TWEET