Killie were raging when teenager Ross Barbour appeared to have squeezed the ball over the line in the 37th minute only for referee Calum Murray to wave play on after Lukasz Zaluska retrieved the situation.
Their frustration was compounded three minutes from the interval when substitute Joe Ledley fired the home side into the lead with a long-range drive.
Former Celt Cillian Sheridan levelled two minutes after the break before Adam Matthews arrowed in a shot from 25 yards and after Anthony Stokes, on for Kris Commons, grabbed a third, the Wales international sealed the win from close range.
He said: "The ball was apparently over the line, I don't know but our boys were going absolutely ballistic because the boys in the box swore it was over the line.
"Barbour came in the back and scored and the kid was adamant the ball was over the line, that's why our boys were going ballistic.
"I was disappointed for them because it should have been a goal but there were positives apart from the result in the end."