The Killie midfielder scored from the spot in the 26th minute after Gorgie defender Andy Webster had fouled the fleet-footed James Dayton. The Tynecastle men laid siege to the Kilmarnock goal at times only to be denied by dogged defending and some fine saves from goalkeeper Cammy Bell.
The Ayrshire club held on to move into sixth place in the table and McGlynn, whose side dropped to ninth, said: "Every game is very close, our 2-1 win against Dundee United at the weekend was very close as well but it is goals that win you games and we did everything today bar get the ball in the back of the net."
He added: "We have been scoring goals recently so I can't moan too much. I thought we dominated the second-half, we had plenty of chances but the ball didn't fall kindly enough for us to get a good clean connection on it in front of goal."
The best chance of the match fell to Hearts' second-half substitute Callum Paterson, who ran through one-on-one with Bell but by taking his shot early the Killie keeper was allowed to make a comfortable save.
McGlynn said: "It was a big chance for Callum. He had a similar chance against Ross County, I think, earlier in the season and he hit that one early as well. Hopefully he will learn because he did really well to get into that situation."
Hearts striker Gordon Smith did have the ball in the net in the first half but Steven McLean's whistle had clearly sounded for a Killie foul before his sweet volley from 20 yards landed behind Bell.
McGlynn said: "It looked like two of their players clashed together, it didn't look like it was a foul to me. I'm not sure it didn't have something to do with the fact that we didn't give them the ball back from the throw-in, I don't know if he was being generous there."
Kilmarnock assistant boss Jimmy Nicholl was gracious enough to admit the Ayrshire side had at times rode their luck. "It was backs-to-the-wall stuff in the second-half," said the Northern Irishman. "I thought we did reasonably well in the first half, I was quite enjoying it, there was good pace to the team and it was much better than it has been.
"Sometimes the opposition don't allow you to play the way the manager likes to play, so you have to give them credit. It's about how to win a game of football and we are just glad of the three points."