The 56-year-old will act as number two to Pat Fenlon, having worked for two years under Kenny Shiels, with whom he won the Scottish Communities League Cup in 2012, before his fellow Northern Irishman left Killie by mutual agreement earlier this week.
Nicholl was also assistant at Aberdeen for five years from 2004 and 2009 and Fenlon is delighted to welcome someone with vast experience of the SPL.
"I am delighted to have such an experienced coach as my assistant. Jimmy has a deep knowledge of the game and of Scottish football in particular and will be a great asset to the club," said Fenlon.
"He is particularly good in the development of young players and this is another important factor for the club."
Nicholl, who won 73 caps for Northern Ireland during his playing career, has been a manger in his own right at Millwall, Raith Rovers and Cowdenbeath but is looking forward to the new challenge at Easter Road.
"When Pat spoke of his plans for the club it was an easy decision for me," he said.
"I am delighted to be his assistant and hope to contribute to the on-going development and success of one of Scotland's leading clubs."