The 41-year-old former Manchester City and Barnsley striker takes up his role at New York following previous coaching spells at Bury, Oldham and Liverpool and will work with academy manager Garreth Barker.
"It's something I've done before," said Sheron. "It's something I said I wouldn't do again unless it's a club with progression and this is a club that definitely looks like it wants to progress forward.
"From my first meeting with (CEO) Paul Douglas, and then with the manager and the lads here it's obvious these are all people wanting to do the best for Rotherham United and I want to be part of that team.
"It's a very interesting role and one I'm really looking forward to.
"It's a club with some really talented players in the system and I now want to try and help that and get some of the lads into the first team and making a couple of hundred appearances ideally.
"Hopefully I'll have a good part of the development that will get the lads through to the first team."