Wharton, who has been on the board at Glanford Park for over 20 years and been chairman for a decade, is leaving his position, with Peter Swann set to take up his role after joining the board on Monday.
Wharton leaves the Iron with them suffering relegation from League One this season.
"I made it clear to my colleagues on the Board at the end of last year that I had lost my drive and enthusiasm for the game and that we must have a new chairman," he said in a statement on the club's official website.
"I was persuaded to stay in office until the end of the season, primarily to avoid making our bankers nervous at a time when we were struggling for finance.
"The season has now ended and regrettably I bow out on a very low note.
"I have always done my utmost to make Scunthorpe United successful without compromising our long-term viability.
"Our club is well respected throughout the football world and it is only the savage cut in income caused by the drop from the Championship that has caused us to become somewhat insecure.
"Fortunately we have a strong balance sheet and with careful financial management we will regain our enviable reputation as a well-run club.
"My sincere thanks go to my fellow directors, all the staff and all our supporters for making my 20 years at the club so memorable."