American Stan Kroenke, the club's majority shareholder, paid tribute to Hill-Wood who joined the board in 1962 when his father became chairman.
"Peter has been a wonderful custodian of this club," Kroenke said in a statement on the club website.
"He has been a member of the Arsenal board for 51 years and his vision and steadfastness have been central to the successes of this great club over that time," he added.
Arsenal have remained one of England's leading clubs during Hill-Wood's tenure, a period which included the move in 2006 to the Emirates Stadium from nearby Highbury.
Hill-Wood, 77, had been in poor health after suffering from pneumonia and a heart attack late last year. His family connection with the club spans three generations, his grandfather also having served as chairman.
Hill-Wood no longer owned any shares in the club after selling to Kroenke.
Keswick, who is 73 and a former chairman of Hambros Bank, has been on the board at Arsenal since 2005 and is a lifelong fan of the club.
"He will provide further direction and support in our ambition to compete at the top of the game here and in Europe and to win trophies," Kroenke said of the new chairman.
Chief executive Ivan Gazidis is in charge of day-to-day operations at Arsenal, who are without a major trophy since 2005.