A statement on the Shakers' official website cited personal reasons as being behind the 28-year-old midfielder's decision to see a move south.
Sweeney joined Bury on a free transfer from Grimsby in August 2010 and scored five goals in 95 appearances, with the last of those coming as a substitute in the FA Cup penalty shoot-out defeat at Southend last month.
"Peter left the club on Monday afternoon after signing off the release on his Shakers contract, and has now taken up a contract with League Two AFC Wimbledon," read the Bury statement.
Youngsters Liam Boswell and Luke McCarthy have also had their contracts mutually cancelled.
