The 24-year-old centre-back, a free agent after leaving the Kassam Stadium, becomes Justin Edinburgh's third signing of the summer after he snapped up two other former U's players in Adam Chapman and Chris Zebroski.
"Harry is a player we have had on our radar for some time and I'm thrilled that everything is now complete," Edinburgh told the club's official website.
"There are a couple more targets I would like to add, but as in the past I will not name them until its done and dusted and the ink is dry."
Worley began his career with Chelsea and was loaned out to Doncaster and Carlisle. Spells at Leicester, Luton and Crewe followed before he spent three years with Oxford.