Rooney regained the lead for Oldham in the 56th minute with a simple tap-in and James Dayton sealed victory with 14 minutes left with a super strike.
"Key decisions and key moments define games," said a disappointed Adams.
"There was the penalty save and the referee's decision (a rejected penalty appeal for handball) which didn't go our way.
"I fancied us and at half-time I was pleased as there was a good tempo to our play.
"We faded away in the second half, which was disappointing, but well done to Oldham who played some terrific football."
Adams added his newly-promoted side is struggling to find its feet in Sky Bet League One.
He said: "We are struggling in the way we defend and always seem to have a lack of concentration, which we cannot allow to do.
"We are creating chances, but not sticking the ball in the net like we did last year. We will work at it and it is not all gloom and doom."