Gervinho, Santi Cazorla and Olivier Giroud put the Gunners three up before Hal Robson-Kanu headed one back.
But there was to be no recovery to mark their new manager's first match in charge as Mikel Arteta's penalty secured Arsenal's win.
However, with a home clash against former club Southampton next, Adkins is not about to throw in the towel just yet.
He said: "We have had a bit of a fragmented week with the change, and players away on international duty, trying to get to grips with where we want to go and what to do is going to be challenging.
"We wanted to get something here, but didn't. Arsenal kept the ball very well.
"We will analyse it now, debrief back with the players, take the lessons from it and work very hard on the training ground for the next one.
"You can't change everything straight away, but we will all work hard together to give us a chance in the next game."
Adkins added: "The spirit is there in the group, and all we can do is take it one match at a time - there here and now is putting this one to bed, working hard over the course of the week for the next one."