Boro comfortably secured their passage to the second round of the Capital One Cup thanks to goals from Marvin Emnes and Emanuel Ledesma.
"It's all right but I sat there at the end and thought it's still a bit young and inexperienced around the top end of the pitch," said Mowbray.
"We'll try and address that if we can. We do have experience elsewhere. We've got something to build on and some scope still to work over the next few weeks. It was a tough start to the season and you'd have to say Bury played well. They've got some good footballers and asked some questions of us.
"Concentration, apart from one occasion from a corner kick, was good, we looked solid enough and were always a threat going the other way. It was a bit uncomfortable in the last 15 minutes but in cup competitions you have to get through and we got by a difficult obstacle.
"Marvin (Emnes) has the ability to score goals. We've talked to him about not over-elaborating around the box and to shoot when he gets the chance. He did and it flew in."
Middlesbrough never looked genuinely threatened after Emnes had opened the scoring with a superb 25-yarder. Ledesma extended the lead after Trevor Carson spilled Nicky Bailey's shot before Mark Hughes reduced the arrears.
Bury saw Efe Sodje sent off late on to compound their woes but caretaker manager Peter Shirtliff was happy with his audition for the full-time job.
"I enjoyed it apart from the result," said Shirtliff. "I'm quite comfortable in the role so there's no problems there. It's not difficult to plan ahead. I know the players here and I know the squad I've got.
"I'm happy with it at the moment and if I need to tinker with that I'll speak to the chairman and the board. At the moment I'm quite happy with things. We were always in the game and we were disappointed to lose it. When you assess the game and the opposition we had, I think we performed creditably."