The Yorkshire side's striker Jermaine Beckford scored the game's only goal after 11 minutes at The Valley, firing past Charlton's debutant goalkeeper David Button from eight yards for his 16th FA Cup goal in his last 16 games.
Charlton's Dorian Dervite then deservedly got his marching orders after 57 minutes when the defender tripped Beckford as the Huddersfield man ran through on goal.
Powell was angry that his side created few chances throughout in the third-round clash, which drew a crowd of just 6,657.
"To say I'm frustrated is the understatement of the year," Powell said. "I'm not enamoured with what happened. I know we went down to 10 men but we went out with a whimper.
"It was a poor goal for us to concede and it gave us a mountain to climb. I'm not happy with the way it has panned out.
"I put in a few players to give them a chance to show they're ready to play a part in our season and also it's the cup and I really wanted us to be in the fourth round.
"I know we didn't have as many fans as we usually do but we wanted to perform. Huddersfield are on a bad run but we really didn't get a foothold in the game.
"We showed last week at Watford what we can do when we get in good positions and create chances, but we never really did that.
"We've been okay away but not done enough in front of our fans. I want us to be better at that in 2013 and it starts next week at home to Blackpool."
- Sports & Recreation
- Jermaine Beckford
- Dorian Dervite
- David Button