The 24-year-old scored 14 goals in 37 appearances during a season-long loan spell with the Terriers last term and now pens a three-year contract, joining for an undisclosed fee.
"James' arrival is a significant signing for the club and one that I know our supporters will be very pleased with after his excellent debut season in a Huddersfield Town shirt," boss Mark Robins told the club's official website.
"The exciting thing is I believe there is even more to come from James.
"He is only 24 and although he already has excellent qualities in terms of strength, pace, work rate and much more, I believe he will have learned from the games he played last season and we will benefit from his knowledge and understanding of the style of football we want to play here.
"The club has been very patient to get the right financial deal in its negotiations with Norwich City and I believe we have secured that, but negotiations with James took almost no time at all - that was agreed some weeks ago after Norwich gave us permission to speak to the player.
"He has shown such a huge desire to come to Huddersfield Town and turned down other, potentially more lucrative, financial offers.
"This transfer is, without doubt, a significant step forward ahead of the new Championship season."