The visitors, who chose to bat after winning the toss, reached 280 for five at stumps with the last two wickets falling in the last two overs of the day on a slow and low surface in Chittagong.
Left-arm spinner Abdur Razzak, who played his last test in August 2011, took two wickets as the slow bowlers picked up all the wickets to fall for the hosts.
Williamson (114) walked in following the wicket of opener Hamish Rutherford (34) after an opening stand of 57 runs and kept the Bangladesh spinners at bay to build a solid platform.
He stitched together a 126-run stand for the second wicket with opener Peter Fulton (73) and a 61-run partnership for the third with former captain Ross Taylor (28).
Taylor fell just before the second new ball was due giving Bangladesh a window to stage a comeback.
Williamson, 23, hit 12 boundaries before being adjudged leg before to left-arm spinner Shakib Al Hasan in the penultimate over of the day.
Captain Brendon McCullum, who made a quick-fire 21, followed in similar fashion in the next over to give Razzak his second scalp.
New Zealand handed test debuts to 20-year-old leg-spinner Ish Sodhi and all-rounder Corey Anderson while Bangladesh all-rounder Marshall Ayub also earned his first international cap.