The Chiefs visit relegation favourites Sale Sharks on Friday night, with Saracens, London Welsh and Harlequins also on their immediate fixture schedule.
A successful run would put Exeter firmly in the play-off mix and maintain hopes of qualifying for next season's Heineken Cup following an impressive European debut campaign this term when they beat the Scarlets home and away and twice ran Leinster close. "These are really important games now. We want to do well in this year's Premiership," Baxter said.
"We've been in the Heineken Cup this year and really enjoyed it. We would like to be there again, and that is going to depend on how well we go in the Premiership now.
"Every game is going to be very important. It's exciting entering these four games and where you could potentially go. That's how I want to approach it.
"I don't want us to be fretting about where the season might go. I want us to be excited about how far we can fire ourselves."
Sale, five points adrift at the Premiership basement, enjoyed a successful break from league action by progressing into this season's LV= Cup semi-finals, where they will meet Saracens on March 10.
After hosting Exeter, Sale visit London Welsh on Sunday week, and chief executive Steve Diamond said: "We are rock-bottom of the Premiership and we have got to move up the table.
"If we can reach the final of the LV= Cup and maybe finish 10th in the Premiership then suddenly the season won't look quite so bad. Exeter will be a big challenge for us. They are a very good side and well coached."
League leaders Harlequins, meanwhile, entertain Wasps in the pick of Saturday's action, when Northampton host Gloucester, Worcester visit Bath, London Welsh travel to Leicester and London Irish have home advantage against Saracens.