Scarlets recovered from 17-3 down to get back on level terms early in the second half, but a late try from James Scaysbrook sealed victory for the Chiefs.
"The next two Heineken Cup games [away to Clermont and home to Leinster] can go to bed for a while now because we have some very important Premiership rugby to take care of," said Baxter, whose side face upcoming domestic games with west country neighbours Gloucester and Bath.
"If we don't take care of that we would be in danger of letting the season fall flat a little because we are in a nice place now, ticking along quite well but showing some consistency and improvement in some areas but getting a little dented in others.
"But as long as we keep fighting to take those steps forward then I will be very happy. We have backed up a win in the Heineken Cup with a second win and this was our first in front of our own supporters.
"I am confident that we can play better, which I have said to the players is a nice place to be in. There was a lot of endeavour out there and a lot of hard work. Sometimes you just have to graft your way through to a win, and that is what we did."
This was the first of five games on the road for the Scarlets, and head coach Simon Easterby knows they must find some improvement in the way they start games before they face Cardiff Blues on Friday night.
Easterby said: "The game got away from us in the opening 20 minutes and you can't give any team two soft tries, but that is what we did. The charge down was one which they didn't have to work for and then they scored off a set-piece.
"When you gift them 12 points in the first half then it is always going to be an uphill struggle, and we did that last week as well when they got into an early lead.
"Our defence on the whole was good and we came back well in the second half, but when you give sides soft tries it is difficult to take that. I can't really put my finger on why we started slowly and we have had a couple of them recently - I think it needs to be addressed before we go in to the Blues match."