The Terriers were mauled by Leicester on New Year's Day, with a 6-1 defeat at the King Power Stadium stretching their run without a win to 10 matches, while they have slipped to 17th place in the table, seven clear of the drop zone.
"You have to be realistic and we're in a position, and run, at this moment in time when we're not doing enough to win matches," Grayson said after his side's latest defeat.
"We're going down the table rather than up it and we're honest enough to realise that we can get dragged into a relegation battle if we're not careful."
Grayson added: "The players have to get the belief and confidence back and do the things that got us results in the early part of the season. I still believe we have a group of players that is capable of doing that, but we have to get results quickly.
"One or two players are not helping the cause over the last few weeks with their performances, so I have to sit down and figure out what we need to do to end the run that we're on and get back to winning ways.
"There are players who will certainly be looking over their shoulder now to see if they are in at the weekend and then for the Birmingham game a week later."
Huddersfield travel to Charlton on Saturday for their third-round FA Cup tie.