The Rugby Park side were the favourites from four teams to secure a place in the upper half of the table but will now contest the final five games of the campaign with the top flight's also-rans after losing 2-1 to Dundee.
Shiels said: "It's a hard one to take. I'm very low but I've got to pick myself up so I can pick the players up for Monday and get back to work.
"I don't know what went wrong. We had two games to win one game. But, as fate would have it, Gary came back and scored two goals while Rory Boulding has scored a goal for United in the last minute.
"So I have got to take the responsibility for letting those players go. But that's what happens in football, they come back to haunt you. Gary is a good player, I have always said that, and it's just one of those things.
"I feel sorry for the supporters and the club because they trusted me to deliver and I failed. I have got to accept responsibility for today."
Dundee interim boss John Brown was delighted with his players as they secured the victory that keeps their tiny survival hopes alive for another week.
Hailing their enthusiasm, he said: "They have been that way since I came in. We had hunger in the game against Motherwell last week but we gave away cheap goals. But that hunger is the dressing room.
"What surprises me is how they didn't have more points on the board. I don't know if I have had a reaction but they are fighting for each other.
"It's a great result. We have not had many wins this season but to come to Kilmarnock and win, when there is a lot of money at stake for Kilmarnock to get into that top six, is fantastic."
- Sports & Recreation