Rotherham strolled into a three-goal lead after just 35 minutes and Taylor made no excuses.
Ben Pringle capped a fine solo run by firing home after 35 minutes and although Cody McDonald put the frighteners on Rotherham with a 65th-minute goal for the visitors, on-loan debutant James Tavernier restored the Millers' three-goal advantage in the 75th minute.
"In the first half, Rotherham were miles better than us," Taylor said.
"They were first to everything and deserved to be at least 3-0 up.
"It was a very, very poor start from us and if you don't start right and let teams get momentum like we did you are going to get punished and we were lucky it was only three.
"At half-time I told them you cannot start like you fancy. You have got to start strong, hard to play against and start getting first to the ball.
"You have got to give credit where credit is due and Rotherham played exceptionally well in the first half.
"In the second half we calmed it down a bit and sometimes teams take their foot off a bit and give you a chance.
"It was nice to get a goal back, but as much as you would like to say we were going to get back after that first half performance, we had to score quickly and then we had to to play much, much better than we did.
"Not a good day. I can't say there were excuses because there are not."