The 51-year-old, who guided the All Whites to their first World Cup in 28 years at the 2010 finals in South Africa, steps down with Phoenix mired at the bottom of the 10-team league with five rounds to play.
"The results this year have not been what anyone would have wanted and I felt that it was time for someone else to take a fresh look at things at the club," Herbert told Phoenix's website.
"The coaching staff and players have all put in a huge effort this year but we haven't got the results we wanted.
"So with the need for a fresh look, along with the fact that this year is an extremely important one for the All Whites, I felt I needed to step down as the Phoenix head coach."
Herbert would be replaced by assistant coach Chris Greenacre for the final games of the season but remain as a technical advisor, the team said.
Herbert, a central defender for New Zealand at the 1982 World Cup, is contracted with the national side through the qualifying rounds for the 2014 World Cup in Brazil.
Under Herbert, New Zealand were dumped out of the group stage in South Africa but won plenty of admirers for emerging unbeaten from a group featuring Italy, Slovakia and Paraguay.
The 90th-ranked All Whites are top of Oceania's World Cup qualifying group, three points ahead of New Caledonia, who they face in their next qualifier on March 22.
The winner of the final Oceania round advances to face the fourth-placed team from CONCACAF in a two-legged play-off for a spot in Brazil.
- Sports & Recreation