They will travel with rights activist Graca Machel, wife of former South African president Nelson Mandela, on November 22-23 on a mission to prevent the crisis from worsening, Annan said in a statement.
"Relieving the suffering of millions of people must be the priority of Zimbabwe’s leaders," Annan said in a statement. "But global attention is also slipping as Zimbabwe’s humanitarian crisis worsens."
Zimbabwe’s economy has been in free-fall for years, battered by hyperinflation last estimated at 231 million percent in July.
Although the country was once a food exporter, the United Nations estimates that five million people – nearly half the population – will need food aid in January.
Annan insisted that their mission was purely humanitarian and would not touch on the protracted negotiations to form a unity government under a two-month-old power-sharing deal.
"However, we urge Zimbabwe’s political leaders to move swiftly to fully implement the September 15 agreement, particularly the provisions on humanitarian and food assistance," he said.
"Delays in forming a government are prolonging the suffering of the people, Annan added.
Annan, Carter and Machel are members of the Elders: 12 world-respected statesmen with hands-on experience in conflict resolution.
The group was formed last year by Machel and Mandela on his 89th birthday.