"As long as Mugabe rules Zimbabwe, he remains a stain on that country," Miliband wrote in a letter to the Times newspaper.
"I acknowledge that he is also a stain on the international community, which has not been able to deliver the will of the Zimbabwean people."
The letter appeared after Mugabe dismissed as "stupid" US calls for him to go after its top Africa envoy Jendayi Frazer said the US had lost confidence in the power-sharing deal between Mugabe and the opposition.
An outbreak of cholera has killed over 1 000 Zimbabweans in recent weeks, according to UNICEF.
Former colonial power Britain has been one of the most critical voices against 84-year-old Mugabe.
"Any sane person realises that Mugabe’s misrule is only spurring Zimbabwe’s descent into chaos," Miliband wrote.
"If Zimbabwe is to haul itself – with the help it needs and deserves – out of its current meltdown, Mugabe has to go."
He said Britain wanted stronger action by the UN Security Council and said that Russia and China "must now answer for their approach" after vetoing a sanctions resolution last July.
Britain would continue to emphasise South Africa’s "role and responsibilites" over Zimbabwe, he added. – Sapa-AFP