Malaysia vowed to "thoroughly investigate" after admitting that one of its envoys was the diplomat who fled sex charges in New Zealand after invoking diplomatic immunity.
Muhammad Rizalman was charged in a New Zealand court on May 10 with burglary and assault with intent to commit rape.
But the case was dropped when he invoked diplomatic immunity. He also reportedly refused to submit a DNA sample.
Both Muhammad Rizalman's name and home nation had been suppressed by the court, but the suppression order was lifted after a challenge by media organisations.
Muhammad Rizalman was arrested in Wellington after allegedly following a 21-year-old woman on the night of May 9 and attacking her in her home.
He returned to Malaysia on May 22, where he is now being investigated by the defence ministry, Anifah said.
"The Malaysian government acknowledges that the incident is a serious matter, and we do not have any intention to sweep the matter under the carpet," Anifah said.
"I have made it very, very clear that this case must be investigated thoroughly."
New Zealand said officials in Kuala Lumpur refused a request to waive Muhammad Rizalman's diplomatic immunity.
But Anifah said his country had been prepared to do so but New Zealand had offered to allow the accused to return to Malaysia.
Anifah said Muhammad Rizalman, who was working in Wellington for a year in his first foreign posting, was undergoing psychological evaluation.
Anifah did not rule out that the 38-year-old, who is married, would be sent back to New Zealand if Wellington was unhappy with the investigation given the two nations' "good bilateral relations".
"If it's absolutely necessary... I will consider sending him back," he said.
He said the envoy could be brought to a military court, but he could not comment further on the penalties he could face or how long the investigation would take.
He said Malaysian authorities were working with New Zealand to get the evidence in the case.
Malaysia has a large diplomatic compound in Brooklyn, the Wellington suburb where the attack allegedly took place.
New Zealand Foreign Minister Murray McCully said he called in Malaysia's top diplomat on Monday to express how seriously Wellington viewed the matter.
He said he made it clear to the senior envoy - High Commissioner Rosmidah Zahid - that there would be consequences if no action was taken against the man.
"It will have a bearing on how we deal with the country concerned," McCully said.
Prime Minister John Key expressed similar sentiments on Monday, saying New Zealand preferred immunity was waived so the man could face trial in New Zealand but, failing that, he should be prosecuted in his homeland.
Malaysia is New Zealand's eighth largest trading partner, with exports dominated by food and beverages, while imports from Malaysia largely consist of petroleum and electronic goods.
Additional reporting by Associated Press