About 250,000 voters risk being struck off the electoral roll if they fail to validate their addresses by June 29.
They have ignored requests from the Registration and Electoral Office despite being given six weeks to reply, the office said in a document for lawmakers' discussion.
The office embarked on 'improving' the voter registration system after November's District Council polls were dogged by accusations of vote-rigging by people giving false addresses, or being found to be registered at the same address as other voters.
Some gave addresses that were non-existent.
With the Legislative Council polls due in September, the Constitutional and Mainland Affairs Bureau, which oversees the office, is keen to avoid a repeat.
The office said it had tried to confirm addresses in several ways, including checking their details against the Housing Authorities' records.