The ballot was declared invalid with only 40.1 percent turnout, below the 50 percent required to validate the referendum. However, some 98.3 percent of people who voted backed Prime Minister Viktor Orban's stance on migrants.
The EU migrant quota proposal, driven by German Chancellor Angela Merkel and approved by most EU states in 2015, aims to ease pressure on Italy and Greece, where most migrants enter the EU. Implementation has been slow, with just 5,651 out of a hoped-for 160,000 people relocated so far, with eastern and central European countries in particular opposed.
Over 400,000 refugees made their way through Hungary towards northern Europe in 2015 before Budapest sealed off its southern borders with fences topped with razor wire in September.
The EU said last week that it hoped to relocate half of them by the end of 2017. A deal struck in March with Ankara to reduce numbers appears shaky in the wake of a coup attempt in Turkey in July.
Orban calls mandate 'unprecedented'
Orban said the referendum result was an "unprecedented mandate to stop Brussels encroaching on national sovereignty," adding that he would introduce a bill in the coming days to amend the constitution and reflect "the will of the people in the spirit of the referendum."
"In the history of Hungarian democracy, no party or party alliance has ever received a mandate of such scale," Orban told parliament in Budapest. "We won't let these 3.3 million people be tricked, or have their opinions downplayed."
Government billboards and TV adverts during the campaign linked migrants with "terrorism" and crime. Comparing immigration to "poison", Orban has emerged as a standard-bearer within the EU opposing providing asylum to refugees, as the bloc battles its worst migration crisis since World War II.
Opposition parties, non-government organizations and activists called for a boycott of the referendum or for voters to spoil their ballots. More than 230,000 invalid ballots were recorded, a record for a Hungarian referendum.
European Parliament President Martin Schulz warned on Sunday that Orban was playing "a dangerous game" by attacking EU treaties to cement his power at home. Hungarians "did not heed Orban's call. What the EU now needs is dialogue to deliver solutions, not artificial tensions," Schulz tweeted on Monday.
jbh/msh (AFP, dpa)