Authorities began the mass evacuation on Saturday in the northern city of Thessaloniki ahead of the planned detonation on Sunday.
The bomb, containing nearly 250 kilograms (550 pounds) of explosives, was discovered in the country's second largest city last week.
More than 300 disabled and bed-bound patients were the first evacuated on Saturday, using 20 ambulances, authorities said. When the operation is completed Sunday morning a nearly 2 kilometer (1.2 mile) radius surrounding the bomb site will have been evacuated.
Evacuation is "obligatory," according to regional security chief Apostolos Tzitzikostas.
The operation is unprecedented in Greece, "where a bomb of this size has never been found in an area this densely populated," Tzitzikostas added.
Regional authorities said defusing the bomb would take up to eight hours, but local military spokesman Colonel Nikos Phanios was not so sure.
"We don't know what we're going to find," he said. Defusing the bomb and then moving it to a military shooting range "could take us up to two days."
A thousand police officers have been deployed for the operation, and residents were given several days' notice via the media, leaflets and posts on social network sites.
Refugees to be evacuated
Bus and train networks are being disrupted in Thessaloniki while facilities are being set up to host evacuees in need of shelter.
People living in a nearby refugee camp are among those being evacuated. At their request, the refugees will use the evacuation as an opportunity to visit the local archaeology museum.
Officials said the device's exterior is too degraded to determine whether it was a German or an Allied bomb. But one resident says he recalls the day it fell.
"The bombing was done by English and American planes on September 17, 1944," said Giorgos Gerasimou, 86, whose home is 800 meters (half a mile) from where the bomb was found." It was Sunday lunchtime. We could see the planes coming."
More than 70 years after the end of World War II, unexploded bombs from the conflict are still being found around the globe.
Dozens of people were evacuated near a Hong Kong university after a bomb was found there last month. At about the same time Britain's navy disposed of a suspected WWII bomb found near the parliament building in London.
In the German city of Augsburg, nearly 55,000 people had an unwelcome Christmas present on December 25 when they had to leave their homes while authorities dealt with a bomb dropped by Britain during the war.