Ruud Lubbers, the UN High Commissioner for Refugees, said yesterday that the EU was right to put the issue at the top of its agenda but wrong to focus on closing borders and punishing poor countries.
Mary Robinson, the UN Human Rights Commissioner, also urged caution and a sense of proportion, saying that asylum applications in Europe had actually dropped, and were 7% lower in the first three months of this year than in the final quarter of last.
The UN and charities are afraid that EU governments are getting carried away by their own rhetoric after the string of electoral successes by far-right and anti-immigration parties.
Tony Blair strongly supports the Spanish prime minister Jose Maria Aznar's plans to accelerate measures giving the EU common asylum and immigration policies.
Mr Blair and Jacques Chirac of France discussed the issue over dinner in Paris last night in the hope that new measures could resolve the row about the Sangatte refugee camp near Calais.
EU leaders hope the two-day summit will endorse proposals to strengthen frontier controls, including financial arrangements, and set a timetable for a common asylum policy.
But they will have more trouble agreeing controversial proposals to link external aid to poor countries to their cooperation in the return of illegal immigrants.
Speaking on the eve of World Refugee Day today, Mr Lubbers, a former Dutch prime minister, told Reuters: "There is far too much about sealing borders, which will not be effective, and far too much about punishing countries which don't take back their people. If that is all there is, they will be shooting themselves in the foot."
Far from punishing impoverished countries, the EU should be working with those countries to improve conditions there, he added.
Mrs Robinson said she was worried that public fears about immigrants could lead to the tougher handling of refugees and asylum seekers in police stations and holding stations.
The European commission president, Romano Prodi, said the EU's war on illegal immigration must not be waged at the expense of integrating much-needed legal migrants who were "a source of vitality and energy, which ageing Europe needs."
But in a letter to Mr Aznar, he backed tough action against smugglers, saying that trafficking in people was "a crime and an affront to human rights".
In Britain, the Refugee Council added its voice to the chorus of criticism, urging the EU not to close its borders to those most in need.
"It is complete madness to penalise refugee-producing countries or transit countries by restricting access to development aid," its chief executive, Nick Hardwick, said.
"EU leaders should focus on preventing the conflicts and human rights violations that force people to flee their homes."
The Refugee Council said it was particularly concerned that the EU would seek the return of Afghan refugees before the war-ravaged country was ready to absorb them.
Mr Blair was criticised for his "eye-catching initiatives" by the British leader of the Liberal Democrat group in the European parliament.
"Aid-cutting and destroyer-despatching proposals by Tony Blair on immigration and asylum are media stunts which are distracting EU leaders from the task of creating a common policy in this area," Graham Watson said.