The warning has come from senior advisers to the Polish Government who say the country needs to prepare for an influx of their countrymen.
It has sparked fears such a large-scale withdrawal could further damage the British economy.
The Polish workforce is estimated to contribute £1.9 billion a year to Treasury coffers.
There are also concerns that building projects, which have been boosted in recent years by gangs of highly skilled, conscientious Poles willing to work for low wages, will be affected.
The rise of the Polish zloty against the pound has meant that some Poles can earn almost as much in their native country as in the UK.
The news came as Prime Minister Gordon Brown spoke publicly about the prospects of a recession for the first time.
The pound also fell by its biggest margin on the currency markets since the aftermath of Black Wednesday in 1992.
Professor Krystyna Iglicka, a senior adviser to the Warsaw Government, told The Daily Telegraph 400,000 of the 1.2million Poles in Britain and Ireland are expected to lose their jobs by next year as the downturn starts to bite.
Most would either go home or to another country to seek work, she said. Many were tempted to go home because Poland has so far escaped the credit crisis which has brought British banks to their knees.
A spokesman for the Polish Embassy in London, said: "The Polish economy is growing because the credit crunch didn’t ravage the Polish banking system.
"Our regulations don’t allow banks to involve people’s money in such risky operations as banks in western Europe.
There is a shortage of jobs in Britain but in Poland job prospects are better than two or three years ago."
Poles were especially a victim of the downturn because the vast majority of British-based Poles were occupying low-skilled jobs at the bottom of the labour market, and with no protection from contracts.
With the increasing value of the zloty against the pound, many over-qualified Poles were likely to feel that they do not need to work long hours in poor conditions.
Booming wages have meant that some workers are paid more in Warsaw than in parts of western Europe, analysts say.
The departure of so many skilled workers could prompt fears that building sites will face a major labour shortage. The extensive building programme ahead of the 2012 London Olympics could also be at risk.
Polish labour agencies are already reporting that the trickle of returning countrymen is steadily turning into a flood.
Artur Sawastian, who runs an agency in Szczecin, north west Poland, said: "This year we’ve seen a really great increase in Poles calling from England, looking to move. They can either stay put or come home."
However Professor Iglicka, who is also an expert in economics and demographics at Warsaw’s Centre of International Relations, warned that the wave of returning migrants could have a "destructive effect on Poland’s economy".
Poland could struggle to absorb the returning workers, she said, with third-quarter growth this year expected to fall from 5.8 per cent to 4.1 per cent according to official Government figures.
Poles started to arrive in the UK in their hundreds of thousands after the expansion of the European Union in 2004 made it easier for people from eastern Europe to come to work here.
There had historically been a sizeable Polish community in this country since World War II, when Polish forces fought alongside British forces against Nazi Germany.
Last night, the Tories’ Chris Grayling, the shadow work and pensions secretary, said: "This just underlines how absurd Gordon Brown is when he boasts about record employment in the UK.
"Most of the new jobs created in the past ten years have gone to people from overseas, and the number of British people in work has been falling steadily for years."
A UK Border Agency spokesman said it estimated that half of the Poles who came to live here had gone home.
He said: "Our figures show that fewer Polish workers are coming here. Our new points system means only those with the skills we need will be able to work or study here and no more.
"This strict new system, plus our plans for newcomers to earn citizenship, will reduce overall numbers of economic migrants coming to Britain, and the numbers that stay."
Mr Brown also announced moves to help people who are struggling to pay their mortgage avoid losing their home unless all alternatives to keep people have failed. A global credit crisis summit will be held in Washington next month.
A poll last night also showed that voters believe Gordon Brown is a better leader in a crisis than David Cameron.
In the YouGov/Channel Four poll, 41 per cent of people said Mr Brown is the best prime minister for the UK during an economic crisis. The Tory leader polled 27 per cent. Source: The Telegraph (UK)