One in seven cases in the so-called legacy files are unlikely to be solved because officials have no idea where the individuals are, a report concluded.
The Home Office admitted in 2006 that a backlog of up to 450,000 files had built up, some dating back to the 1990s, which had not been concluded.
Officials are on course to meet a pledge to clear the backlog by this summer but it is expected some 61,000 cases will have to be closed without the applicant having been traced, the report by the Home Affairs Select Committee said.
It means tens of thousands of individuals may be left to live in the UK without any formal right to stay here.
The report also warned a fresh asylum backlog could be building up because only six in ten new cases are being dealt within the target time of six months.
The report said: "While we agree that the UK Border Agency should not spend unlimited time trying to track down missing applicants, we are concerned about the high proportion of cases which will be left, in effect, in limbo.
"Again, this points to the vital need to deal with cases as expeditiously as possible and not to let backlogs grow."
The report also calls for tougher checks on suspected bogus colleges and better training for those involved in the removal of failed asylum seekers.
Keith Vaz, chairman of the committee, said: "Much of the delay in concluding asylum and other immigration cases stems from poor quality decision-making when the application is initially considered.
"The UK Border Agency has made some progress over the last few years in relation to new procedures and approaches, but is still failing to meet expectations.
"More consistent and rigorous scrutiny of applications would lead to fewer delays, fewer appeals, less uncertainty for the applicant, less pressure on the officials themselves, and probably lower costs for the UK taxpayer.
"This may well require greater investment in staff training. It is also likely to require more consistent and considered direction from those setting policy for the Agency than has sometimes been the case."
In a separate move, the committee called for a ban on bonuses for senior Home Office staff in the current economic climate.
It added that the successor to Lin Homer, the previous head of UKBA, should be paid less after it emerged she was earning more than £208,000 a year.
Ms Homer has become permanent secretary in the Department for Transport, where she is now earning £170,000 a year.
Mr Vaz said: "In the current climate we believe it is inappropriate for senior Home Office officials to receive any bonuses.
"We also believe that the new head of the UK Border Agency should not receive a salary greater than either the Permanent Secretary of the Home Office or the Prime Minister." -Telegraph