The long-running political row over the latest anti-terrorism proposals to detain people without charge for up to 42 days has been in stark contrast with the silence over the 2,000 men, women and children detained in the UK without charge.

These are the failed asylum seekers who are held for days, months or years in one of 10 prison-like, immigration removal centres scattered around the UK, and they are invisible.

But when they are noticed, it is generally by the tabloids and right-leaning think tanks. A recent example: The Sun reported that the ‘foreign lags’ in Colnbrook Immigration Removal Centre are living in a ‘holiday camp’.

Well, for the past 18 months I have spent a couple of hours each week visiting these so-called foreign lags in Colnbrook and Harmondsworth next door, and from what they have told me and from what I have seen, it couldn’t be further from Butlins.

The two centres are next door to the Heathrow Sheraton and you could mistake them for anonymous hotels designed for delayed passengers and tired cabin crew. However, after the first glance, you see swathes of barbed wire, barred windows and uniformed officers milling around.

As a visitor, you require two forms of ID; finger prints and a photo are taken; and there is a thorough search. The atmosphere is a heady mix of loud, oppressive, emotional and distressing. Children are crying, couples are holding each other; and there’s me, talking to a stranger about politics, Hollyoaks, torture, music, desperately missed wives and children, mothers, fathers, brothers, sisters and the infamously bad food.

I started visiting detained, failed asylum seekers during the summer heat wave in 2006. The political temperature was also running pretty high and the tabloids were baying for the blood of the now ex-home secretary, Charles Clarke.

The tabloids were scandalised to learn that foreign residents were released after completing a prison sentence. “They must be deported!” they shrilled, and the home secretary hastily complied. Inevitably mistakes were made, like attempting to deport British citizens. Abdul, my first client, told me he was British passport holder, who had come here with his family at 18 months old.

He had committed a petty crime and on completion of a short prison sentence, he received a notification of intention to deport to Bangladesh. After a speedy intervention from his solicitors in the High Court, he was allowed to go home to his British wife and children.

Aside from British citizens, torture victims also should not be detained according to the Home Office’s own guidelines. Michael, 20, from Uganda, and I got to know each other well as the many months passed in first Colnbrook and then Harmondsworth and he told me why he came to the UK: he came to the UK after the 2006 Ugandan elections, he was, and in exile, still is a prominent member of the youth wing of Uganda’s opposition party.

He was tortured by the Ugandan authorities – it is clear to see that some fingers nails on each hand have been pulled out - and he told me that he was badly beaten as a warning. He came to the UK soon after. Michael was on the ‘fast-track’ system, whereby the Home Office makes a decision on a ‘straightforward’ case within 24 hours, with a 99 per cent refusal rate.

According to Michael’s documentation from the Home Office, one reason why his case has been refused is due to the fact that the Home Office does not believe his claim that he was very politically active from 16 and became a constituency campaign manager for the youth wing of the party.

Perhaps, thanks to the political torpor among much of Britain's youth, the Home Office’s scepticism isn’t surprising. His case continues two years later and will be heard in the High Court in the coming months. He is no longer detained and receives counselling for post traumatic stress disorder.

The Home Office states people should be detained for as short a time as possible before removal. Currently, I visit Kak Ahmed from Kurdistan, who has so far been detained for 16 months after a short prison sentence.

He has told me that he can’t get out of Colnbrook because someone with the same name as him failed to comply with their bail conditions.

He feels that he and his solicitor have presented proof of the mistake, but judges repeatedly refuse him bail, believing he will abscond.

His solicitor is now taking this issue to the High Court. Kak Ahmed has severe medical problems, which the Kurdish government have stated cannot be treated in Kurdistan. So he is stuck in the system and being stuck means being deprived of liberty with no end in sight.

The UK is one of a handful of European countries that detains failed asylum seekers indefinitely, with no automatic right to legal advice, with no automatic right to bail and with no automatic case review.

Detention of failed asylum seekers is at the cutting edge of human rights but the majority of people, campaigning organisations, lawyers, journalists and politicians skip over these invisible thousands to politically safer territory.

I try to make a small difference to individuals where I can, but this isn’t scratching even the surface of the problem; there needs to be a sea change in government legislation that will end the indefinite deprivation of liberty that even terror suspects (rightly) avoid.

All the names in this article - including the author's - have been changed