The new tax is aimed at raising $117bn over at least the next ten years from about 50 of the biggest financial institutions on the Wall Street depending on the magnitude of their liabilities.
It will impose a levy of 0.15 of a percentage point on the balance sheets of companies which have assets in excess of $50bn.
In a short speech at the White House, Obama swore to recover "every single dime the American people are owed."
He said he was even more determined to make the banks reimburse the bail-out costs after reading reports about their "massive profits and obscene bonuses."
The bailout tax, called 'financial crisis responsibility fee,' will help recoup the losses incurred by the government in arranging the $700bn Troubled Asset Relief Programme (Tarp) to refinance hundreds of banks in 2008.
The bigger and riskier the institution, the more it will have to pay in an initiative that Obama said is designed to prevent future "abuse and excess" and discourage banks from falling back into 'business as usual' behaviour.
The president stressed that the new tax should not be seen as a punishment levy but as an incentive to change behaviour of the banks for the better. He added that it was only fair for the financial industry to bear the bailout burden rather than the American taxpayer.
Industry lobbyists have complained that they are being forced to pay the entire cost of a bail-out package that also involved rescuing some of the Detroit bigwigs.