It said little progress had been made in making the system more transparent and less complex, and that confidence in it had become "very fragile".
The eurozone debt crisis remained the main cause of concern, the fund said.
On Monday, the IMF downgraded its forecast for global growth.
It estimated growth in 2013 to be 3.6%, down from 3.9% in its previous forecast in July. This included sharp downgrades for the UK, Brazil and India.Remaining risks
In its latest Global Financial Stability Report, published every six months, the IMF said "significant efforts" by European policymakers had "allayed investors' biggest fears".
The European Central Bank offered cheap loans to banks early this year; the bank's chief Mario Draghi said he was prepared to do whatever it took to save the euro in the summer; while on Monday, eurozone governments announced the launch of the long-awaited European Stability Mechanism (ESM), the bloc's new permanent fund to bail out struggling economies and banks.
The ESM, with 500bn euros (£400bn; $650bn) at its disposal by 2014, will be able to lend directly to governments, but it will also be able to buy their sovereign debts, which could help reduce the borrowing costs of highly-indebted countries such as Italy and Spain.
However, the IMF said that "despite recent favourable developments in financial markets, risks to financial stability have increased since April".
It said concerns about countries leaving the eurozone had led to "capital flight" away from the bloc that "undermined the very foundation of the union".
It added that the need for banks to build up capital protection, together with high borrowing costs for governments, was "generating very strong headwinds for the corporate sector".
The fund talked of a benchmark set of goals for the financial sector - institutions and markets that are "more transparent, less complex, and less leveraged".
"Although there has been some progress over the past five years, financial systems have not come much closer to those desirable features.
They are still overly complex, with strong domestic inter-bank [links], with the too-important-to-fail issues unresolved".US debt
The IMF highlighted a number of measures that were needed to help resolve the crisis.
It said individual governments needed to cut debt levels without choking off growth and push through reforms to clean up the banking sector, including recapitalising viable banks.
The ECB would need to help in this process, it said. The IMF also reasserted its view that much closer ties were needed between eurozone banks.
Outside the eurozone, the fund pointed to risks in the US and Japan.
It highlighted the looming so-called fiscal cliff in the US, when automatic spending cuts and tax increases kick in at the beginning of next year.
In Japan, the fund highlighted high budget deficits and record debt levels, as well a "growing interdependence" between banks and the state.