The International Energy Agency laid bare the challenge to OPEC's efforts to boost oil prices Tuesday, noting that the cartel's output hit all an all time record in September while demand growth continued to slow.

Oil prices retreated following the bearish report, falling about 1% from the one-year highs hit Monday after Russia said it was ready to back OPEC's initiative to control output. Brent Crude futures traded Tuesday morning at $52.58, down 1.05%, while West Texas Intermediate slid 0.95% to $50.86.

"Recent comments by Saudi Arabia and Russia point to a greater probability of a production cut, although higher production from Libya, Nigeria and Iraq are reducing the odds of such a deal rebalancing the oil market in 2017," Goldman Sachs analysts including Damien Courvalin and Jeffrey Currie noted. "Failure to reach...a deal would push prices sharply lower to $43/bbl in our view as we forecast that the global oil market is in surplus in 4Q16."

The IEA said global oil supply rose by 600,000 barrels per day in September, compared to August, to a total 97.2 million, boosted by an extra 500,000 of Russia and Kazakh output.

OPEC production likely clocked in at an all time record 33.64 million barrels per day in September, up 160,000 barrels on August's average, driven higher by record output from Iraq and Libya's reopening of oil ports, according to the agency.

On the other side of the equation demand is likely to grow at a slower rate year-on-year in 2016, climbing by 1.2 million barrels per day, according to the IEA. "Growth continues to slow, dropping from a five-year high in 3Q15 to a four-year low in 3Q16 due to vanishing OECD growth and a marked deceleration in China," the agency said in its report.

Late last month, OPEC agreed an in-principal deal to cut output to between 32.5 million and 33 million barrels of oil per day, providing a boost to oil prices, which have risen by about 15% since that agreement was flagged. Details of the deal are yet to be finalised and the IEA and analysts remain sceptical that an agreement will be reached in time for the cartel's next meeting in Vienna on November 30.

"Now the real work starts," noted the IEA. "Our supply demand outlook suggests that the market - if left to its own devices - may remain in oversupply through the first half of next year."

Goldman, meanwhile, expressed skepticism that a deal, even if it is agreed, will be applied in a meaningful way.

"The odds of a successful implementation remain low," the bank noted, citing three key risks: "likely poor compliance from non-core OPEC producers," increased production from Libya, Nigeria and Iran - all of which are exempt from the agreement - and increased production from non-Opec countries in 2017.

"Any of these three forces is sufficiently large in our view in 2017 to make the require cut in Saudi production too large," wrote Goldman's analysts.