"We were unable to reach an agreement -- this is one of the worst meetings we have ever had," said Ali al-Naimi, oil minister for Saudi Arabia, OPEC's biggest producer.
The failure to do a deal is a blow for consumer nations hoping the Organization of the Petroleum Exporting Countries would take action to stem fuel inflation.
It also underlines concerns about OPEC's willingness to help control prices, perhaps leaving the oil market more open to speculative attack.
"It is absolutely amazing," said Alirio Parra, Venezuela's former OPEC president. "This is not market leadership.
Brent crude rose $1.30 a barrel to $118.08.
The United States had put pressure on Saudi to deliver a credible deal to cap crude prices and underpin faltering economic growth.
"We have noted with disappointment that OPEC members today were unable to agree on the need to make more oil available to the market," said the West's energy advisor the International Energy Agency.
Analysts said that while there were opposing views on whether markets required more crude, the backdrop to the disagreement revolved around political tensions in the Middle East and North Africa and differences over how to respond to consumer demands.
"One factor is a diverging market view. Another is politics," said analyst Samuel Ciszuk at IHS. "At times of heated politics/ideological debate, Saudi struggled to dominate as much as it could have given its size vis-a-vis others in OPEC.
Gulf Arab producer Qatar has given support to Libyan rebels fighting the government of Libya's Muammar Gaddafi. And Saudi Arabia has angered Shi'ite Iran by using force to support the Sunni Bahraini government in suppressing a Shi'ite rebellion.
Saudi's Naimi said OPEC's four Gulf Arab countries proposed the 12-member group increase output by 1.5 million barrels a day to 30.3 million barrels a day, including Iraq which is not bound by an OPEC quota.
But they were left isolated by a majority of seven -- Libya, Algeria, Angola, Ecuador, Venezuela, Iraq and Iran -- who wanted to keep production unchanged. Nigeria's position was not known.
Iran said the view of the majority was that supplies were adequate for the time being and that it had proposed delaying a decision on more oil by two or three months.
"Iran believes there is no shortage of supply," acting Oil minister Mohammad Aliabadi told Reuters. "There is no request that we cannot meet, therefore there was no need to raise output and that was the opinion of many other OPEC members."