It’s gratifying when someone takes on a topic I had wanted to address and does a better job than I could have. Various readers have explained how prices are formed in oil markets, most importantly, how for the vast majority of oil sold, prices are specified in long-term supply contracts and the benchmark is not spot but prices in the futures markets.
JD at Peak Oil Debunked (wish the blog has a less partisan name) describes how oil is priced in a straightforward and well referenced post (hat tip reader Michael).
Recall that the role of the futures market in oil prices is disputed. Most people outside the oil business assume that the oil markets work like many other markets: the spot market is a mechanism for price discovery (or trades in reasonable proximity to other markets where price discovery occurs). Near dates futures will similarly reflect expectations of supply and demand because at contract maturity they can be arbitraged to physical. Thus the effect of speculative activity in futures market will be constrained by the ability to take delivery.
Well, it ain’t that simple. JD explains:
A number of high-profile economists, like Paul Krugman, have recently been making the argument that trading in oil futures can’t really influence the price of physical oil because it doesn’t remove any oil from the market. Here’s a classic statement of this argument by Jon Birger, a staff writer from Fortune:
Here’s a suggestion: The next time a Congressional committee wants to hold a hearing on how “speculators” are driving up oil prices, each committee member should first be required to demonstrate – preferably in their opening remarks – a basic understanding of the mechanics of futures trading.
Even better, they should be required to explain in detail how it is that investors who never take delivery of a single barrel of crude – and thus never remove a drop of oil from the open market – are causing record high oil prices.Source
I will now provide that explanation, and in the process show that both Krugman and Birger are grossly misinformed about the way physical crude is actually priced in the global oil market.
Most crude oil is traded based on long-term contracts, and the prices in those contracts are set by a system known as “formula pricing”. In this system, the price of delivered crude is set by adding a premium to, or subtracting a discount from, certain benchmark or marker crudes, namely: West Texas Intermediate (WTI), Brent and Dubai-Oman. Generally, WTI is used as the benchmark for oil sold to North America, Brent for oil sold to Europe and Africa, and Dubai-Oman for Gulf crude sold in the Asia-Pacific market (Source1, Source2).
JD then provides a description of how the spot market became increasingly subject to manipulation, which led oil exporters to turn to the futures market, which was more liquid and hence less subject to games-playing, as the basis for contract pricing. JD cites a paper by Bassam Fattouh of the Oxford Institute for Energy Studies:
[T]he futures market has grown to become not only a market that allows producers and refiners to hedge their risks and speculators to take positions, but is also at the heart of the current oil-pricing regime. Thus, instead of using dated Brent as the basis of pricing crude exports to Europe, several major oil-producing countries such as Saudi Arabia, Kuwait and Iran rely on the IPE Brent Weighted Average (BWAVE).11…
[11] The BWAVE is the weighted average of all futures price quotations that arise for a given contract of the futures exchange (IPE) during a trading day. The weights are the shares of the relevant volume of transactions on that day. Specifically, this change places the futures market, which is a market for financial contracts, at the heart of the current pricing system.
JD concludes:
As you can see, Krugman and Birger are profoundly confused about the way the international oil markets actually function. Futures aren’t a paper bet on the direction of prices determined by some independent process. Futures themselves *determine* the price of most physical oil traded today. The futures price (+ or – the differential) literally *is* the price of oil.
He also provides further references on oil pricing.
44 comments
Comments are closed.
Wow! I have been looking for a week for something that explained what is obviously happening. It is quite clear that the bulge in oil price is NOT related to supply problem, but how the speculators were manipulating was not obvious. Turns out they may not be manipulating at all. This price system seems to be perfectly compatible with the idea that more investors flooding in will push the price up inherently as the bid the contracts up back and forth.
And how does this explain the run-up in other markets when they have no such special features?
You have to explain the general run-up in commodity prices in markets with and without speculators, and in markets with and without futures markets. Something like this can explain the departure from the common trend, if, in fact, this argument is correct (that is still an open question), but it cannot explain the biggest part of the story, the trend itself.
You know, I’m not a trader, a quant or a banker, but it doesn’t take much to see what is happening right before our very eyes.
This is not a supply and demand probelm. Supply has been steady, demand is now falling, prices are rising.
The Fed is pumping tons of liquidity into the markets via their BS “facilities”. Anyone in financials the has access to the numbers are astounded at the use of the facilities. Toxic paper that Wall Street created has been monitized by the Fed.
Further, liquidity is being drained from all of the credit markets and now the equity markets. Where the hell is this money going? It sure as hell is being lent out by the banks.
This is the CDO BS all over again. The brains on a stick wanted to talk about tranches and slices and CDO’s squared and blah, blah , blah. In retrospect, it was all a huge ponzi scheme underwritten by J6P taking out a mortgage he could never afford, the consequence of which is the utter destruction of the housing market for years if not decades.
Now they are doing it with oil and food. For you the blind who once could see, the bell tolls for thee.
I like Soros’ characterization of the oil runup, which is that there was a fundamentally-driven price increase which then has a speculative element in addition to it (and here I mean speculative in the narrow sense: anyone who is not a seller or buy of the raw material itself).
Who in the dot-com era wanted commodities? They were unloved and very cheap. Some investors took note, particularly Jim Rogers (he started on the bandwagon in the later 1990s) and Marc Faber. Then you had some validation by the consultants to institutional investors, who are always looking for a new angle so they can claim to be adding value: commodities as an asset class (there are academic studies that question that concept, BTW).
But even with increased attention, cheap interest rates gave greater preference to leverageable assets, that is, ones that generate cash flow, particularly since agents got clever at how to lard them up with fees and still have the appearance of them being a viable investment.
Then when the credit crunch began, you have reports of hedge funds moving into commodities because they could be levered (contracts have low margin requirements) when their prime broker lines were cut, Not sure how true that was.
On the ag commodities side, it appears to be a combination of bad harvests (Australia in particular is normally a big exporter which has fallen considerably short due to its droughts) but the impact of the oil runup via biofuels appears to be greater than recognized. From a Guardian story today:
Biofuels have forced global food prices up by 75% – far more than previously estimated – according to a confidential World Bank report obtained by the Guardian.
And even there, there have been anomalies, such as increasing failures of futures contracts to converge to physical prices at contract maturity, and high prices for sugar despite large inventories
On metals, gold (and to a lesser degree silver) are inflation hedges. You may have seen stories that gold’s use in jewelry is plummeting at current prices.
Industrial metals ex copper (I’m not certain of fundamental conditions there) are reported to be in considerable oversupply. Metals are cheap to store. Readers have written me saying that companies they work with have four year stockpiles! This fits the Krugman pattern of “prices in excess of supply and demand lead to inventory accumulation.” Big time, apparently.
Thomas Palley argued yesterday that the runup in iron ore is due in no small measure to oligopoly pricing.
Anon 11:05 PM;
One of the other markets which I think Krugman recently used to ‘prove’ no spec in crude oils was iron ore; so far as I could tell he never bothered to ask whether there might be such thing as an ‘iron ore cartel’ and how it may have driven price. Google the term and see what you come up with, I’ll only say that it has little to do with free market theories.
Another that I seem to recall as being ‘proof’ is platinum group metals such as rhodium. Do you think funds haven’t speculated in these?
If you go back to 2002, you will find that at least one very large investment bank had begun recommending its clients diversify into commodities. Did this happen all at once? No, but gathered steam until by January 2006 it had become sufficiently evident to provoke one major global bank’s commodity analyst to note:
A flood of investment funds is driving base metal prices much higher than can be supported by fundamental analysis of supply and demand. It’s a bubble which could grow a lot bigger before bursting. (Alan Heap & Thomas Price, Citigroup, Beyond fundamentals – the funds phenomenon, 25 January, 2006)
‘JD’ gets it on the oil price system; if you would like to read a short section on how the present price regime came about, see Section 2 in the below linked paper:
http://www.oxfordenergy.org/pdfs/WPM31.pdf
It is not a matter of ‘an open question’ but historic fact. An ‘open question’ would, instead, be why seemingly so few have bothered to perform basic research but allowed problematic theories, beliefs and possibly desires to guide their pronunciamientos contra what has been an evident financialization of commodities.
My answer to that one is overly long for this time of night.
Buena Suerte
This looks more and more like hot money generated by monetary and treasury policies chasing commodities.
Clearly if there is a mania or an imbalance in market opinion, futures prices will get driven up in speculation since those writing futures contracts will demand a higher premium because of the perceived upward momentum risk.
It is this momentum risk premium that is driving the oil and commodities boom. When the momentum stalls, the boom will go boom like 4th of July fireworks.
That is my big gripe with commodity futures is that they represent momentum, not intrinsic value. Their notional value is not representative of actual commodities. I advocate limited commodity futures to a limited multiple of the monthly spot volume at settling. I know that this would be bad for brokers, but it would give farmers all the hedging that they badly need.
Yves, Pardon, I’d not seen your comment re. iron ore. Still, the oligopoly pricing may be something I’d mentioned either at Palley’s or AB. It’s sufficiently well known in any event.
PrintFaster,
One thing to keep in mind re commodities futures markets (and I don’t have the data readily at hand, but I have read this repeatedly, so I am pretty confident the statement is accurate) is that the trading volumes on commodities markets are very small compared to that of stock and bond markets. So if you subscribe to the weight of money theory, that it’s too much dough cashing too few assets, it wouldn’t take a very big reallocation by major institutions in aggregate to have a price effect.
In other words, yes it may indeed be hot money, but it would take only a fraction of the action that product the dot com and real estate bubbles.
Juan, no need to apologize. In fact, the point re checking on Google allows skeptics to cross check Palley.
The comments here sadden me. It seems that most of you are pretty intelligent people yet you seem to be falling prey to conspiracy theories. Essentially, you appear to believe that various components of markets are driving prices in a manner inconsistent with underlying fundamentals. Specifically, they are being manipulated. You seem to be arguing that market economies are disfunctional. So my question is, after you have reached this conclusion what do you propose for an alternative?
There we have it at last in plain English from JD: in oil, futures prices *are* the real prices. In principal, short sellers would serve as a brake on momentum speculators, but the small size of the crucial oil futures markets relative to the geared speculative money pouring in has sheared the fingers from the short sellers several times during this past Spring, as has been remarked in comments here, and commentary elsewhere. Without the control of shorting futures, refiners are garroted by their limited just-in-time inventories of raw oil into paying the quoted price: they can’t wait, have no alternative sources, and can’t short the market down. We have a forward squeeze on prices playing off the upward fundamental momentum for oil and agricultural prices of recent years, just as you say Yves, following Soros’ summation.
Academic economics and market economics play by entirely different rule books. One is theory; the other is praxis. Guess which one we live by?
One thing missing in economic text books is any discussion of dynamic market pricing. All of the discourse that I remember was always about some sort of steady state.
Any sort of mania whether it be tulip bulb, 20s US stock market, dot.com, real estate, is all about momemtum being the engine driving the mania.
Economics can predict the steady state, but there is no model for momentum based price action. Essentialy what happens is that momentum begets momentum, partly because of the perceived increase in wealth due to momentum.
The best descriptions that I have seen talk about the physics model of state change, a variant on catastrophe theory.
@tom lindmark
I don’t see many discussion of conspiracy theories, merely people pointing out that commodities prices are in an unsustainable bubble. I don’t think anyone proposes that the dotcom bubble was driven by conspiracies, yet it plainly existed, and burst. Are market economies dysfunctional? I could ask, is a bubble economy functional?
You know, that is no different than Wal-Mart saying “many our of customers are speculating that the price of razors is going to increase by 50% in the next 12 months, and are locking in their delivery contracts today by purchasing dozens of units, and since they are expecting prices to increase, we will do exactly that…. today”. Of course they could follow that pricing strategy if they wanted, but that in itself doesn’t cause the clearing price of the market to increase unless someone takes the other side of that trade — people must agree to buy at the higher price — it doesn’t really matter how Wal-Mart sets the asking price, someone must sign on the dotted line or it’s meaningless. No one oil producer has a monopoly on the entire market.
I don’t buy your explanation, it’s a bunch of nonsense.
Let me add further, that anybody who sells anything will also try to set an asking price that maximizes their profit. It’s no different then how Burger King prices hamburgers or Wal-Mart prices vacuum cleaners.
pointbite:
thanks for clarifying all this … because, up until your comments, I did not understand that razors and vacuum cleaners at Wal Mart, and burgers at BK were purchased through contracts that linked the actual price paid to futures markets…. I had, nonsensically, always thought I was expected to pay the price on the price tag…..
I think I saw a graph in FT today that was showing non-commercial open interest in the oil futures at NYMEX (that is, OI by people other than oil comps and people who actually use oil (petrochems, electricity comps etc.).
The highest was around the $80 mark, and there was a couple of spikes after $100 (each of them lower than the previous), and right now it was pretty low.
How about this theory: There are different qualities of oil. Wouldn’t it make sense for you, as a producer – especially if you think this is relatively unsubstantiatet price – to sell more of the worse oil, for high price, and keep your quality oil in the ground?
That would mean: The price of the quality oil would go up (scarcity), because there’s a marginal price they are willing to pay for the better oil (which, in my meagre understanding is quite high). I speculate that as the lower quality oil is priced off the high-quality oil, it would bring up the sour one up at the same time, making you better gains on this one.
Think about it as apples. You can buy apples only from me. Should I sell my best ones first, and risk that when I will be selling the partly-rotten one someone else will show up with peaches and all I will be left with is a load of composteable stuff; or (assuming that people cannot craving fruit), should I sell the slightly bumped, rotten etc. ones first and keep the best till the last (as a good apple will still be able to compete with a peach)?
Assuming peak oil (at least local), it might be a viable strategy to make the sweet/light oil scarce (and thus expensive), and get the bad oil out first. If you use up your best stuff first, people will have higher incentive to find a better source when all there’s left is the bad one – so you might not be able to sell the bad one. If you sell the bad one first (because, after all, people have to buy it in the end, right now), and then switch to the quality stuff (which you know you will sell much more than the former, and people will have smaller incentive to do something until you almost run out of it).
There you are, a conspiration theory (which I can see as a valid business strategy) on the producer’s side!
I can’t help but feel that this explanation far from settles anything, and is likely to be itself proven to be somehow misinformed.
As an extreme skeptic, my first question is, if the explanation of how futures speculation could determine the ultimate price so damned simple, why is it no one else noticed this? And why did this JD wait so long to let us in on it?
I have no idea where the truth lies, and it seems obvious that oil prices will eventually fall simply because if they don’t they will lead to some profound intermediate-term demand destruction.
But this explanation seems analogous to Krugman’s in that it is suspicious simplistic and comes from someone with no real expertise and certain preconceived notions.
This could be one of those things people argue about for years to come.
I do not believe that the current price of oil is high. It think that it is low. I also think that “old” price was ridiculously low and led to the “auto transportation bubble” in the USA. The latter partially was made possible by the effects of dissolution of the USSR with the subsequent looting and dollarization of the region which had driven oil prices down as well as crazy monetary expansion in the decade after that (1991-2001). I
In other words the current price is partially connected with the weakening dollar and fading perspectives of converting oil-producing subset of xUSSR countries into banana republics. Actually both “blowback” policy aspect from looting of xUSSR region and weakening dollar story were so far ignored in the posts. Just compare the dynamic of the price of oil in euro and in gold with the dollar dynamic.
Also Greenspan-inflicted easy monetary policy (with narcissistic and obsessive interest rate masturbation which kept rates below real inflation rates) was another historic aberration that is now being fixed in a very interesting way.
If we assume that oil is limited and is vital for chemical industry commodity, then burning it to heat homes or in SUVs should be stopped ASAP and that’s what market is doing, but very slowly and inconsistently. The current generation have some obligations toward future generations, is not it ? In this sense policy of oil producing countries should probably be more restrictive as this is endowment for the future generations of people in those regions (and the mankind in general).
Actually the absence of rational energy policy on a state level is a crime and punishment might well be lying ahead. I think that the current European price ($8 per gallon) is more realistic from the point of view future generations, then the current USA price and should probably be used as a guidance for the future. At least that might help to fix the bad habit of driving Suburban to the mall instead of driving a bike. the latter also help to cut the amount of useless imports we consume :-).
I am just curious.
A future is an obligation to deliver or take oil and as far as i know, those oil futures are settled in physical delivery.
So, all speculators for obvious reasons want to get out of all obligations on expiration date – unless they want to drown in or dig for oil to fulfill the contract.
So,shouldn’t around expiration date, the real price of oil become visible ?
As all media say the demand for oil is increasing, the buy side is stronger and prices increase.
In order to get those speculators out, oil producing countries would just have to put enough oil on the market to satisfy all demand. Exemplatory, if this is done for one future contract, all speculative buyers need to get rid of their long positions and drive the prices down.
However, for obvious reasons, the oil producing companies/countries will earn less by fulfilling all demand.
Krugman probably did make some mistakes in his assumptions about the workings of the oil market. Of course, JD probably did too. Yet the undeniable fallacy behind both their analyses is that this market is subject to simple explanations. The world is almost always far more complex than we give it credit; our heuristics have to be seen for what they are. It’s probably best if the idealogues on both sides of this debate calm down a little. The narrow positions each has staked out are almost certainly incorrect to some degree. The truth is likely some muddled middle ground. I’d like to see some solid policy ideas about the “speculation problem” (if it exists) and then we can weigh the costs of implementing the policies against the potential benefits.
apologies, but this is the least sophisticated post i’ve ever seen on this amazing blog.
it establishes absolutely zero, because, while accusing krugman of ignoring futures mechanics, it confirms his thesis by completely ignoring inventory realities — as if cost of carry and convenience yield were stupid anachronisms and had nothing to do with the price of real oil, in the present. absurd. my fave part is how the alleged “manipulation of spot” that led to the overdetermining futures market is elided.
the moral: don’t trust anything written by somebody whose blog is named “peak oil debunked.”
Hi there Yves,
Peak Oil Debunked completely misunderstands oil supply dynamics and how they apply to oil pricing. In the post of theirs you quoted earlier in the week, POD purported to add up changes in demand and supply across major countries operating in the oil market both as producers and consumers. POD then concluded that since they perceived the net changes in the supply and demand balance appeared (at least in their figures) to be small, they decided that changes in supply and demand were not the proximate reason for changes in prices.
The fallacy in this reasoning is quite serious. POD is saying, in effect, that an increase in oil usage in a net exporting country such as Saudi Arabia of, say 300 kb/d has an EQUAL effect on market prices that a 300 kb/d increase in usage does in a net consuming country, such as India. Absolutely false, and this is crucial.
If the Saudis use 300,000 more barrels of their own oil per day, these barrels are no longer available in the export market. The Saudis use their own oil at subsidized prices. The Saudis are NOT bidding against the Chinese for the incremental use of their OWN oil. The Saudis are subtracting 300,000 barrels from the supply of NET EXPORTS, which is the supply of oil for which international consumers must bid. The key number that determines the international price on the supply side, the price that net consuming countries must pay for their imports, is not total supply but NET exports.
It is crucial to understand that when the Saudis use 300,000 more barrels per day and total world supply is constant, supply that international consumers can scramble for has been reduced by a significant amount, and the price should go significantly higher given crude’s steep price elasticity.
It is proper, of course, sum the net changes in demand among NET consuming countries to arrive at effective changes in demand, for the supply that is available on world markets. One should also sum up the net changes among net exporters, to get the total net export number. But to sum demand changes among net exporters and net importers is highly misleading. To do so is a basic error of summing apples and oranges.
Yves,
I wonder what JD’s qualifications are to comment on oil and oil markets. Has he taken Geology or Fluid Mechanics (the mass conservation equation comes to mind)?
Once and for all: (a) Demand and supply are terribly inelastic in the short term so that doubling or halving of the price of oil is nothing mysterious. So is food. (Yves, draw your conclusions from your article on the effect of a small amount ethanol on world food prices.) (b) Only producers can effectively “manipulate” oil prices by producing more or less. They tend to do both — the industrialized world has benefited a lot form that (remember 1998?)
I propose a new definition of speculation: Any change in price we do not like. Example: We have speculation today but we did not have speculation in 1998. Another example: The US Government has (or has not) manipulated the price of food through its biofuels policy.
JD is correct about how spot and prompt future prices are related, but appallingly confused about the implications.
Suppose that I’m a producer of Permian basin light sweet crude oil, 42 degree API. I have the choice of locking in next month’s production on Nymex at the contract closing price, or I can market my own production daily based on customer posted prices at the refinery plantgate. In both cases, I’m responsible for transportation to the storage tank at Cushing, OK.
Transacting at Nymex gives me convenience, and that’s worth something. But if the refinery posted price (the spot at a particular location for my specific grade of crude) varies too much from the Nymex price, I can arbitrage the difference PROVIDED I HAVE STORAGE AND TRANSPORTATION CAPACITY, by no means a given.
So, really, the spot is illiquid. It’s determined bilaterally and posted in publications such as Platts’. But that doesn’t matter, because the futures price is an arbitrageable benchmark.
The converse is also true. What happens if “rampant speculation” drives up the futures price beyond the spot price? Well, then I should be shorting the Nymex delivering that crude to the refinery today. The only reason that I wouldn’t do that is if I couldn’t find the transportation and storage capacity to do so.
Does the same hold true for illiquid blends, such as Maya, Lloyd, or Edmonton? Absolutely. I can take the Nymex price less differential, or I can deliver to the plantgate and take my chances.
JD says that because the plantgate is based on Nymex anyway, I have circular price discovery. But that’s not true. If there’s a pipeline break and the refinery has to buy a lot of spot oil to keep running, spot prices in that location for that time period will run higher than Nymex. If the crude quality in a basin changes, as it has in Western Canada, then desirable blends will increase in price relative to undesirable, at least until processing capacity is increased.
So the notion of a spot price differential to Nymex is a mere convenience of notation; it’s not a statement of economic truth.
My long-winded point is that this is how futures markets WORK. They aggregate supplier and consumer behavior into a portfolio of standard contracts.
Here’s the issue: every commodity contract requires a long and a short, and right now the shorts are terrified, having looked stupid for the last 5 years now. This is not the same distinction as commercials and non-commercials: it’s vacant-headed that a trade is any different if put on by BP than by Goldman. So you have producers and other natural shorts refusing to play in the futures market, combined with natural longs needing to lock in prices and gain certainty. Result? Higher prices. Is this “rampant speculation”? Sure, if you think that risk management is speculation.
And yes, you have ETFs and other pure financial players coming in, using their 9 figure portfolio to to participate in a 12 figure market. The thing that conspiracy theorists miss out on is that the financial players MUST SELL OFF THEIR POSITIONS before the contract expiry, as they have no capacity to receive physical barrels. So the Yale endowment goes long July crude starting Jun 1, but it has to sell that position off before contract closing on June 20-ish, otherwise it had better rent some storage capacity. If you can’t hold physical, you can’t move the market. Period.
How does this cycle end? The same way that it always has. High prices destroy demand, faster than people can believe, and the inelasticity of supply and demand start working for the consumer rather than against him. Only producers were crying when oil hit single digits in 1998. We’ll never go back to that, but anything can happen in the next year.
Paul Krugman-majored in economics (though his initial interest was in history) as an undergraduate at Yale University. He earned a Ph.D. from MIT in 1977 and taught at Yale, MIT, UC Berkeley, the London School of Economics, and Stanford University before joining the faculty of Princeton University.
Wiki bio.
So some guy named “JD” from a blog called “Peak Oil Debunked”, which carries the disclaimer that peak oil is in fact a fact, is gonna school Paul Krugman because Krugman is “confused” about the market? Happy fourth.
Aeon
Great post macndub!
Any info on how much physical storage space is available? Obviously if we had unlimited storage space, the price could go ballistic. It would be interesting to know if there has been a dramatic increase in the building of new storage facilities.
Also, I don’t see any commentary on the fact many governments such as China and Mexico subsidize the price of gas at the pump. Hence there is no demand drop from these consumers as prices rise!
Have you cats touched on the fact that most of this trading is dine on The ICE exchange, which is un-regulated in terms of US law.
https://www.theice.com/homepage.jhtml
http://www.oilfuturesmarketfacts.com/
Re: In recent weeks and months much has been said about ICE and the way our London-based futures exchange, ICE Futures Europe, is regulated. Much of this commentary has been inaccurate and misinformed. ICE Futures Europe is a fully regulated exchange by the UK Financial Services Authority (FSA). Like all exchanges, ICE Futures Europe undertakes extensive daily position …
Chicago,
It is not correct that buyers of futures contracts can take or make physical delivery. The only ones that are permitted to do so are those who have demonstrated delivery and storage capability. I am not clear on how that works on a practical level (as in how customers get designated as being able to take/make delivery).
macndub,
The point re index buyers is not that they are participating, but that their aggregate commitment to commodities strategies has increased. A buddy told me two new oil ETFs were announced (launched?) this week. The contention is that increased commitments from passive investors can have a price impact.
The much derided MIke Masters testimony to Congress does provide a good deal of data on the fund flows into these strategies.
Further, there has been a very large increase in OTC activity, which by definition no one can track, but it appears to be considerable. Moreover, there is evidence that the OTC traders are doing deals with commercial buyers and sellers (for instance, the New York Times reported on AIG buying grain from a farmer, paying to have it stored for six months, and selling it back to him at a pre-set price. Thus one cannot assume that all OTC action would be hedged by trading on futures exchanges.
As to the question on physical storage, we have mentioned several times that oil storage is costly and limited. This quote came from Dan Dicker:
…in the world of energy trade, there is lack of fungibility that has always broken down thus: Electricity, non-storable — Natural gas, more storable — Crude oil, mostly storable
Although easiest of the three, oil storage has been historically inelastic, no matter the price… Over the last 4 years (and for most of my trading life) forward stocks have always hovered between 50 and 55 days — storage is expensive and limited, and just not efficacious.
This is why, at least in my view, that supply arguments are often overblown in oil pricing theory — supplies remain closely aligned to demand and rarely overrun — as OPEC members have time and again explained but are ignored. This is why President Bush can walk in to a meeting with Saudi ministers and be patted on the head like a silly schoolboy — “you don’t understand, Mr. President — we’ve got nowhere to SELL any more right now” and Bush will run home and talk about increasing domestic supply as if he hasn’t heard a thing
There is also a lengthy discussion in that post from Philip Verlegger on the role of oil inventories, and how producers look at the potential profit of holding inventories versus other uses of funds in deciding how much inventory to carry. I suggest you read the post.
As for the defense of Krugman, in this case, he has explicitly said in his posts that you don’t need to understand how futures in these markets work, you only need to look to inventory levels. If they are high, prices are high. If not, prices are correct. He has offered more elaborate iterations of that stance and has not considered (indeed, appears unwilling to consider) whether there might be structural or behavioral issues that make the market inefficient.
In general, I don’t know why some readers are trying to make the information by JD mean more than it says (I think some people are very put off by his blog’s name).
The discussion in the press and policy circles has assumed, as we said at the outset, that futures prices do not influence spot prices, that spot prices are where price discovery in the oil market occurs. JD provided information from reputable sources that says that is not the case, and gave the history as to why price formation shifted from spot to futures.
I don’t see why this notion is that controversial. You see it in other financial markets. and that development often has unfortunate side effects. For instance, credit default swaps prices now determine the prices at which companies can issue cash bonds. And imbalances in the CDS market HAS led to distortions in the prices at which companies can sell debt, as we discussed earlier this year. This section of the post comes from Bloomberg:
Credit trading models used by Wall Street have gone haywire, raising company borrowing costs even as Federal Reserve Chairman Ben S. Bernanke cuts interest rates.
General Electric Co. is one of five U.S. companies rated AAA by both Standard & Poor’s and Moody’s Investors Service, making its ability to repay debt unquestioned. Yet when the Fairfield, Connecticut-based firm sold 2.25 billion euros ($3.35 billion) of five-year bonds last week, its annual interest payment was $17 million higher than on a sale nine months ago.
Borrowers from investor Warren Buffett’s Berkshire Hathaway Inc. to Germany’s HeidelbergCement AG face the same predicament. Yields on $5.12 trillion of corporate bonds tracked by Merrill Lynch & Co. average 2.05 percentage points more than U.S. Treasuries, the most since at least 1997.
The higher costs are an unintended consequence of securities that allow investors to speculate on corporate creditworthiness. So-called correlation models used to value them have become unreliable in the fallout from the U.S. subprime mortgage crisis. Last month some showed the odds of a default by an investment- grade company spreading to others exceeded 100 percent — a mathematical impossibility, according to UBS AG.
When supplies are tight the price for all oil is the price of the next barrel produced
An important thing to remember is that all of this speculation and \or manipulation couldn’t take place with a plentiful supply of oil.
Why is the price the next barrel produced in a tight market? Because of oil depletion and the extra energy and investment needed to pump older deposits. We now see many formerly capped wells being brought back into service because they are seen as economically viable for at least $100 a barrel oil. Why the discrepancy? Because of the wide swings of oil prices in the past.
Any new investment must take into account the time lag of production with historical swings in price. Add this to the fact that newer deposits are in more difficult geographical areas, are smaller and in deeper deposits. As far as oil production, the low hanging fruit has been picked.
So with supply tight and demand increasing, increased existing production and transportation costs, long range investment risk and the low hanging fruit having been picked prices have nowhere to go but up.
OBTW
The world is now consuming 6 barrels of oil for every new barrel discovered.
A recent paper from Legg Mason supports the idea that speculation is a cause of the high price of oil. It basically says that the demand for oil is taxing the world’s immediately available production. In this environment, relatively small increments in supply or demand can have outsized effects on price. “The incremental “investment demand”, from institutional investors allocating a portion of their assets to commodities, primarily through the futures market, has created just such an outsized effect on oil prices.” The quote is from the Hong Kong newspaper The south China Morning Post, the link is subject to registration, and the link is too long to post.
So everyone’s arguing about whether oil prices reflect speculation or fundamentals.
For consumers, does it matter? In either case, the answer is to use less oil. If the increase is fundamental, then we might as well get used to it. And if the increase is speculative, then a decrease in demand is the first step towards popping the bubble.
Though I’m personally surprised that serious economists are giving so little attention to long-only funds. No one disputes that speculators can influence prices in the short term; any of the usual arguments for how speculators decrease volatility assume that. But these arguments also assume that the speculators are making good decisions. They’re supposed to buy when the price is “too low”, and sell when it’s “too high”, and thus provide negative feedback. But what if they get it backwards? In that case, they become a destabilizing influence. (And lose money, of course.)
A speculator who always wants to go long (i.e., an “investor” in commodities) provides negative feedback on the way down, but positive feedback on the way up. Any fundamental price increase will therefore get amplified. That does not seem inconsistent with what we’re seeing today.
We have both The Dotcom and Subprime Housung Bubbles to use as models of modern supply and demand dynamics and thus the resulting impacts of speculation. Unregulated mkts left to themselves will wreck havoc with equilibrium, as speculators flood markets with inefficiencies which distort fundamentals. If we allow trading in futures to become too much of a casino we will end up in a market that feds off chaos and the resulting entropy of confusion.
This is why we have investors delaying delivery and withholding supplies as they manipulate on-the-fly the performance metrics linked to dynamic contracts that are adjusted every second along the delivery route. This price fixing manipulation is illegal and linked to the collusion of the commodity akts!
The contention is that increased commitments from passive investors can have a price impact.
It cannot, unless they have the ability to store the commodity, which, as you point out, is expensive (in terms of engineering, land, environmental costs, physical plant, and credit support).
A long-only ETF must sell down its position every month before contract expiration, and reinvest in the next month’s contract. Think about what would happen if everyone did this: you’d get a contango in a falling market. Contango, because demand for the prompt + 2 contract would be driven up at the beginning of each month. Falling market, because the financials would have to sell down their position throughout the month.
For the long only, prompt contract investor (ie, ETF) this is a rapid road to ruin. I cannot think of any investment strategy that could possibly evaporate capital faster.
macndub,
You appear to be missing my point. The statement was that INCREASING commitments from index buyers, which is what we have seen in recent years, CAN have a price impact.
Prices for most physically traded oil, per the EIA via JD, is set via an average of various futures prices. If you have passive index investors who must sell their contracts each month (note that some have latitude and are not limited to a monthly roll, but we’ll stick with base case), the effect of increasing commitments to the strategy is that they will buy a larger number of contracts the next month. No doubt there is noise in the pattern, but the aggregate amount of passive money dedicated to commodities has grown in recent years. In addition, more retail ETFs have been created in the last year and a half as well.
And in fact, that monthly roll strategy DOES wind up being ridiculously costly; we discussed this in early 2007. John Dizard at the Financial Times has written about the way the dealers (it turns out Goldman is the prime suspect, not locals) exploit that strategy
I was describing the practice known as index roll congestion, or “date rape”. This involves profiting from the requirement that public investors’ positions in commodities indices be “rolled over” from one contract month to another over a known five-day period. The price of the old month’s contract is depressed and the price of the new month’s contract is inflated. This can be a huge source of profit for those ready to take advantage of the naive public.
In the column, I was correct to point out that index roll congestion costs people who use indices such as the Goldman Sachs Commodity Index something in the order of 150 basis points of return a year. Given that formula-managed commodities index funds have $100bn in assets, of which GSCI-linked funds account for $60bn, a lot of money is being lost by the public to someone….
had thought that the mountain of index fund money, with its fixed, known periods of buying and selling, would be a source of profit principally for the speculators.
Actually, the problem is that there probably isn’t enough speculative capital relative to the huge weight of the index funds. And, one might add, firms that manage the index funds. Firms such as Goldman Sachs.
Locals besieged me with e-mails insisting on their innocence and said that Goldman was likely to be the principal beneficiary from the index roll. I finally did get a response from the firm. First, Goldman pointed out that it was not the only seller of funds – or notes or swaps – linked to the GSCI. (It is, however, the largest user of GSCI-linked product). Thefirm said it was obligated only to deliver the closing price on the reference days for each commodity contract in the index.
That means Goldman knows the size and position of the target it must hit and can, as its people say, “manage our corresponding position”. That means that it has to deliver a price at the end of the roll period. If it can cover that obligation at a better price, it will, and pocket the difference.
Moreover, we have the unknown impact of OTC trading, which is reported also to be increasing, and per the AIG example mentioned in the New York Times, some ARE contracting for storage.
There’s a simple way to resolve the debate. Simply apply normal speculator position limits to the new breeds of speculators. The funds (public and institutional) are exempt from the position limit rules.
Just make ALL non-users of oil, etc., play by the same rules rather than creating special exemptions for the benefit of Wall Street. In fact, if an annoucement were made that position limits for these boyz were under consideration, we would probably learn very quickly whether they were having much impact on the prices of oil and other commodities.
While I don’t agree with all he said, this issue was raised by Gene Epstien in his Barron’s column last Monday “A simple old Reg that needs dusting off.”
TomD
Okay, let me try again.
Long-only index funds need to roll their positions. If the flow of money into long-only funds increases, it would steepen the contango. Spot prices always reflect the supply, demand, and inventory position on the ground, at the time and place of the trade. Regardless of the trade’s notation.
We agree that in the base-case, prompt ETF, the selldown during the roll costs a lot of money. a lot of money is being lost by the public to someone….. That someone is the owner of storage facilities, by the way, who is the only person that can buy the underpriced (date raped) August contract and immediately lock into a September sale. The financial-only traders have to eat it. Many banks lease storage for this reason, to keep this trade in their drawer if they need it.
Consider any other long-only future index strategy. Suppose you buy a 2010 swap (an OTC product) today, hold it for the next six months, and sell it some time in Jan 2009 and roll it into a 2011 swap. Now, suppose that the rate of money flowing into this trade is increasing. As a result, 2010 swap prices are increasing. Owners of storage see a steeper contango, lock in the more favorable storage spread, and drive up the spot price by filling inventory.
Therefore, the increasing rate of forward contract speculation drives up spot prices (or prompt Nymex, if you prefer), but only through the mechanism of increased inventories. And inventories aren’t increasing much right now, because we aren’t seeing a contango and the storage spread is currently brutal in light sweet crude. If increasing capital into long only funds are even partially responsible for high light sweet crude prices, then where is the contango? Where is the increased inventory? (I would look in China, myself, but until it’s found, there is no evidence of speculative demand).
Krugman is dead right in this case. Speculation = inventory increases. Period. And, as you point out with oil, there is a physical limit to how much can be put into storage. Eventually, the contango will build and storage will be full. And the pure financial traders will utterly explode–contango + falling spot market. See Amaranth.
By the way, I’m not put off by the blog title, “Peak oil debunked.” Peak oil, the notion that we can’t scrape up additional production even as oil prices increase, is bunk. What is not bunk is that the world can’t consume oil at the American rate. It would require 400 million bbl/d. Demand caused this problem. Demand destruction would solve it, but not without prices that hit the pain point.
Maybe we’re there now. It can still get worse. But focusing on speculation is like invading Iraq to stop the next September 11… a distraction, pure and simple.
On the ‘must make physical delivery’ notion, what does Nymex say?
The Exchange’s core energy futures contracts…stipulate physical delivery, although deliveries usually represent only a miniscule share of the trading volume — less than 1% for energy —
(NYMEX Energy Complex brochure)
What did the IMF have to say about this in 2005?
While the new investors could be instrumental in translating expected future fundamentals into current prices, excessive activity based on limited information may lead to a disconnect between the futures and physical markets. In particular, excessive activity by newcomers or herd behavior by investors may exaggerate the impact of concerns about current and future supply conditions… Given that only about 5 percent of futures contracts are ever delivered as a physical product, increased uncertainty can encourage speculative behavior in the futures market. This, in turn, may push up futures prices beyond that warranted by future market fundamentals.
(The Structure of the Oil Market and Causes of High Prices, 21 September, 2005)
Physical deliveries through the ICE? Who knows?
I have often said in this room that the benchmarks we use in gauging the state of the market, such
as Brent or WTI, do not always reflect a change in the physical market fundamentals. As a result
of the vacuum created by the eclipse of OPEC’s power, the market is now increasingly dominated
by pure financial players mostly involved in non-commercial trade and moved by their perception
of future price trends in which the interaction between supply and demand play a minor and
diminishing role.
…
The most important lesson that we learned from the exceptional market conditions
experienced over the last couple of years, in particular since the beginning of this year, is that oil prices have taken their direction primarily from non-fundamental factors which frequently
overwhelmed market fundamentals. Clearly, it is difficult to rationalize the continuing market
nervousness on the basis of simple oil supply, demand and stock levels. Extraneous factors, such
as geopolitical uncertainties, exaggerated anxieties over resource exhaustion and speculative moves play, today, a preponderant role in determining day-to-day oil price swings. Speculators, in particular, seize every traumatic news headline to reinforce their collective belief that prices will be driven even higher. The recent excessive popularization of the “peak oil” theories is an example of how the undeniable fact that oil is a depletable resource can be exaggerated to produce a sort of self -fulfilling prophecy, i.e. “oil prices soar because everyone believes that oil prices will soar”.
(Nordine Ait-Laoussine, President of Nalcosa, a Geneva-based consulting firm, and a former Energy Minister of Algeria, 27th Oxford Energy Seminar, September 15, 2005)
Saudi Aramco, whose pricing system is followed by most exporters in the Middle East, insists that the buyers of its crudes are refiners and will under no circumstances be resellers.
It has a dynamic marketing unit, which applies formula pricing for crude oil sales adjusted monthly to the following benchmarks and markets: (a) the Brent Weighted Average (Bwave-1), which covers the extended trading session as the basis for its crude sales to Europe and gives protection against price distortion; (b) Dubai/Oman for the east of Suez, and (c) spot WTI for the US and the Bahamas.
The prices of WTI and Bwave-1 are set by futures trading on NYMEX and the IPE.
(APS Review Oil Market Trends, 8 October, 2007)
Is this really so hard to understand — if so, please take a few minutes to study the history of changing oil price regimes. The three page section 2 in my above linked PDF should not be a chore.
Notions of market efficiency have not and do not apply yet continue to form the basis for what must only be called “understanding” by so many who, apparently, do not care to understand.
Someone above wondered why it is only now that the futures as benchmarks comes to light. No matter, it’s been known for years. To expect MSM, most pundits and politicians to bother going beyond preconceptions is, clearly, to expect too much, especially if these same are caught up in the business of selling ‘doom’. Back in the later 1960s, LTV founder James Ling said something on the order of ‘first create the problem and then sell the solution’. Ask what is being sold.
Someone else above asked whether ‘JD’ was a petroleum geologist. To my knowledge he is not but I certainly have a close friend who is and happened to operate as an independent oil producer for over fourty years — he believes price of WTI Light Sweet is 75% higher than should be the case. But heck, he is just another old ‘denier’.
It’s a nice theory. But the reality is a little more complex:
1) Yes, it’s clear that futures prices and spot prices are linked. They do not diverge and have not diverged significantly in the last 20 years.
2) There’s nowhere near enough evidence to make technical pronouncement on causality here. Whilst there are reasons to believe that spot prices are influenced by futures prices (trading for delivery etc.) the evidence is that as soon as you’re talking about price changes of more than $10 a barrel, the market does not have these priced into futures. As such, futures move around to reflect changes in spot prices. Which is what has happened in the last year. Futures have moved from $70 to $130, following the move of the spot.
In essence, there’s no reason to believe the market has any ability to price future information into the futures price and so it isn’t currently ever going to diverge much from the spot price so making distinctions between them is probably irrelevant.
Anonymous who commented on my comments:
What’s the difference? Whether you have futures or not, SOMEBODY HAS TO PURCHASE THE PRODUCT for the spot price to change. It doesn’t matter what the asking price is, if I am selling ANYTHING (yes, vacuum cleaners and hamburgers included) I will always try to get the highest price possible, why the hell would you expect oil prices to be any different? So ridiculous. If Saudi Arabia could sell their oil for $500 per barrel without losing market share they would do it, and I promise you some stupid futures contract calculation isn’t the reason they don’t.
Annonymous at 5:57 AM said:
“Futures have moved from $70 to $130, following the move of the spot.”
This is also one of the arguments that Krugman makes:
“And here’s one more fact: by and large, futures prices over the period of the big price runup have been slightly below spot prices. The figure below shows monthly data from the EIA; as the spot price shot up, the futures price (that’s contract 4, the furthest out) actually lagged a bit behind.”
http://krugman.blogs.nytimes.com/2008/06/23/
macndub: “Here’s the issue: every commodity contract requires a long and a short”
The “a seller for every buyer argument” is misleading as it ignores buying/selling pressure. And that’s the whole problem. Too much buying pressure.
macndub: “The thing that conspiracy theorists miss out on is that the financial players MUST SELL OFF THEIR POSITIONS before the contract expiry, as they have no capacity to receive physical barrels. So the Yale endowment goes long July crude starting Jun 1, but it has to sell that position off before contract closing on June 20-ish, otherwise it had better rent some storage capacity. If you can’t hold physical, you can’t move the market. Period.”
Rolling has little to do with avoiding delivery anymore. The whole point of the otc and swaps is so that there is cash settlement only. The bbl amounts are merely “notional.” At expiry, an exchange of the differential between notional values occurs between dealer and customer. No bbls change hands. That’s the whole point of this problem. The notional value of the markets is now orders of magnitude larger than the actual physical market.
GS and MS go into the futures market and hedge themselves by buying exposure. These guys do have storage capabilities. This gives them much more latitude during roll periods — if they can’t find anyone to take their contract, they can take delivery. But then they can deal that oil to refiners on their own clock.
macndub: “…it’s vacant-headed that a trade is any different if put on by BP than by Goldman. So you have producers and other natural shorts refusing to play in the futures market, combined with natural longs needing to lock in prices and gain certainty. Result? Higher prices. Is this “rampant speculation”? Sure, if you think that risk management is speculation.”
Before anyone thinks they are “vacant headed” for disagreeing with the all-knowing “macndub,” why don’t we cite Mack Frankfurter:
“…traditional forces of supply and demand cannot fully account for recent prices. To be precise, the normal price-inventory relationship has been altered…Specifically, dynamics have changed because securitiezed commodity-linked instruments are now considered an investment rather than risk management tools“
As JD pointed out, cash markets are woefully illiquid. According to Richard J. Feltes, Senior VP & Director of MF Global:
“weak basis levels preceding and during delivery month reflect the fact that there is more demand for futures longs (via index funds) than there is for cash. And although there is not necessarily a shortage of cash (commodity) for sale, there is a shortage of futures for sale amid an index fund business model for carrying long positions for extended periods.”
FROM:Confused in Ann Arbor
After reading through these posts confusion (at least my own) seems to be running rampant. Major contentious points.
1. Some believe a futures contract always results in physical delivery somewhere.
2. Others believe that only a small percentage 1-5% of futures contracts expire in delivery.
3. Yet more assert the majority of players in the oils futures markets are net-long, and there exists a scarcity of shorts.
If there are an absence of shorts, how can the market function with a growing number of longs? Do the commodity exchanges themselves pay off the opposite bet, since it appears no actual delivery takes place? It seems an impossible situation if there is no one to play a put or take short position, akin to perpetual motion. I’m aware that in the metals markets bulk owners of gold or silver lend out their stocks for long or short positions. Is that what’s going on in the oil markets?
It seems a growing interest in oil futures must always result in a growth of both short and long positions in volume of dollars. Obviously this is only absolutely true at expiration.
So where the heck is the longs profit coming from?
Here are some random stats from an article in today’s wsj(not sure if sub required):
“CFTC said 85% of index investing is done outside of regulated futures exchanges.”
“$5 trillion futures market for oil and other commodities”
“The Bank for International Settlements, a global body that surveys central banks, puts the notional value of all over-the-counter commodity instruments at $9 trillion.”
And regarding buying pressure (yes, there is a seller for every buyer…but sellers ask for higher and higher prices…that’s why markets go up):
“A CFTC study released last year showed that while commercial traders as a whole are net sellers, swaps dealers were typically net buyers of the near-term futures contracts that are quoted as the Nymex benchmark.”
“Because an estimated 50% or more of this market consists of instruments related to crude oil, a report from research company ISI Group says over-the-counter oil trading could be as much as 18½ times larger than the total oil bets outstanding on the main regulated energy-futures market, the New York Mercantile Exchange.”