By Ian Fraser, a financial journalist who blogs at his web site and at qfinance. His Twitter is @ian_fraser.
What has become of our markets? Nanex, the market analysis firm, has animated a half second of trading activity in Johnson & Johnson stock. The animation is both intoxicating and mindblowing, not only because of the sheer quantity of trades, each of which is made by computer algorithms (i.e. without human intervention) in such a miniscule timespan, but also because of the tremendous scope that high-frequency trading creates for what Nanex calls “abusive behaviour” — including arbitrage and market manipulation — and systemic risk.
The video illustrates an actual half second of trading in J&J stock from May 2nd, 2013, slowed down so it takes five minutes to watch. In the video each box represents a stock exchange. As Time magazine explained in an article last year (“Wall Street’s Doomsday Machine“) high frequency trading has nothing to do with the efficient allocation of capital, and everything to do with socially-useless proprietary trading that runs counter to the interests of long-term investors:-
High frequency trading is a catch-all term that describes the practice of firms using high-powered computers to execute trades at very fast speeds – sometimes thousands or millions of trades per second. These systems have developed over the past ten years, and began to really dominate Wall Street over the last five. For example, a high-frequency trader might try to take advantage of miniscule differences in prices between securities offered on different exchanges: ABC stock could be offered for one price in New York and for a slightly higher price in London. With a high-powered computer and an “algorithm,” a trader could buy the cheap stock and sell the expensive one almost simultaneously, making an almost risk-free profit for himself…
Nanex created the animation in order to try and explain how today’s equity markets work to head-in-the-sand US regulators, including the Securities & Exchange Commision and the Commodities Futures Trade Commission (CFTC). Here’s what Nanex said about the animation and its significance:-
We got the idea after realizing, in face to face meetings with them, they didn’t understand market structure or the importance of latency and the consolidated feed. That was several years ago. We still aren’t sure if they get it, or are just playing dumb.
The bottom box (SIP) shows the National Best Bid and Offer. Watch how much it changes in the blink of an eye.
Watch High-Frequency Traders (HFT) at the millisecond level jam thousands of quotes in the stock of Johnson and Johnson (JNJ) through our financial networks on May 2, 2013. Video shows 1/2 second of time. If any of the connections are not running perfectly, High Frequency Traders can profit from the price discrepancies that result. There is no economic justification for this abusive behavior.
Each box represents one exchange. The SIP (CQS in this case) is the box at 6 o’clock. It shows the National Best Bid/Offer. Watch how much it changes in a fraction of a second. The shapes represent quote changes which are the result of a change to the top of the book at each exchange. The time at the bottom of the screen is Eastern Time HH:MM:SS:mmm (mmm = millisecond). We slow time down so you can see what goes on at the millisecond level. A millisecond (ms) is 1/1000th of a second.
Note how every exchange must process every quote from the others — for proper trade through price protection. This complex web of technology must run flawlessly every millisecond of the trading day, or arbitrage (HFT profit) opportunities will appear. It is easy for HFTs to cause delays in one or more of the connections between each exchange.
Set to lowest resolution for an “artistic rendering”, or highest resolution for science.
Here is another explanation from Nanex (added 14 May 2014)
Each colored box represents one unique exchange. The white box at the bottom of the screens shows the National Best Bid/Offer, which often drastically changes in a fraction of a second. The moving shapes represent quote changes which are the result of a change to the top of the book at each exchange. The time at the bottom of the screen is Eastern Time HH:MM:SS:mmm, which is slowed down to be able to better observe what goes on at the millisecond level (1/1000th of a second).
In the movie, one can observe how High Frequency Traders (HFT) jam thousands of quotes at the millisecond level, and how every exchange must process every quote from the others for proper trade through price protection. This complex web of technology must run flawlessly every millisecond of the trading day, or arbitrage (HFT profit) opportunities will appear. However, it is easy for HFTs to cause delays in one or more of the connections between each exchange. Yet if any of the connections are not running perfectly, High Frequency Traders tend to profit from the price discrepancies that result.
Lambert here: I cross-posted this for a couple of reasons: First, the lava lamp-like animation is totally suitable for a Saturday night, especially for those of us who actually remember lava lamps. More importantly, Fraser’s lead — “What has become of our markets?”* — chimes very well with a consistent although minority feeling in our “Bulls vs. Bears” link collection, best summed up by Jesse: “No one I have read or spoken to really knows what the heck is going on. They don’t.” “Knows,” that is, even less than usual. So perhaps HFT — which is legal, why? — is a piece of the puzzle. Readers? What do you think HFT means for “our” markets?
105 comments
Comments are closed.
During the entire run up of the market to current record price levels, trading volume has also set new records for being so low. In fact if there were no HFT there would be virtually no volume at all. Hard to find more convincing evidence that the market is broken, and no longer exists as a means of price discovery but merely functions as a casino for players with access to zero interest loans guaranteed by the taxpayer.
The solution is glaringly simple: Simply tax each transaction at a progressively higher rate inverse to the time the equity is held. Even a flat transaction fee would do the job.
But wait a minute: In order to do that we’d have to have legislators who were not already bought—. Good luck finding more than one or two of those.
..now throw in “front running”…which is illegal…??
Grand Theft Market: High-Frequency Frontrunning CME Edition;
http://www.zerohedge.com/news/2013-05-01/grand-theft-market-high-frequency-frontrunning-cme-edition
http://en.wikipedia.org/wiki/Front_running
The markets are not the markets anymore. As long as the Fed is pumping massive amounts of money into the system, the prop trading banks can keep pushing the markets higher. There is very little volume moving in the exchanges, this makes it even easier to push them up. It won’t stop until Bernanke pulls the QE plug. The problem is he cannot pull the plug without destroying the economy. He will continue to use QE until it is no longer effective. The big question is, what will make it no longer effective?
Your understanding of QE is quite confused: QE is not “printing money” – it is an asset swap that does not create new assets – i.e. it cannot “prop up the market”.
Your conclusion is thus necessarily nonsensical as well.
Is it not propping up questionable asset valuations, MtMyth[?], alternatively, sponging up contagion.
The duration of these assets swaps are subject to price quality fluctuations [on both sides of the coin], this action requires quite the heavy invisible hand… eh.
Skippy… is it not unlike feeding the herd its own feces and calling it feed enhancer?
A Tobin tax would put an end to this socially useless nonsense.
Of course HFT is a mere pimple on the catastrophic flaws in the financial system which directs our economic destiny and our species interactions with the real world.
Yes… and it would go a long way toward making the market fair again. The normal market participants are already paying this tax… just not to the government… but instead to a bunch of “penny-stealing… wanna-be criminal… men”.
(Quote from Office Space)
I agree with Thor. HFT makes up the volume left from retail investors leaving the market. Retail investors in turn have left the market in large part due to the sense that the market is rigged, partially due to high tech HFT. HFT would easily be discouraged with a transaction tax, or even imposing a small transaction latency time, similar to the latency experienced by say, trades made from across the country using the local ISP. Pure and simple, HFT is condoned cheating.
HFT is not qualitatively different from LFT. It is legal because banning it makes no sense since it causes no harms that regular trading wouldn’t cause slower.
Better to focus on opaque financial instruments and re-instating Glass-Steagall.
Disagree. HFT has the advantage of shorter transaction or response times, multiplied over thousands of transactions. So for example, if the HFT servers detect a spike in the price of gold when Chicago opens in the morning, and the HFT’s can get in first with a bunch of buy orders on gold mining stocks, they get the best bid prices. Since their computers are right inside the NYSE building (thus having the shortest cabling to the NYSE servers, and yes, a few feet count in this game, thus all have exactly the same length cabling), they WILL get there first. The advantage may be small on any individual trade but adds up with the heavy volumes they execute. The HFT people wouldn’t have invested billions into the necessary infrastructure if they weren’t being rewarded handsomely.
Such price differences are inherent, unavoidable, intrinsic properties of geographically distributed semi-private, diverse markets, resulting not from greed but from physical properties of matter, such as the speed of light.
You could “solve” it only by centralizing it into a single market. Where should that central market be? London, New York or Beijing?
You’ll also have to outlaw secondary markets to monopolize price making – in all 200+ jurisdictions on this planet.
Do you really want to replace market economies and free choice with such a central monopoly on contracting between consenting adults?
If not, what’s your solution? I submit that there’s no other solution: slowing down the ticker won’t work, a Tobin tax won’t work unless you forbid all secondary markets, etc.
Hmm. Doesn’t the horse racing industry solve or at least address this issue with handicapping? I’m sure there’s a technical way to address that (granted, a new thing to game)…
Not sure I follow – you’ll have to be more specific.
@Anon As I understand it, and I may not, a lot the gaming and the arbitrage can take place because of latency effects in the network. That’s why it’s better to be closer to the NYSE because the laws of physics say that all other things being equal people closer will get prices earlier than people farther away. So in the same way that tracks put weights on the faster horses so there is a real race, why not slow down the packets that are destined for closer destinations to there’s a (more) globally uniform system of price disclosure? Force the bits destined for NJ into a cache or something until the bits destined of Shanghai have had time to make it over there. Obviously, I don’t have the real technical expertise to make that work, and it may be a dumb idea. One could even charge for “time in cache”…
What you propose is similar to putting the exchange on the Moon: it will be slow to access to everyone.
But this idea misses the fact that most trading happens between consenting adults, who generally pick the fastest, most liquid exchange.
So if a big exchange is put on the Moon then trading and liquidity will flow to other, faster, better platforms.
This is how over the years liquidity migrated from the pit to electronic exchanges. Pit trading still exists today, but attracts only a fraction of liquidity.
So if any of the big central exchanges moves to the Moon, freely contracting traders and investors will gradually abandon it.
@Anon There would have to be only one moon, yes.
Handicapping will not work because most of the liquidity will voluntarily flow/migrate to faster, better, non-handicapped exchanges. In fact it already does that.
Well, managing that flow would be the job of a functioning regulatory system, eh?
No. As I mentioned above trading is voluntary contracts between consenting adults – who will use the best infrastructure available on the planet.
If you want to regulate it to make it all slower you’ll have to regulate all secondary markets in all of the 200+ jurisdictions on this planet, to under a single central monopoly and regulatory system – while outlawing all voluntary contracts between consenting adults and all secondary electronic markets.
Good luck dealing with the astronomical technical and political difficulties of that.
It’s simple. Affording HFT is like affording a good lawyer, accountant, etc.
Exactly, HFT is a tool to trade your intented investment ideas.
With early HFT the “idea” was typically: “extract information from slow/dumb traders and use it to make a profit”.
The solution is simple: more people should use HFT algorithm assistance to better trade their investment ideas.
But of course we can try to slow down those cars as well, to give horse carriages a fair chance.
That’s ridiculous. You’re recommending a totally wasteful and absolutely unnecessary financial arms race. In all of those transactions, some agent gets a cut.
Blow it all up. It’s insane.
I assume you work in financial services.
This kind of competition between (voluntary) market participants to use new information faster always existed, for the past 150 years they used the fastest technology available to gain an edge – be it a pigeon, a telegraph, telephone or a computer system.
Speed was never regulated, the limit was always technology.
So now that you don’t like how fast technology has become, what do you propose to do about it?
“HFT a tool to trade your investment ideas” in what universe?
A tool to create a techno-casio arms race which generates profits for the winners is more like it.
Controlling it is simple: just place a price on the execution of each trade such that no significant burden is placed on purchases for investment and upon trading with human scale (vs auto-digital) levels of turnover, but HFT becomes prohibitively expensive.
Many big investors use HFT techniques to hide their existing positions and future intentions.
The video in the article does not seem to show any actual abuse – it shows how prices sync up in a distributed network – it’s very fast price discovery in essence. You’d see something similar if you visualized Netflix or Google server data traffic.
I do agree that activities like front running customers or jamming network capacity is asocial and should be regulated.
Your idea of taxing every order will only work if a substantial majority of exchanges adopts it, and if secondary exchanges not implementing the tax are outlawed in all significant jurisdictions.
Just a single major exchange missing from that taxing scheme would attract all the liquidity. Good luck pulling that off – tackling world peace would be far easier in comparison.
You have ignored Thor’s argument, ie you have committed non sequitur.
He may not be making his point clearly but I don’t think it’s a non sequitur.
The argument seems to be that any attempt at regulation is doomed because liquidity will flow to whichever exchange most effectively evades the regulation. There’s nothing unusual about that–it’s the case in any industry in which the cost of evading a regulation is less than the profits to be garnered from the evasion.
The solution, for an effective regulator, is to increase penalties until that is no longer the case. That might seem impossible in the present case, but I have my doubts. If, for example, the proprietors of and participants in Tobin-free exchanges were treated as international criminals, if they were subject to drone attacks should they happen to have taken refuge in a jurisdiction in which extradition was difficult or impossible, if jurisdictions refusing to cooperate were subject to embargo or attack….
It’s just a question of political will.
The quote-stuffing is reason enough to ban HFT.
DRAMATIC FAIL!
You obviously don’t understand high frequency trading and the reason it was created.
Given that I’ve worked on HFT platforms, I suspect I do know why HFT was created. There is literally no difference between the types of trades executed on HFT and the types of trades executed by other market makers/portfolio insurers and quality brokers.
While I’ve heard rumors about quote stuffing, I never observed it myself and no major players have been caught doing it.
Indeed.
Now I see why the interest in quantum computing – to outdo the speed of HFT, since all markets will be traded from teh future. Not a 3-second lead on the world, but 3 nanoseconds:
https://www.youtube.com/watch?v=nQCbdLGrlpY
Amen, Amen, Amen, Jon Claerbout. Why is this not obvious to the most casual observer?
yeah the timings about right for the gov to step in on HFT, as its so YESTERDAY :O)
stepping up to the plate…Field-Programmable Gate Arrays
FPGA based network card which is optimized for QuickFIX. FIX 4.2 Order Cancel messages are generated entirely inside the FPGA. The latency from the Order Cancel trigger to when the first byte is on the wire is 314 nanoseconds. The latency from the Order Cancel trigger to when the first FIX Order Cancel message is entirely on the wire is **1,874 nanoseconds**.
http://www.wallstreetfpga.com/resources/fix-on-an-fpga/
In contrast to processors that you find in your PC, programming an FPGA “rewires” the chip itself by allowing the user to program logic gates and allocate random-access memory blocks to implement functionality.
http://247wallst.com/2013/03/24/high-frequency-trading-profits-crushedbut-danger-remains-unchecked/
I know there’s a proverb which that says ‘To err is human,’ but a human error is nothing to what a computer can do if it tries.
agatha christie
Since when is arbitrage considered “abusive behavior” in the market? :>
It’s one thing to manipulate the market to produce an artificial arbitrage opportunity, but closing inefficiencies through arbitrage plays is hardly abusive!
The two seem to go together, according to Nanex:
Jamming thousands of quotes at the millisecond level seems like a pretty good way to cause delays in the network, which would mean that HFT is price-manipulative by definition. That’s the whole point, if I understand correctly…
When an order is placed at a brokerage, the broker decides where to route it. The client doesn’t get to pick, unless they are in a premium trading program. Why? Presumably market makers pay for order flow. I’ve seen limit orders not get filled, on the basis that better prices were at other exchanges. The best answer would be to eliminate conflicts relating to order routing. But if that doesn’t happen, and I don’t think it is, then I’m in favor of anything that equalizes prices across exchanges, including HFT.
That seems like more of a case of artificial arbitrage opportunities manufactured by brokers or shady order request methods.
But if there’s legitimately a price difference across exchanges it’s not abuse to close that gap and run liquidity across exchanges. That’s certainly not abusive in and of itself.
HFT is yet another reason all transactions ought to be taxed. It is yet another way to game a system already gamed nearly to death. At one time the financial markets made a lot of sense now less so.
specifically an Order Cancellation Tax…but they’ll just off shore that too
mathbabe writes some pro/con
http://mathbabe.org/2012/01/14/high-frequency-trading-and-transaction-taxes/
We live in a global Empire not a specific country. All markets can be internationalized and systemized if there’s the will to do that.
What makes this extra depressing is the waste of human potential it represents. So many of the brightest minds working on useless crap like HFT (and the science and tech necessary to pull it off) instead of work that could be socially useful. It reminds me of a scene from Quants – a short movie from a few years ago about the computational and algorithm side of trading. One picture was a shot of the faces in an classroom – flat, bored, affectless. And they were the smartest kids at Columbia. What a waste.
To those who have not yet succeeded
HFT programers needed
It’s a great deal
We train you to steal
Morals should be unimpeded
The Limerick King
http://www.flickr.com/photos/expd/7772873604/
Following on Jesse’s observations, HFT offers the perfect cover for an official PPT operation.
The PPT simply plugs into the systems of their favourite TBF financial insitution, or two, and away they go, smashing gold, driving the EUR illogically higher on bad data, ramping SP500 higher at 3.30pm every day, the futures at 3.30 am when Asia starts to wilt.
Think financial market “drones” cutting out intervention via rumour driven traders like they had to in the old days.
Just two or three people need to know on the bank side what is going. What a perfect set up for official control freaks – Timothy comes to mind.
To make it easier they probably don’t even need to throw big capital about because a bold tilt at a bid or offer at the right time will have hundreds or thousands of other algos following in the blink of an eye.
As someone who, unfortunately, watches the market second to second, there is absolutely no doubt that HFT allows for the total manipulation of markets, and that in fact this manipulation does occur.
A simple and very effective solution would be to impose a very small transaction tax on every transaction, just enough to make these insane, and unproductive transactions, unprofitable. Alternatively just make it necessary for any bid/offer to have to exist for 1 second before it can be cancelled. Unbelievably this too would virtually wipe out the manipulative impact of HFT.
Of course the powers that be will never allow this because it is in their interest to be able to manipulate the markets as they wish. Now that they have this power why would they ever want to give it up?
Markets are broken. Capitalism will suffer. The only question is when does it all fall apart.
how bout that 45b flash (poof) rendering the s&p @ 1,666
they laff at us steve
An intelligent hell would be better than a stupid paradise.
victor hugo
Reminds me a bit of Yves posts of ‘devolution’. Whatever justification you want to make about market efficiences or li quidity, having these systems that operate so fast and are so interconnected and that I doubt anyone truly knows how all the pieces work, is a bit scary. When the wheels really pop off at 200mph it’s gonna be ugly.
This begs that question that stock trading and even stock investing are economically useful behaviors in the first place.
mathbabe seems to think not (from today’s links)
Stock trading isn’t socially useful, but when sufficiently regulated it’s a harmless form of gambling.
Stock investing… well, if you have a capitalist economy, the oldest form of stock investing *is* socially useful. That involves a company with a specifically stated purpose which it is required to perform, and stockholders who have actual power over the board of directors.
We don’t see that very much.
The real issue is that there *also* no observable difference between so-called stock trading and stock investing except for the time period in which the stock is held.
Traders didn’t cause the panic of 2008. Structured finance, foolish insurers and crappy iBankers did. The ways in which economists typically believe traders can amplify a crash (by buying into momentum and escaping before the bubble burst) were by and large *not* observed in 2008. Both the players holding mortgages and the players shorting mortgages believed they were making an *investment* – at least according to The Big Short.
You’re absolutely right. Equities weren’t a causal factor in the GFC. That doesn’t mean that HFT can’t amplify volatility. Witness the flash crash a couple years ago that warranted pulling the plug on the NYSE. Nobody still knows exactly what happened. With HFT things can go really, really wrong and nobody will ever know why. The complexity adds an inherent systemic risk to the system.
The other problem, and one can argue whether it is true or not, it”s one more item contributing to the perception that the markets are rigged with profits reserved for only the big players. Trust is a resource in scarce supply right now. OTOH, is there an argument for HFT serving a useful purpose other than generating profits for those engaged in using it?
Not useful to most of us for sure. Just a way for evil people to thrive.
I also think the answer to HFT is a transactional fee.A percentage of the amount traded.figured in proportion to time held would be good.all the way down to nothing if you actually hold onto the stock for a year,or something.
Nevermind the nanosecond race, all this technology, and indeed the entire strategy, would become obsolete, if there were taxes associated with them. These taxes/fees could then be used to pay for, some of what corporate tax avoidance is costing the rest of us,right now.
Ever since I noticed in the 80’s and 90’s, how the “closing numbers” were reported every night, is akin to making everyone think this is a score in some ball game where we are all supposed to feel like we have a stake in a “high score” ,or something.The marketing of the stock market to the masses..
Wall street is a parasite. we just get the arrogance of guys winning rigged games and feeling justified…because no one can stop them.
With all this senseless violence going on in the world, why can’t these frustrated nutjobs, aim their guns and pressurecooker bombs at the people who deserve it.wall street types and washington regulars?
Technology is being used to solve all kinds of problems, but it is also being used as a weapon. Proprietary tools in the wrong hands allow for previously unheard of control and potential malice. The keystone cops have long celebrated and publicized the low level disgruntled hacker, or the porn hoarder, or to fire workers for looking at nudity at work, or to exert new levels of control. Meanwhile, Banks (information technology companies) weaponized (intentionally) a vast attack. Keeping information away and having someone else do the work, is a long standing tradition for this country’s rapid economic growth, look at almost any period in time. Liars loans are incredible violence, in the long standing tradition of slavery. House ni##ers don’t participate in HFT, but they are allowed to toss their thougts into blogs.
Somebody won Powerball, that’s HFT for the proles, legal, heavily secure, every state in the union, even then they’ll tell you it’s for a good cause, sport stadiums, edu’macation.
I’m never sure how young or nerdy the NC commentariat are, but I’ve always found HFT hilarious because it demonstrates that the the economies of online video games are literally better regulated than the US economy.
HFT is a classic, textbook example of a software exploit: some actors are able to generate profits and alter prices by taking advantage of highly temporary price differences that exist only due to imperfections in the underlying network infrastructure. The fact that it’s even possible for such differences to exist and be acted on is prima facie evidence that the trading system was badly designed – this is the kind of thing programmers get fired over. (Incidentally, in my opinion the correct way to fix the situation is to replace the exchange system with one designed by someone competent and honest rather than some kind of transaction tax, but good luck with that.)
If someone were to figure out a way to implement HFT on, let’s say, the World of Warcraft auction house, the practice and its practitioners would be immediately banned and that decision would not be even slightly controversial among the players. I’d venture to say that even in the real world HFT would have resulted in jail time if it’d been invented by basement hackers rather than people with a cozy relationship to the SEC – it’s essentially a form of salami slicing, which people have gone to jail for and you may remember from the movie Office Space.
Great comment on “software exploit.” Are the system architecture requirements for Massive Online Multiplayer Games similar to those of the markets?
* * *
As nerdiness and demographics, I like to think the NCers are open to any new analytical toolkit in our collective quest for understanding… So do feel free to comment from that perspective.
The stock exchange has a much larger scale (in both the number of users and the number of transactions per unit time that have to be tracked) and I’m sure there are other differences I don’t know about, but the fundamental problem is basically the same: you need to be able to handle transactions in which you have problems like varying network latency, dropped packets, etc. which can make ambiguous the order in which the users at different ends of the transaction did things or result in users making decisions based on out-of-date information. Over the years comp sci academics have worked out various methods for solving this which ensure that all parties to the transaction get what they think they’re getting, the transaction takes place only once, etc. Latency exploits (which HFT is an example of) have been around in online gaming for as long as online gaming has existed, and have been used to do everything from robbing other players to duplicating in-game items.
I suspect HFT is being used as a lever to help exploit the liquidity running out of The Fed’s QE program, but QE itself is the primary driver of the rise in equity markets.
HFT is useful as a highly sensitive stethoscope, by which HFT participants will be able to detect when the market begins its downturn, ahead of the muppets who will shoulder the losses. This will be important, since The Fed doesn’t want HFT participants (of which TBTF banks are members) to fail.
The whole point of the hyperactive participation in equity and commodity markets is to help build the war chests of the TBTF banks so that they can weather the next downturn without an explicit bailout from Washington.
http://the-small-r.com/2013/03/05/bankings-excess-reserves-are-a-war-chest/
QE works if the goal is keeping zombie banks from dying It has been too targeted to trickle down into the economy. The Fed would have to do public QE to get a public effect. They would have to pay off half of everyone’s mortgage or something like that if they wanted to help the economy or stabilize housing. But, that has never really been their goal. The Fed’s concern is not the real economy. They are so divorced from the real economy they don’t even want full employment as part of their mandate anymore. The only reason that they would want full employment is to keep them from having to arrange a market crash for bank profits. It is much easier to destroy than build. All the Fed wants to do now is support banks, fund corporate welfare, and protect their Ponzi scheme.
I dunno, on the warchests point. I remember being two degrees away from people with actual money when Lehman went down, and the picture I got of matters on high was that people were running around screaming and waving their hands and shutting down their terminals and going out to get drunk because the amount of money in derivatives was unparalled, like bigger than all the stars in the universe or something. So when it all goes pear-shaped again, will HFT’s warchest be big enough to cope? And how much really is skimmed off by the HFTers? Do we even know?
Good question. I think if you include all people living as parasites on the system though the figure would be well over 50% of wealth is created through criminal or almost criminal activity. Some years back Misha Glenny said that something like 15-20% of all economic activity is criminal and at least twice that is what I would call “nearly criminal.” Front running, legal insider trading, programmed trading of all kinds, derivatives and a number of industries like health-care which (in the U.S.) skim somewhere around half the cost to what I would call corrupt and often technically illegal practices.
This is not sustainable, and as I advise anyone–cultivate private networks, forget about “reforming” the system it cannot be done–I have yet to read about or hear about a scenario where anything much can change, and spread love wherever you can.
“So when it all goes pear-shaped again…” (Lambert)
Never enough of Daumier —
http://cdm.lafayette.edu/cdm4/item_viewer.php?CISOROOT=/mdl-prints&CISOPTR=1482&CISOBOX=1&REC=1
How much is skimmed off by the HFTers is unknown, but to clarify, I wasn’t suggesting HFT was being used directly to fill the war chests (although I’m sure it’s helping). Rather, HFT algorithms are useful for keeping an ear on the ground while stocks pursue a run-up unbased on substantive economic activity. When the music stops, the HFTers will be the first to know and cash out, leaving the muppets in the lurch.
Ah. Thanks!
UPDATE Adding… What builds their warchests is rigging the market?
Not sure if this is what you’re looking for:
The market rigging comes from the QE, wherein liquidity is loaned at very low rates, and at very large amounts, to TBTF banks and protege hedges. These large amounts allow market caputure in equities and commodities, creating temporary phantom markets that produce returns -as if- substantive economic activity undergirds the trading.
The trick, of course, is in the timing, when the TBTF banks and hedges must bail on the markets before -the- drop… and it must be a substantive drop, not a short-term reaction or correction. For equities, HFT algorithms provides one type of monitor, and a good one.
Of course, as you already suggested, even if the retreat is properly orchestrated when needed, whether the profits realized will help prevent any given bank’s collapse is unknown. Here is where the whole thing may be a pipe dream, in that the outstanding derivatives are so large that the war chests may not be sufficient.
Nevertheless, the -intent- behind the war-chest operation is still there and in the short term, is truly bending the markets.
And to clarify, the war chests are not dependent on some future success in market timing. The TBTF banks have -already- made billions off commodities and equities. Again, whether it will be enough remains to be seen. The multiple maneuvers to save the banks in 2009 worked at least temporarily, and continued maneuvers at the margins may be able to keep everything upright indefinitely (with a “???” hedge added in parting).
Suggestion: Instead of open bidding, stock markets could collect sealed and binding offers at regular intervals, say a minute, and then cross overlapping offers at a midpoint that maximises the volume of trade.
Sounds better than my handicapping idea. Essentially, slave all transactions to the same clock.
Just wait till they start HFTing labor and do labor derivatives to offset any losses.
Susan, within you burns an admirably wicked essence!
Yeah, what do you mean “our” markets.
This is an important subject unfortunately this article and the discussion so far come accross a little bit as an attempt to instill fear on people and generally weak on arguments. HFT is a sort of skynet like force that is going to swallow us all, just look at the lights moving, isn’t that scary, isn’t that evil?! Well joking aside feels a bit luddite.
High frequency in itself (executing trades very fast) is not neccesarily bad. Automatic or algorithmic trading (writing computer programs that execute trades) is not neccesarily bad. It seems to follow the trend of automation and optimization that information technology has brought to other industries.
It might be that (some forms of) HFT as it’s currently practiced are toxic and this is a worthy discussion but the case should be stated properly and evidence provided. The discussion should start with historical examples, references of studies proving or hinting at it, how it affects the average investor who holds for non HFT-relevant periods, etc. There are also arguments that HFT contributes liquidity, market efficiency, etc. how important are these arguments? advantages vs disadvantages?
I don’t know the answers to some of the questions above and I’m prepared to listen to the evidence either way but a video of some fast lights and shouting “the machines are coming!” sure is not a good frame for the discussion.
PD: I know this subject has been brought up before in NC, in a more insightful way. I’m personally still not convinced and I think this merits more discussion as opposed to resting the case and enjoy like pigs in the mud (as much as we all like that, I know I do).
Tobin tax: small tax on every transaction.
The most fundamental problem with high-frequency trading is that the speculative HFT volume swamps the real investor volume in the market, making the prices meaningless. When the HFT trading is 90% of the trading, the prices will swing wildly and have nothing to do with the interests of any of the actual stockholders.
This can actually create some really massive profit opportunities for big-money long-term investors (making their money at the expense of HFT trading), but it’s socially awful because it ruins the value of public quotes.
Tobin tax gets rid of the HFT nonsense.
Where is the evidence to support that assertion? HFT tends to be short term based, no overnight positions, etc. I have yet to see good evidence of long term effects or it render prices meaningless. If I look at the price chart of Johnson&Johnson it looks quite smooth to me, especially on relevant timeframes (3-6m).
I think there’s good evidence that HFT has created temporary glitches or price moves in the past (flash crash, Knight capital) but I’m not sure about systemic control of the prices, which is what you imply.
Actually, the May 6 flash crash was caused by a large, manually entered order, not by any HFT.
The Knight capital incident affected them mostly – as it should – HFT shops can go (near) bankrupt at high speed as well, while picking nickels.
There seems to be a consensus that HFT, although it didn’t start the flash crash it did make it much worse. Agreed about Knight.
I think there’s evidence that HFT causes more movements intraday and they can be significant in the case of something rare happens and algos react badly. I don’t think this isn’t impossible to fix.
There are laws of physics and information that – clearly spell out – the inevitable out comes of these types of applications, tech only further leverages the out come.
skippy… yet gamer’s will be gamer’s… as large slice neurological/etc. scans and testing are revealing. Virtual reality’s do not correspond well with physical reality’s, like one big Enders Game being played out. If memory serves, the hole point was – annihilation – of ones opponent.
Market myopia… barf~~~
PS in movies soon, let see if they do it justness…
http://www.youtube.com/watch?v=vP0cUBi4hwE
I’m not sure I follow but I sure want to smoke the same stuff
I under stand the math utilized to construct such applications and their interactions within the gaming platform know as the market exchanges. I also confer with individual’s that understand the psychological aspects such endeavors bring about, in addition the hard and soft ware tech which facilitates these activity’s.
For a taste:
http://rpgresearch.com/blog/role-playing-gaming-can-potentially-provide-greater-catharsis
How many times have they had to re-harden VaR or any other mathematical market tool since 08?
Derivative models shown to be mathematically incoherent.
This list keeps building, yet, at the end of the day it is the physical reality that allows this game to be played, one has billions of years of Empiric Seigniorage.
skippy… fact-less retorts are just a form of armchair priory. Enjoy your game whilst you can.
I don’t understand your point very well, seems to me you just muddy the issue. The internals of the HFT programs are irrelevant here. We are discussing if the existence of HFT is nocive or not, and in what way. If some of those algos use VaR, flawed models or other internal rules, too bad for them but I don’t think this should alter the discussion, in fact we should have a systems that is robust against bad behaving algos if it’s to be trusted.
It’s like law and thought. Law doesn’t care what you thought process is to perform your actions, just avoids (or penalizes) harmful actions.
You attempt to isolate HFT from the market in a T or F narrative, attempt to humanize it, and not question the validity of it in the first place.
Do you have a grasp on – or grok this stuff…
Information and Entropy
Course Description
This course explores the ultimate limits to communication and computation, with an emphasis on the physical nature of information and information processing. Topics include: information and computation, digital signals, codes and compression, applications such as biological representations of information, logic circuits, computer architectures, and algorithmic information, noise, probability, error correction, reversible and irreversible operations, physics of computation, and quantum computation. The concept of entropy applied to channel capacity and to the second law of thermodynamics. – MIT
skippy… look in a closed lab experiment, yeah good times, but, as an ongoing experiment* in real time (*virtual religious numerology applied to physical world) and the hi-jinks of the past as a base line… I’ll just invoke simple sampling as my optics… the intellectual grunt applied is a factor of the paycheck deployed… not scientific observation – theory.
PS… its a gamers exploit, full stop.
I have to be honest here my background is computer science and I still haven’t a clue what you’re getting at. The discussion *I* was having was definitely about the viability of HFT but now I feel more like I’m discussing my tarot reading. How does information theory resolve this discussion? Your coursera summary is not enough. I hope you elaborate, this is going to be either very enlightening or we get to the bottom of the gibberish, please don’t be afraid to be technical.
IF your point is that all HFT is gambling/gaming and it shouldn’t be allowed because of that, I think that’s a valid but broader discussion since it’s applicable to much of modern finance. Surely we don’t want to argue that doing something bad slowly is ok but doing it fast is not.
Dark pools and HFT are a form of information arb, so there’s that effect (a subject heavily unpacked on this blog, in the day – have a look).
The speed and complexity of HFT will be its own undoing according to Information and Entropy laws (fat tail leverage by computational power will manifest at some point), imagine if everyone in the market constructed and deployed personalized algo’s, yet still, the battle is about physical distance to the node @date. Further to the point, when one thinks they have control over such abstracts, is when the opposite its actually true. Its all theoretical, same with VaR, Derivatives, FFS.
Modern finance is becoming disconnected from the diminished physical realities of this world, hence numerology, claims on the future without time travel. Dork of Cork points out some of those highlights on this blog.
@holygrail… yeah its like that… a quest… with in the exchange nodes of the biggest MMOG eva.
The actual physiology and psychology of such MMOG’s is probably the most influential tool of it all. That’s where the B grade gamers get pawnt by the owners, those who build the map have advantage right from square one.
skippy… I’ve watched the online gaming evolution and the academic studies done, the crowd sourcing, the online real time experiments, AI, helped build bits and pieces (like some top secret project where you only see one bit at a time and not the hole) and it scares me… like the ipads influence on a particular part of the brain (look it up).
PS. this is not the market, for better or worse, of our ancestors. This is – something – different. Good luck out there.
Here is some of the evidence:
http://www.bloomberg.com/news/2011-10-05/with-high-speed-trading-market-cannot-hold-commentary-by-mark-buchanan.html
and here:
http://physicsoffinance.blogspot.com/2011/09/high-frequency-trading-downside-part-i.html
Thanks, interesting articles. 3 out of 4 of them point to the same event, the flash crash (the other article seems to be about an exploit but it’s behind a pay wall). I think this very strongly hints that HFT does induce or increase disruption under unexpected events. Definitely that should be thought about and possibly some stopgap measure implemented.
Still doesn’t seem to be there is evidence that HFT has a meaningful influence long term, and it does seem spreads and liquidity is better because of them, so doesn’t it look like we should address the existing problems instead of declaring the technology evil?
Personally I think it’s inevitable that most of the trading eventually switches to electronic, same way as people use databases and not paper now and same way as cars will eventually drive themselves. I think we need to make sure we fine tune the technology when it’s emerging as opposed to demonize it.
It’s far from certain that HFT narrows spreads and improves liquidity. More and more articles are being published disputing these claims, along with distorting true prices to retail investors. You wanted data? Check out this summary of a two month study done using regulatory data from the CFTC.
http://www.zerohedge.com/news/2013-03-12/when-hft-steals-liquidity-exploratory-trading-emini
And it’s not necessarily doing something bad slow or fast that is the issue, it’s the unfair advantage given when some have a fast access that literally allows them to use (cancelled) trades to create new prices, while others have slow access that prevents this. It would be called cheating in any other scenario.
From another CFTC study, two years this time (none have been done on equities:
http://www.bloomberg.com/news/2012-12-25/high-frequency-trading-prospers-at-expense-of-everyone.html
This is more the kind of discussion I was looking for, thanks. This goes along with the evidence that intraday activity is being shaped and perhaps bullied by HFT to a certain extent. Again, still no evidence for bad effects on long held positions, let’s keep this in mind: I don’t think that an investor who thinks in fundamental terms that TESLA is going to do well over the next 2 weeks is affected by HFT, which is what’s implied by some of the other comments.
Regarding “unfairness” related to distance, I think it’s a red herring and amounts to more luddism. Please note that to a retail investor (or any human trader for that matter) this is milliseconds, it’s marginal. Even if all servers had equal latency, an algo will always be faster than you. Co-location (machines next to or in the exchanges) is about beating other computers, not to squeeze retail investors. And yes, I am afraid that technical traders and strategies that rely on fast execution or minimal margins/high volume are no longer viable without computers the same way as a letter is never and will never be as fast as an email. You could create a level field for all electronic trading and you will still face very much the same issues so this is a minor point and I don’t even think this is negative, let’s move on from watching charts and trying to speculate on sub-second price movements on stocks, leave that to the computers.
Anyway, back to short term effects, seems there’s a tension in liquidity and volatility that might need addressing. To me that’s the real issue I see that remains backed by evidence. How to address this, if it needs addressing?
Red hearing indeed…
High-frequency computing — the use of extremely fast computation to track and
measure rapidly changing data — has moved from the laboratory to the world of hightech trading platforms, where it is used to automatically analyze stock data with the
ability to execute millions of transactions per second. High-frequency trading (HFT) generated about $21 billion in profits in one year alone, 1 and was estimated in 2009 to
have accounted for 73 percent of all equity trading volume in the United States. 2 HFT technology is all about lightning-fast performance and lowest possible latency.
Competition around the fastest processing times for completing trades has arisen among the exchanges, who collocate HFT servers with the machines driving the marketplace itself to minimize latency due to distance. In 2000, HFT trades had an execution time of several seconds. In 2011, speeds of computer connections measured in milliseconds and even microseconds have become a critical factor in the practice. Ultra Low Latency Direct Market Access (ULLDMA) is a burning issue among investment institutions from major investment banks to small hedge funds, as well as among technology vendors such as IBM® and Intel®. System configuration strategies, such as collocation and the use of large multicore servers, **are employed to further reduce latency by keeping network hops and transport distance to a minimum.**
2011 but still relevant and much more!
http://www.theedison.com/pdf/2011_Samples_IBM_eX5_HFT.pdf
Figure 1. page 3. has a nice graphic illustration.
http://www.cisco.com/web/strategy/docs/finance/c22-658397-01_sOview.pdf
skippy… The capability to leverage distributed computing and parallel processing techniques dramatically transformed the landscape and dramatically reduce latency. There are special cases, such as High Frequency Trading (HFT), in which low latency can only be achieved by physically locating servers in a single location…. en fin.
Your point being? There’s an obvious race in HFT for maximum speed and efficiency, nobody disagrees but I don’t see how that’s a problem in itself. As I said this is a battle between computers and exchanges to begin with. To a human trader, a co-located FPGA array or a computer under my table doesn’t make a difference. 2ms vs 200ms doesn’t make a difference, he’s still much much slower. Ie: manual technical day trading is doomed regardless, and from my point of view, rightly so.
What we’re discussing is if these increasing automation harms legit investment.
Again I see your post as an attempt to cause fear based on scary sounding tech sound bites. The technology to minimize latency and increase throughput is not something special to HFT, it is all pretty standard in IT and meaningless in itself. Let’s focus on the issues.
The Volume of HFT trading is dominate, that in its self is a distortion in market information.
Whilst were at it, humans make the ware and algos, hence:
The Goal of Consistsency is a Cause of Information Distortion
http://forum.johnson.cornell.edu/faculty/russo/The%20Goal%20of%20Consistency%20as%20a%20Cause%20of%20Information%20Distortion.pdf
You have a multi tier problem and its not just HFT or dark pools, it us. So the line of questioning/argument is myopic. You have to step back and look at everything, including the humans that create it and not drop tired old market truisms as a priori concept.
Tech leverages individuals or groups efforts beyond the human scope, for the most part ie. how many people thought derivatives were hot shit before they blew up the world, how many people even understood them, and yet in the end were found to be incoherent.
skippy… you dally over technical minutia in a means to argue effect, good or bad, when the track record of such machinations is rubbish. The more complex and the more leverage applied the bigger the boom, its evident.
> The Volume of HFT trading is dominate, that in its self is a distortion in market information
This is too vague to be useful. Distort how? How much? What consequences? That’s the conversation we are having. I think the evidence is that there’s a liquidity and volatility distortion intraday, no evidence for long term distortion (the drift of the stock for fundamental causes). What to do about that?
I’m not sure what your proposed solution is but according to your posts trying to scare people of the technology itself, it seems to me you are saying “people are killing each other with swords, let’s ban steel”.
I think the discussion that you are interested in having is if and how finance is broken at the root and how to deal with that, which I think it’s very interesting but more complex of course. Right now you are considering your position that electronic trading is bad proved a priori and arguing from there, it’s very difficult for me to follow that up.
@holygrail… how are algos treating JPY today, what portion of that sigma 4 does it own, as its dominate, what does an algo stampeded look like… eh.
skippy… real world debugging… good times… good times… eh.
You keep hiding in vagueness, lack of evidence and cryptic remarks.
HFT in FX is still less significant than in stocks as market is much larger. Evidence in FX is still more patchy than in equities and if anything evidence points to a mixture between beneficial and yet unknown effects:
http://www.bankofcanada.ca/wp-content/uploads/2012/11/Clara-Vega.pdf
http://www.bis.org/publ/mktc05.pdf
what is that 4 sigma you talk about? 4 sigmas with respect to what model? weren’t you the one who didn’t like models? Where is your evidence that JPY moves today are due to HFT “stampedes”?
I have to voice my unwillingness to continue this conversation. You assume your position is correct and not subject to be challenged or defended with evidence and rational discourse.
@holygrail…
I called you out on this position:
“Regarding “unfairness” related to distance, I think it’s a red herring and amounts to more luddism.”
I provided fact and you provided opinion w/the luddism trope.
Now all anyone has to do is check out the wiki page to see the pros and cons:
http://en.wikipedia.org/wiki/High-frequency_trading
My fav is pulling out the volume rug when things are shitty, risk off, no skin in the game.
Alternatively on can go to an industry forum and read the uncertainty with their own two eyes.
http://www.wilmott.com/categories.cfm?catid=38
http://www.wilmott.com/messageview.cfm?catid=38&threadid=93923
You provided evidence that co-location makes algo trading faster, which I never challenged and wasn’t the subject. I said that co-located or not, electronic trading is and will always be faster than humans therefore your paragraph about high efficiency co-located machine *IS* a red herring because if it didn’t exist at all we would still face the same issues.
Again you assume that electronic trading is nocive, so faster electronic trading is more nocive. I challenge that fact. I say that it’s just a natural evolution of trading in the first place. If trading in the first place is not nocive, electronic trading is not either. If trading in the first place *is* bad, then electronics has nothing to do with it.
Now, there’s some fine tuning to do to that logic. Some stuff points to electronic trading distorting intraday liquidity and volatility. I accept that and perhaps needs addressing. That’s the point in the conversation where I am. You run along and assume *all* electronic trading is plain bad for all uses, and that’s where I get off the train unless you provide better proof.
Remember this:
https://www.quantnet.com/threads/explosion-rumor.13224/
“Re tarted news algo’s, makes the market into a super correlated asset dumping everything in seconds. How can we respect a market that has signs of such stupidity.”
“Trade-off: people rather be fast than smart. This is another reason why I´m not into HFT.”
BTW, i didnt make or loose money, I saw the volume spike and stayed clear of it
http://www.nanex.net/aqck2/4176.html
For all the sell side “its good stuff” [unproven imo] it has already had some BIG oopsies, its all theoretical, the laid off derivatives kids fell into it after lay offs [sub prime ring a bell] and best yet is… as I type top educational facility’s are churning out quaint’s and techies that are assembly line institutionally configured…. kids without life experience or ability to contrast their world view unless it comes from some ithunkit portal.
skippy… the hardware is fine in of its self [you should see my stuff], its the basis for HFT [or any other virtual monies skimming app] that I deplore.
PS. disclaimer… totally out of the market… electronic gaming brings out the stooopid in people… read the research i offered and expand. Try using HFT to move you across a real mine field and send a post card… eh.
@holygrail… how about NextEra Energy (NEE) and American Electric Power (AEP).
That was fun watching 30B go poof in under one second and then called a trading abortion, I mean aberration. Do you understand the ramifications of such an event? The winners and losers effect. Seriously, its just a version of Bitcoin with some software bolted on to mine electrons of price.
Wait for the “HFT is a bubble comment” ROFLOL…
skippy… like I linked to and tried to inform… there – is – and *mounting* psychological evidence of – the dark side – to MMOG platrom activity’s, social impact on both an individual and societal level.
I understand where you are coming from with this and don’t get me wrong, I think that HFT probably had an influence in that event. However I think you should really take a step back and look at it with a cool head. You are copy-pasting comments from an internet board. If you scroll down a little bit more you will see people with diametrically opposed opinions. Who’s right? Why? This does not evidence make, it’s all internet gossiping and chit chat, you’re not going to convince anyone looking at the problem seriously with that.
Was this event, like the flash crash, *caused* by HFT? Probably not, just it made it worse. Would it have happened anyway, over a slightly longer timeframe? Is the HFT effect bad to begin with (wouldn’t it just be a loss for algos)? Also this is an “unexpected event”, which is a case I already conceded HFT might influence. Still no information or evidence against the backbone of the fundamental drift of the stock.
We need more and better studies about this, that much I’ll agree with.
@holygrail… did you read the header to this post?
You offer opinion, I have offered mine in addition to links of peer reviewed works [application and larger social effects], industry white papers, industry forums, etc.
The very fact that 50ish% of – total market volume – is a direct result of theoretical math and physics, which at the end of the day are location constrained and being de-bugged in real time [try that shit out on a star ship to mars]… is insane. 30B market cap in less than a second, do you grok that?
What part of the – in the last decades of making money financial innovation blowing up in our face – did you miss.
skippy… go back and read the paper on decision making processes – aka- cognitive dissonance. All you see is money – en fin.
There is also:
http://online.wsj.com/article/SB10001424127887323798104578455032466082920.html
as well as…
http://online.wsj.com/article/SB10001424052748704791004575520363764665240.html
HFT is nothing more than than taking advantage of technological edge that you might have over other market participants (software + hardware + communication lines), though it does not last forever. From my own experience I can tall that building such a system requires advanced engineering skills (software and sometimes hardware related) but overall it is not a rocket science. Personally I do not think it has any economical value for the market in spite of claims that it “improves liquidity”. If a rule that requires a limit order to sit in the market at least for half a second will be adopted, the HFT will be dead.