hi all. the dream of trading stocks via algorithms and thereby generating profit is an old one. it occurred to me as a teenager and have spent quite a bit of time over the years dabbling/ investigating/ researching it. there are many remarkable stories going back decades now. one of the early pioneers in this area is Doyne Farmer and have tracked his trajectory in the field over many yrs, starting with the breakthrough 1994 wired article. the growth in the field of automated trading also closely mirrored/ matched very major advances in machine learning.
the topic has emerged in the forefront lately with many refs to HFT, high frequency trading in the media. the shocking 2010 flash crash put it on the radar of many, incl the public, regulators/ political leaders. yesterday there was a congressional hearing with the chairwoman of the the SEC, White [a2] (and theres my cue to finally write up my big bag of great links collected over several years). the 2010 crash is still somewhat mysterious and controversial/contested but some evidence seems to point to triggering by a rogue or malfunctioning HFT program (but clearly the whole crash, like all crashes, was a complex cascade).
the topic is big in the news lately with Michael Lewis book. a great writer who can bring arcane topics to the masses. but, its being called sensationalized and it does seem to be verging on one dimensional in its HFT-negative pov. its tone/theme sound a bit shrill and lewis’s irreverent style while highly entertaining strikes me at times as approaching a bull in a china shop. a more evenhanded/balanced Patterson is another very effective writer on the subj. saw the great documentary “ghost exchange” recently, a remarkable and engaging documentary.[d5]
this overall is a very complex field to study with many interacting parts. a few themes emerging over the last years
- the sophistication of the algorithmics and brainpower in this area is staggering and has increased dramatically in the last few years. its a real life spiralling technology arms race. there has been a mass flux of very high trained scientific & mathematical & engineering types into this field (as a taste [d10] is a very insightful ref by Loveless in ACM). some amazing theoretical elements for sure but unfortunately a lot of it is tightly protected company trade secrets.
- there is a fear that technology is outpacing regulation and orderliness and affecting market stability, but the reality is very delicate to determine precisely.
- the issue of transparency vs opaqueness is huge here. even experts in the field say they cannot comprehend its complexity and the whole thing operates like a large “black box” (which does not inspire confidence in its stability).
- quants definitely had some role in 2008 crash but my thinking is that they are more the canaries in the coalmine, possibly getting hit first, rather than being a causative factor. some blame bad modelling but to me, the greed of corps not restrained by regulations tends to drive optimistic models. the tail wagging the dog. (or the dog wagging the tail from the pov of the fiscally-borderline profiteers at the time.)
- SEC/regulators seem to be lagging and not able to keep up with changes. a technological backwater.
- sometimes there is more “heat than light” on this occasionally extremely contentious subj. there are rumors and claims that short term traders are gaining profit at the expense of long term traders (eg institutional investors and large retirement funds etc.)… this is difficult to prove outright. there will always be overhead in trading stocks and it is not clear if HFT extracts or minimizes those costs (by “providing liquidity”). its likely a mix of both. even academics who study this very closely cannot really answer the question of whether HFT is a net negative or positive force. again it is likely both and it is virtually impossible to separate/disentangle one from the other.
- this technology is indeed highly disruptive and likely squeezing traditional human traders. but, to me this sounds like the old story of the jacquard loom and the luddites except in a very high tech context.
- think some of the “sky-is-falling” fear is nearing hysteric/hysteria when a simple animation of trading communication is called “terrifying” by the media headline… how is HFT robot trading different than communication routers passing messages? in many ways the two are are virtually indistinguishable. there is a lot of fear based on lack of knowledge and even those that attempt to become intelligently well informed are overwhelmed by the daunting complexity and inherent mysteriousness/ impenetrability of the subject.
- do however think that the US economy has been over financialized with too much going to the financial sector and HFT is certainly in the crosshairs here. but how do we decrease overall financialization? it is a problem that arguably cannot be attacked with conventional tools eg Fed interest rate adjustments etc. … re revocation of Glass-Steagall firewall, it appears that large elements of banking have morphed into speculation and we have a large increasingly centralized banking CARTEL in this country (with estimates about 5 banks controlling more than 80% of wealth/deposits) that has been aided substantially by IT innovation. to me that is the real issue/elephant in the room and the epic political capital to take on that reality is just not here or anywhere on the horizon when even the 2008 economy-theatening crash did not lead to any major structural reforms (FDR II, waiting for Godot…).
- the stock market has an intrinsic “commons-like” property such that we are struggling to determine what is good or bad for the ecosystem, and individual agents/participants do not have the overall health of the system in mind in their individual actions, and while usually pursuing their rational self-interest leads to a coherent market, occasionally this appears to lead to catastrophe, but the exact aspects of deleterious behavior are unclear & hard to narrow/ delineate/ define. ie from this macroscopic pov what is the difference between healthy/unhealthy trading strategies?
- a dark horse/wildcard in this area is Bitcoin. hard to say where it will go but it is a new model for trading and stock exchanges. they are very much in small-scale, embryonic starting stages however real, paradigm-shifting innovation is going on here. something to keep an eye on.
- 1. Old-school traders being squeezed out by high-tech talent – NYPOST.com
- 2. SEC chair to Congress: ‘The markets are not rigged’ | Reuters
- 3. Jeff Connaughton: Wall Street Lobbies to Prevent SEC Changes to High Frequency Trading and Stock Market Structure
- 4. Eunuchs of the Universe: Tom Wolfe on Wall Street Today – Newsweek and The Daily Beast
- 5. A Transaction Fee Might Save Capital Markets… & Protect Us From The Terminator!
- 6. Honey Badger Hedge Fund: Hackers Predict Stock Market With Open Source Mojo | Wired Enterprise | Wired.com
- 7. Physicists and the financial markets – FT.com
- 8. How the “Navy SEALs” of trading are taking on Wall Street’s predatory robots – Quartz
- 9. Mark Gongloff: Knight Capital Debacle: The Latest High-Frequency Nightmare — Regulators, You Paying Attention Yet?
- 10. research – what was the quant role in the 2008 crash? – Quantitative Finance Stack Exchange
- 11. Flash Crash – Wikipedia, the free encyclopedia
- 12. 2010 Flash Crash – Wikipedia, the free encyclopedia
- 13. BBC News – Black-Scholes: The maths formula linked to the financial crash
- 14. Recipe for Disaster: The Formula That Killed Wall Street
- 15. From the archives: Beating the Market: Yes, it can be done | The Economist
- 16. Wired 2.07: Cracking Wall Street
- 1. How Moore’s Law Affects Wall St. Trading – NYTimes.com
- 2. Market Data Firm Spots the Tracks of Bizarre Robot Traders – Alexis C. Madrigal – The Atlantic
- 3. The computers that run the stock market – Jul. 8, 2013
- 4. Algorithms Take Control of Wall Street | Magazine
- 5. Robots take over
- 1. High-Frequency Trading of Stocks Is Two Critics’ Target – NYTimes.com
- 2. Mark Cuban: What Business Is Wall Street In?
- 3. Mark Gongloff: Wall Street Robots Have Taken Over The Market, Horrifying GIF Shows
- 4. This Video Of One Half-Second Of High Frequency Trading Is Insane, Terrifying
- 5. High-Frequency Trading Is Bad For Profits, Including Those Of High-Speed Traders: Study
- 6. Wealth Fund Cautions Against Costs Exacted by High-Speed Trading – NYTimes.com
- 7. High-frequency trading is bad for normal investors, researchers say – Quartz
- 8. research – Has high frequency trading (HFT) been a net benefit or cost to society? – Quantitative Finance Stack Exchange
- 9. Putting the Brakes on High-Frequency Trading – NYTimes.com
- 10. How the Robots Lost: High-Frequency Trading’s Rise and Fall – Businessweek
- 1. Flash Boys: A Wall Street Revolt: Michael Lewis: 9780393244663: Amazon.com: Books
- 2. Michael Lewis: Did Goldman Sachs Overstep in Criminally Charging Its Ex-Programmer? | Vanity Fair
- 3. An Adaptation From ‘Flash Boys: A Wall Street Revolt,’ by Michael Lewis – NYTimes.com
- 4. Michael Lewis explains his new book “Flash Boys” – CBS News
- 5. Ghost Exchange (2013) – IMDb
- 6. Scott Patterson: Is the Stock Market Rigged? (Excerpt)
- 7. How to keep markets safe in the era of high-speed trading; – cfloctober2012_303.pdf
- 8. Algorithmic stock trading rapidly replacing humans, warns government paper – ComputerworldUK.com
- 9. Book Review: ‘Automate This,’ by Christopher Steiner – Businessweek
- 10. Barbarians at the Gateways: High-frequency Trading and Exchange Technology – ACM Queue
- 11. The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It: Scott Patterson: 9780307453389: Amazon.com: Books
- 12. Amazon.com: Dark Pools: The Rise of the Machine Traders and the Rigging of the U.S. Stock Market (9780307887184): Scott Patterson: Books
- 13. The Predictors: How a Band of Maverick Physicists Used Chaos Theory to Trade Their Way to a Fortune on Wall Street: Thomas A. Bass: 9780805057577: Amazon.com: Books