New way of releasing US crop data unleashes an arms race among flash boys for first access
It’s the sort of edge any trader would covet - and one the authorities were actually hoping to prevent.
Yet the US Department of Agriculture may well be clearing the way for some Wall Street speed demons to trade on market-moving data before others.
Abandoning decades of precedent, the agency has decided to only post its reports directly on the web, rather than also release them via accredited media. While that may seem like a democratic move, it actually could set the stage for a winner-takes-all arms race to grab the information first.
The development, announced Tuesday, is the latest saga for crop markets that have increasingly seen high-speed algorithms taking over and running circles around slower human counterparts, a theme popularised by Michael Lewis’s Flash Boys. Fibre optic cables, short wave radio, microwave towers: all have been employed to trade faster.
“Somebody will figure out the fastest way to get the information and trade on it first,” said Jim Angel, a finance professor at Georgetown University in Washington. “What it does is it changes the arms race in a different direction.”
Press organisations use high-speed fibre optic lines to get data to readers out of the so-called lock-up, where journalists are given access to the data up to 90 minutes in advance. It takes the department roughly 2 seconds longer to transmit its reports to the public, according to the USDA. The agency cited this gap when announcing its policy change.
“Everyone who has interest in the USDA reports should have the same access as anyone else,” Agriculture Secretary Sonny Perdue said in a statement Tuesday.
In a statement Wednesday, the USDA said giving journalists early access “pre-determines who gets the information first,” favouring customers of media organisations that pay for fast access to information.
The changes come during a volatile period for agricultural trading. Soybean and hog futures have plunged in recent weeks as China imposed new tariffs against US farm products. Grain prices have also slumped amid ample supplies. CME Group said trading volume across agricultural futures and options set an all-time high of 3.2 million contracts on June 19.
The USDA’s monthly World Agricultural Supply and Demand Estimates report, known as WASDE, is one of the most-followed events in the grain-trading world. The August WASDE will be the first impacted by the policy shift. It’s typically closely watched as it provides the agency’s first survey-based production forecasts for corn and soybeans, the top US crops valued at close to US$90 billion.
The irony is that the USDA’s attempt to ensure everyone gets potentially market-moving information on commodities at the same time could actually do the opposite.
Rather than getting access to data early, members of the media, including Bloomberg News, will now only get the data through the USDA website at the same time as the public. The lock-up process is used by a variety of government agencies and allows for dissemination that reaches many data-trackers at once.
Because the USDA is releasing this information on a website, it’s using a transmission technology called TCP. That means the trading firm with the bot that first scrapes the USDA site will have an advantage over everyone else, a head start on placing trades in commodities markets. Other means of conveying the data could have higher odds of sending the information simultaneously.
“Each server hosting a website provides thousands of simultaneous connections by which clients can retrieve information at the same time,” the USDA said in a statement Wednesday. “Websites hosted by content delivery services scale this up to millions of connections which are available to any client accessing the website content. No prioritisation of connections is done and data is transmitted as requested.”
The USDA said Tuesday it would improve its website to handle an anticipated spike in traffic in the moments after report releases.
The change takes place August 1. That may make post-release moments more volatile while benefiting businesses with the resources to scan-and-trade fastest.
“If the dissemination isn’t handled evenly, it can create the perception that the government isn’t playing fair or it’s beholden to certain vested interests or something nefarious going on,” said Larry Tabb, founder of market research firm Tabb Group.
To be sure, there’s nothing illegal about getting publicly available data first and trading on it. And traders have always sought to get an edge. But in this modern era, an edge can consist of getting important data in just a tiny sliver of a second faster than anyone else — even a millionth of a second.
Grain trading volumes often surge after USDA releases key reports. While some firms use methods like satellite imagery to get a better sense of supply, the government data remains the benchmark in domestic and global production forecasts.
USDA officials have said they notified the Labor and Census departments of the policy decision. Those agencies use different technology to publish their report and didn’t indicate they plan to cancel lock-ups, the USDA’s Chief Economist Robert Johansson and the USDA’s World Agricultural Outlook Chairman Seth Meyer told reporters Tuesday.