The Telegraph My details My newsletters Logout Upgrade to Premium My details My newsletters Logout The Telegraph The big data race reaches the City being sifted for the good of commercial interests as never before Credit: Bloomberg 8 October 2016 • 2:08pm IBM’s Watson supercomputer, once known for winning the television quiz show Jeopardy!
 in 2011, is now sold to wealth management companies as an affordable way to dispense investment advice.
 Twitter has introduced “cashtags” to its stream of social chatter so that investors can track what is said about stocks.
 Hedge funds are sending up satellites to monitor crop yields before even the farmers know how they’re doing.
 The world is awash with information as never before.
 According to IBM, 90pc of all existing data was created in the past two years.
 Once the preserve of academics and the geekiest hedge fund managers, the ability to harness huge amounts of noise and turn it into trading signals is now reaching the core of the financial industry.
 Last year was one of the toughest since the financial crisis for asset managers, according to BCG partner Ben Sheridan, yet they have continued to spend on data management in the hope of finding an edge in subdued markets.
 “It’s to bring new data assets to bear on some of the questions that asset managers have always asked, like macroeconomic movements,” he said.
 “Historically, these quantitative data aspects have been the domain of a small sector of hedge funds.
 Now it’s going to a much more mainstream side of asset managers.
” banks Banks are among the biggest investors in big data Credit: Jason Alden/Bloomberg Even Goldman Sachs has entered the race for data, leading a $15m investment round in Kensho, which stockpiles data around major world events and lets clients apply the lessons it learns to new situations.
 Say there’s a hurricane striking the Gulf of Mexico: Kensho might have ideas on what this means for US jobs data six months afterwards, and Many businesses are using computing firepower to supercharge old techniques.
 Hedge funds such as Winton Capital already collate obscure data sets such as wheat prices going back nearly 1,000 years, in the hope of finding patterns that will inform the future value of commodities.
 Others are paying companies such as Planet Labs to monitor crops via satellite almost in real time, offering a hint of the yields to come.
 Spotting traffic jams outside Wal-Marts can help traders looking to bet on the success of Black Friday sales each year – and it’s easier to do this from space than sending analysts to car parks.
 Some data providers have been feeding transcripts of calls with company executives into a natural language processor – an area of artificial intelligence that the Turing test foresaw – to figure out if they have gained or lost confidence in their business.
 Trades might have had gut feelings about this before, but now they can get graphs.
 There is inevitably a lot of noise among these potential trading signals, which experts are trying to weed out.
 “Most of the breakthroughs in machine-learning aren’t in finance.
 The signal-to-noise ratio is a problem compared to something like recognising dogs in a photograph,” said Dr Anthony Ledford, chief scientist for the computer-driven hedge fund Man AHL.
 “There is no golden indicator of what’s going to happen tomorrow.
 What we’re doing is trying to harness a very small edge and doing it over a long period in a large number of markets.
” The statistics expert said the plunging cost of computer power and data storage, crossed with a “quite extraordinary” proliferation of recorded data, have helped breathe life into concepts like artificial intelligence for big investors.
 “The trading phase at the moment is making better use of the signals we already know about.
 But the next research stage is, can we use machine learning to identify new features?
” AHL’s systematic funds comb through 2bn price updates on their busiest days, up from 800m during last year’s peak.
 Developments in disciplines such as engineering and computer science have contributed to the field, according to the former academic based in Oxford, where Man Group this week jointly sponsored a new research professorship in machine learning at the university.
 driverless car The artificial intelligence used in driverless cars could have applications in finance Credit: Google Dr Ledford said the technology has applications in driverless cars, which must learn how to drive in novel conditions, and identifying stars from telescope images.
 Indeed, he has adapted the methods used in the Zooniverse project, which asked thousands of volunteers to help teach a computer to spot supernovae, to build a new way of spotting useful trends in the City’s daily avalanche of analyst research.
 “The core use is being able to extract patterns from data without specifically telling the algorithms what patterns we are looking for.
 Previously, you would define the shape of the model and apply it to the data,” he said.
 These technologies are not just been put to work in the financial markets.
 Several law firms are using natural language processing to carry out some of the drudgery, including poring over repetitive contracts.
 programme that is backed by Mike Lynch, former boss of the computing group Autonomy.
 Freshfields has spent a year teaching a customised system known as Kira to understand the nuances of contract terms that often occur in its business.
 Its lawyers have fed the computer documents they are reading, highlighting the parts they think are crucial.
 Kira can now parse a contract and find the relevant paragraphs between 40pc and 70pc faster than a human lawyer reviewing it by hand.
 “It kicks out strange things sometimes, irrelevancies that lawyers then need to clean up.
 We’re used to seeing perfect results, so we’ve had to teach people that you can’t just set the machine running and leave it alone,” said Isabel Parker, head of innovations at the firm.
 “I don’t think it will ever be a standalone product.
 It’s a tool to be used to enhance our productivity, rather than replace individuals.
” The system is built to learn any Latin script, and Freshfields’ lawyers are now teaching it to work on other languages.
 “I think our lawyers are becoming more and more used to it as they understand its possibilities,” she added.
 Insurers are also spending heavily on big data fed by new products such as telematics, which track a customer’s driving style in minute detail, to help give a fair price to each customer.
 “The main driver of this is the customer experience,” said Darren Price, group chief information officer at RSA.
 The insurer is keeping its technology work largely in-house, unlike rival Aviva, which has made much of its partnerships with start-up companies in its “digital garage”.
 Allianz recently acquired the robo-adviser Moneyfarm, and Axa’s venture fund has invested in a chat-robot named Gasolead.
 EY, the professional services firm, is also investing in analytics tools that can flag red flags for its clients in particular countries or businesses, enabling managers to react before an accounting problem spreads.
 Even the Financial Conduct Authority is getting in on the act.
 Having given its blessing to the insurance sector’s use of big data, it is also experimenting with a “sandbox”, or a digital safe space where their tech experts and outside start-ups can use real-life data to play with new ideas.
 The advances that catch on throughout the financial world could create a more efficient industry – and with that tends to come job cuts.
 The Bank of England warned a year ago that as many as 15m UK jobs were at risk from smart machines, with sales staff and accountants especially vulnerable.
 “Financial services are playing catch-up compared to some of the retail-focused businesses.
 They are having to do so rapidly, partly due to client demand but also because there are new challengers and disruptors in the industry,” said Amanda Foster, head of financial services at the recruiter Russell Reynolds Associates.
 But City firms, for all their cost pressures, are not ready to replace their fund managers with robots, she said.
 “There’s still the art of making an investment decision, but it’s about using analytics and data to inform those decisions.
” Follow Telegraph Business READ MORE ABOUT: If you would like to add a comment, please register or log in Register Log in Please review our commenting policy Follow Telegraph Business Galleries Gallery 21 Dec 2017, 12:15pm Gallery: Revealed: The 10 highest-paying jobs of 2017 Gallery 19 Dec 2017, 11:17am Gallery: Hamleys unveils its top 10 toys for Christmas 2017 candidates now?
 Gallery 17 Dec 2017, 10:53am Gallery: From Lee McQueen to Alana Spencer: where are the Apprentice candidates now?
 Business latest Serious Fraud Office didn't choose an autism expert when 2018, 7:00pm Libor families hit back at SFO with further claims of non-experts 06 Jan 2018, 6:42pm Former easyJet boss takes on ITV amid troubling Virgin dispute president of OPEC, speaks as Alexander Novak, Russia's energy minister, left, listens during a news conference following the 172nd Organization of Petroleum Exporting Countries (OPEC) meeting in Vienna, Austria, on Thursday, May 25, 2017 06 Jan 2018, 6:28pm Comment: As oil prices start to stir, the only way is up Liam Halligan Premium Liam Halligan 06 Jan 2018, 6:17pm Comment: Theresa May's government is failing miserably on rail, as on much else Jeremy Warner Premium Jeremy Warner 06 Jan 2018, 6:07pm How to manufacture a more balanced economy President and CEO Joe Kaeser announce their deal to merge their rail operations and create a "European champion", in September 27, 06 Jan 2018, 5:56pm Planets align for deal boom as nations set sights on overseas riches Premium 06 Jan 2018, 5:43pm Workers more pessimistic over pay growth than Bank of England forecasts 06 Jan 2018, 5:38pm Fitness First takeover leaves DW Sport with an £8m loss 06 Jan 2018, 5:29pm Garden Bridge designer Thomas Heatherwick nets £2.
6m payout despite falling profits 06 Jan 2018, 5:17pm Comment: Steinhoff's warning signs were there for all to see Ben Marlow Premium Ben Marlow accounting scandal 06 Jan 2018, 5:06pm Poundland owner Steinhoff's fairy tale proves too good to be true amid accounting scandal Premium 06 Jan 2018, 4:43pm Tesco takes the Christmas crown in festive retail wars 06 Jan 2018, 3:26pm Energy networks prepare to blend hydrogen into the gas grid for the first time chief Joe Kaeser announce a deal to merge their rail operations and create a "European champion", in September 27, 2017.
 06 Jan 2018, 3:25pm European champions to form in next deal boom, City predicts President Jean-Claude Juncker, right, prior to a meeting at EU headquarters in Brussels on Friday, Dec.
 8, 2017.
 06 Jan 2018, 3:02pm Comment: Brexit offers the chance to positively reshape our immigration policies Lady Barbara Judge Premium 06 Jan 2018, 2:35pm Bitcoin futures investors set to suffer losses as cryptocurrency fever fades Premium 06 Jan 2018, 1:46pm The businesses making a living from God's work (despite falling attendances) previously served as chief of staff at the Treasury to former Chancellor, George Osborne 06 Jan 2018, 1:30pm George Osborne's former adviser: 'There is no sense that we are due a recession soon' 06 Jan 2018, 1:26pm CES 2018: Time for the world's biggest tech show to get its mojo back Premium Comment: Why can YouTubers get away with posting controversial content?
 06 Jan 2018, 10:02am Comment: Logan Paul's disastrous YouTube video is just a symptom of very modern problem Chris Stokel-Walker Premium 06 Jan 2018, 9:13am What is ethereum and how does it differ from Bitcoin?
 Premium Please support us by disabling your adblocker We've noticed you're adblocking.
 We rely on advertising to help fund our award-winning journalism.
 We urge you to turn off your ad blocker for The Telegraph website so that you can continue to access our quality content in the future.
 Thank you for your support.
 Need help?
 Click here for instructions