How alternative data are shaping decision-making in finance

By Gaia Lamperti

Internal data remains the most popular source for businesses, but there are clear signs web scraping is set to overtake it in the near future, as interest in alternative data and their role to guide decision-making grows. Almost three-quarters (71%) of financial services organisations are already using web scraping techniques to gather those external data, while over two-thirds (68%) plan to increase their data budgets in the coming year.

These were some of the key findings from a new study by premium proxy and public web data acquisition solution provider Oxylabs. The whitepaper, ‘Alternative Data Unlocks Key Decisions in the UK & US Finance Industries’, demonstrates how web scraping is shaping the sector after surveying 500 UK-based and 501 US-based senior data decision-makers from financial services companies to gather insights on attitudes towards data collection.

The rise of alternative data

Alternative data can be used to better understand business performance, market trends and future investment opportunities. Financial services firms that can convert alternative real-time data into accurate and actionable insights are three times more likely to report significant improvements in decision-making, earlier research by the company found.

“Internal data is relatively cheap, efficient and company-unique, so it’s not a surprise that it’s the dominant acquisition method,” explained Gediminas Rickevičius, VP of Global Partnerships at Oxylabs. “Alternative data and web scraping being so close, however, is something completely new. There’s a clear interest and even necessity to uncover new ways of unearthing data-driven insights from untapped sources. It’s clear, financial services companies are expecting to get their edge through web-scraped alternative data.”

The alternative data market, which is estimated to reach a turnover of 1.7 billion by the end of 2020 and will double in value each year, has a high potential as it enables decision-makers to make more precise strategies, in real-time, to define data-driven investments.

Of all respondents, 80% declared they will focus more strongly on web scraping in the coming year, while 56% said that other data collection methods, such as third-party data aggregators, remain an important part of the data-gathering ecosystem, but are now a long way behind web scraping and internal data.

Despite web scraping’s rise, there remain some concerns among decision-makers about how to implement it in a safe, cost-effective, and compliant manner. According to Oxylabs’ report, almost four in ten (38%) state risk and legal complications as the main factors discouraging them from using web scraping. Alongside this, 36% are concerned about budget constraints or the need to introduce complicated new technology, and 34% believe they lack the technical know-how to make web scraping a success.

“Risk is inherent to all new technologies, as pioneering a new field means there are no footsteps to follow. Any web scraping risks, however, can be greatly mitigated by applying data collection and management best practices and employing legal and risk professionals to ensure web scraping is done in a compliant, safe and ethical way,” Vaidotas Šedys, Head of Risk Management at Oxylabs, added. “These practices may take some time to get used to, but done in the right way, there’s every chance that financial services businesses can gain valuable new insights, improve customer experiences and achieve strong ROI through web scraping.”