You’re at a fund that doesn’t want to burn headcount hiring their own data, storage, and sysadmin team, and you’re not alone. Financial institutions of all sizes realize that it is expensive to meet the staffing and infrastructure levels required to efficiently manage data internally. Not an effective formula for growth.
Baron Davis, CEO at financial technology firm Code Willing, suggests that when dealing with the requirements of quants: “There are several challenges that, on the surface, you might think are trivial, but in reality are very difficult.”
Firstly, there’s the issue of the quality of the data itself. “Data errors can be catastrophic to systematic trading,” Davis says. “A missed reverse split on a stock could have you think you are trading a small notional value of something, when in fact you are trading a very large notional value of something. This has very serious risk and regulatory implications.”
The next factor to consider is the housekeeping issues related to cross-referencing: “Matching assets between one vendor’s data and another’s is actually quite difficult. They will have different identifiers, which will change over time and not always at the same time and date,” Davis says.
Next up, the sheer scale of datasets and the unstructured character of many of those sets: “What used to be single price/volume data for each asset can now be streaming order-by-order data,” Davis says. “This adds up to relatively large datasets that can be challenging to store efficiently. Furthermore, there are now news feeds, social media data, and geospatial data. Each has their own challenges in how to ingest, process, and store efficiently for the quantitative researcher.”
To log in or register visit here
For more information visit Code Willing
