Uncategorized

More than two years ago, I published an article on How to build a scalable streaming data pipeline for financial big data. It described a first iteration; a pragmatic foundation that allowed us to ingest, normalize, and persist market data at scale. It worked, but it was just that – a first iteration.

Trading the financial markets systematically with a purely data-driven approach requires a stable and scalable execution environment. This post discusses how the challenge of building such an execution environment can be tackled.

Running a systematic trading business in the cloud poses different kinds of risk, from operational risks to technical risks. Taking care of technical errors in a timely manner can be crucial. This post discusses the details of designing and building a scalable iOS notification app to monitor business operations in real-time.

High-quality historical financial data offers a direct competitive advantage in the systematic trading business, but its big data qualities make it both costly to manage and store as well as very expensive to buy from external sources. To mitigate a part of these expenses, a custom, self-managed data lake can be built by recording and storing real-time financial data through a scalable streaming data pipeline.