Skip to main content

Trade-offs Are Inevitable in Software Delivery - Remember the CAP Theorem



In the world of financial services, the integrity of data systems is fundamentally reliant on non-functional requirements (NFRs) such as reliability and security. Despite their importance, NFRs often receive secondary consideration during project scoping, typically being reduced to a generic checklist aimed more at compliance than at genuine functionality. Regrettably, these initial NFRs are seldom met after delivery, which does not usually prevent deployment to production due to the vague and unrealistic nature of the original specifications.

This common scenario results in significant end-user frustration as the system does not perform as expected, often being less stable or slower than anticipated. This situation underscores the need for better education on how to articulate and define NFRs, i.e. demanding only what is truly necessary and feasible within the given budget. Early and transparent discussions can lead to system architecture being tailored more closely to realistic non-functional requirements, potentially saving costs and focusing efforts where they are most needed.

Stringent NFR requirements are generally very complex and costly to implement, so they should only be enforced if truly necessary. Additionally, some NFRs are mutually exclusive and require a compromise, so stakeholders should be aware of the impact of certain choices. For example, a highly flexible system is naturally less optimized for a specific task and typically also less performant.

Comparisons between the NFR achievements of financial institutions and Big Tech companies like Google or Meta are often unfair and misleading. Big Tech’s ability to meet high standards in NFRs — such as high availability, perfect scalability, and high performance despite enormous volumes — is the result from several key differences:

  • Cost: With their vast scale, Big Tech companies can justify the extensive tuning of software by dozens of engineers, where even minor efficiencies translate into significant cost savings.

  • Technology Stack: Unlike banks that may rely heavily on standard one-size-fits-all technology, Big Tech companies utilize a complex stack of technologies. For example, where most banks predominantly use traditional SQL databases like Oracle for storing data, Big Tech firms deploy a diverse array of database technologies (SQL databases, time-series databases, key-value stores, column-based stores, document-based stores, graph-based stores) each tailored to specific use cases.

  • Legacy Constraints: Banks often deal with legacy systems and infrastructure not designed to meet the performance, scalability, and availability required for modern, continuous 24/7 usage. As a system is only as strong as its weakest link, NFRs should be adapted to these constraints. With the financial sector gradually replacing their legacy software by a modern, open-source-based stack and shifting also to the cloud, more ambitious NFR targets can be set.

  • Infrastructure and Platforms: Unlike Big Tech firms, which have dedicated engineering teams to build and maintain robust underlying technology platforms that abstract away non-functional complexity from development teams focusing on business features, banks often require their feature development teams to build systems from scratch, leading to inefficiencies as these teams are not specialized in NFR tuning.

  • Data Consistency: In banks, data consistency usually defaults to strong consistency to prevent issues like double spending or incorrect account balances. However, many scenarios might be adequately served by eventual consistency, which could simplify other NFRs. In Big Tech, many features operate with eventual consistency and in the majority of cases, end-users never notice this. This less stringent approach towards consistency allows Big Tech firms to deliver much better results on NFRs like performance and availability, which are often more visible to end-users.

Understanding trade-offs and compromises is crucial. As financial institutions increasingly adopt distributed systems to enhance their scalability and resilience, understanding the underlying principles that govern these systems becomes essential.

Distributed systems consist of numerous smaller, replicated servers, allowing the use of cheaper hardware and improving reliability by avoiding single points of failure. However, they also introduce complexities, such as more failure-prone individual servers (nodes) and the challenge of ensuring data consistency across servers.

This is where the CAP theorem comes into the picture, formulated by Eric Brewer, which theoretically proves that trade-offs are inevitable. This theorem states that a distributed data system can provide at most two out of three critical guarantees simultaneously:

  • Consistency: Every read (on any node) retrieves the most recent and correct information.

  • Availability: Every active node can always process requests and update others.

  • Partition Tolerance: The system continues to function even when parts of it lose communication.

Financial institutions, operating under a zero-tolerance policy for data loss, generally prioritize consistency ((i.e. ensure that all transactions are accurately recorded) but must understand that during network failures, a choice may be necessary between high availability and guaranteed consistency.

The CAP theorem is often misunderstood as necessitating a constant trade-off among the three guarantees. However, the necessity to choose arises primarily during network failures. At all other times, no trade-off is required. The choice is really between Guaranteed Consistency and High Availability only when a network partition or failure occurs.

  • High Availability (AP system): This type of system allows reads before all nodes are updated. This means the query is always processed, and the system will try to return the most recent available version of the information, even if it cannot guarantee it is up-to-date due to network partitioning.
    In such a system, eventual consistency can be achieved. This is managed through a background process that replicates all information. When a node is not available or the network connection is disrupted, the updates are buffered until the network is restored. Thus, consistency will eventually be reached, but during the time it takes to achieve consistency, the system may not provide the most recent information.

  • Guaranteed Consistency (CP system): In this system, all nodes are locked for reading—that is, the system will return an error or a timeout if it cannot guarantee that the information is up-to-date due to network partitioning—until all updates are processed.

The CAP Theorem is a prime example of the trade-offs and design choices system architects need to make. System design is, however, a constant balancing act between conflicting requirements. It is crucial to educate all employees about these limitations and trade-offs, so that choices can be made which align as closely as possible with the business requirements.

Comments

Popular posts from this blog

Transforming the insurance sector to an Open API Ecosystem

1. Introduction "Open" has recently become a new buzzword in the financial services industry, i.e.   open data, open APIs, Open Banking, Open Insurance …​, but what does this new buzzword really mean? "Open" refers to the capability of companies to expose their services to the outside world, so that   external partners or even competitors   can use these services to bring added value to their customers. This trend is made possible by the technological evolution of   open APIs (Application Programming Interfaces), which are the   digital ports making this communication possible. Together companies, interconnected through open APIs, form a true   API ecosystem , offering best-of-breed customer experience, by combining the digital services offered by multiple companies. In the   technology sector   this evolution has been ongoing for multiple years (think about the travelling sector, allowing you to book any hotel online). An excellent example of this

Are product silos in a bank inevitable?

Silo thinking   is often frowned upon in the industry. It is often a synonym for bureaucratic processes and politics and in almost every article describing the threats of new innovative Fintech players on the banking industry, the strong bank product silos are put forward as one of the main blockages why incumbent banks are not able to (quickly) react to the changing customer expectations. Customers want solutions to their problems   and do not want to be bothered about the internal organisation of their bank. Most banks are however organized by product domain (daily banking, investments and lending) and by customer segmentation (retail banking, private banking, SMEs and corporates). This division is reflected both at business and IT side and almost automatically leads to the creation of silos. It is however difficult to reorganize a bank without creating new silos or introducing other types of issues and inefficiencies. An organization is never ideal and needs to take a number of cons

RPA - The miracle solution for incumbent banks to bridge the automation gap with neo-banks?

Hypes and marketing buzz words are strongly present in the IT landscape. Often these are existing concepts, which have evolved technologically and are then renamed to a new term, as if it were a brand new technology or concept. If you want to understand and assess these new trends, it is important to   reduce the concepts to their essence and compare them with existing technologies , e.g. Integration (middleware) software   ensures that 2 separate applications or components can be integrated in an easy way. Of course, there is a huge evolution in the protocols, volumes of exchanged data, scalability, performance…​, but in essence the problem remains the same. Nonetheless, there have been multiple terms for integration software such as ETL, ESB, EAI, SOA, Service Mesh…​ Data storage software   ensures that data is stored in such a way that data is not lost and that there is some kind guaranteed consistency, maximum availability and scalability, easy retrieval and searching

IoT - Revolution or Evolution in the Financial Services Industry

1. The IoT hype We have all heard about the   "Internet of Things" (IoT)   as this revolutionary new technology, which will radically change our lives. But is it really such a revolution and will it really have an impact on the Financial Services Industry? To refresh our memory, the Internet of Things (IoT) refers to any   object , which is able to   collect data and communicate and share this information (like condition, geolocation…​)   over the internet . This communication will often occur between 2 objects (i.e. not involving any human), which is often referred to as Machine-to-Machine (M2M) communication. Well known examples are home thermostats, home security systems, fitness and health monitors, wearables…​ This all seems futuristic, but   smartphones, tablets and smartwatches   can also be considered as IoT devices. More importantly, beside these futuristic visions of IoT, the smartphone will most likely continue to be the center of the connected devi

PSD3: The Next Phase in Europe’s Payment Services Regulation

With the successful rollout of PSD2, the European Union (EU) continues to advance innovation in the payments domain through the anticipated introduction of the   Payment Services Directive 3 (PSD3) . On June 28, 2023, the European Commission published a draft proposal for PSD3 and the   Payment Services Regulation (PSR) . The finalized versions of this directive and associated regulation are expected to be available by late 2024, although some predictions suggest a more likely timeline of Q2 or Q3 2025. Given that member states are typically granted an 18-month transition period, PSD3 is expected to come into effect sometime in 2026. Notably, the Commission has introduced a regulation (PSR) alongside the PSD3 directive, ensuring more harmonization across member states as regulations are immediately effective and do not require national implementation, unlike directives. PSD3 shares the same objectives as PSD2, i.e.   increasing competition in the payments landscape and enhancing consum

Low- and No-code platforms - Will IT developers soon be out of a job?

“ The future of coding is no coding at all ” - Chris Wanstrath (CEO at GitHub). Mid May I posted a blog on RPA (Robotic Process Automation -   https://bankloch.blogspot.com/2020/05/rpa-miracle-solution-for-incumbent.html ) on how this technology, promises the world to companies. A very similar story is found with low- and no-code platforms, which also promise that business people, with limited to no knowledge of IT, can create complex business applications. These   platforms originate , just as RPA tools,   from the growing demand for IT developments , while IT cannot keep up with the available capacity. As a result, an enormous gap between IT teams and business demands is created, which is often filled by shadow-IT departments, which extend the IT workforce and create business tools in Excel, Access, WordPress…​ Unfortunately these tools built in shadow-IT departments arrive very soon at their limits, as they don’t support the required non-functional requirements (like high availabili

An overview of 1-year blogging

Last week I published my   60th post   on my blog called   Bankloch   (a reference to "Banking" and my family name). The past year, I have published a blog on a weekly basis, providing my humble personal vision on the topics of Fintech, IT software delivery and mobility. This blogging has mainly been a   personal enrichment , as it forced me to dive deep into a number of different topics, not only in researching for content, but also in trying to identify trends, innovations and patterns into these topics. Furthermore it allowed me to have several very interesting conversations and discussions with passionate colleagues in the financial industry and to get more insights into the wonderful world of blogging and more general of digital marketing, exploring subjects and tools like: Search Engine Optimization (SEO) LinkedIn post optimization Google Search Console Google AdWorks Google Blogger Thinker360 Finextra …​ Clearly it is   not easy to get the necessary attention . With th

Deals as a competitive differentiator in the financial sector

In my blog " Customer acquisition cost: probably the most valuable metric for Fintechs " ( https://bankloch.blogspot.com/2020/06/customer-acquisition-cost-probably-most.html ) I described how a customer acquisition strategy can make or break a Fintech. In the traditional Retail sector, focused on selling different types of products for personal usage to end-customers,   customer acquisition  is just as important. No wonder that the advertisement sector is a multi-billion dollar industry. However in recent years due to the digitalization and consequently the rise of   Digital Marketing , customer acquisition has become much more focused on   delivering the right message via the right channel to the right person on the right time . Big tech players like Google and Facebook are specialized in this kind of targeted marketing, which is a key factor for their success and multi-billion valuations. Their exponential growth in marketing revenues seems however coming to a halt, as digi

AI in Financial Services - A buzzword that is here to stay!

In a few of my most recent blogs I tried to   demystify some of the buzzwords   (like blockchain, Low- and No-Code platforms, RPA…​), which are commonly used in the financial services industry. These buzzwords often entail interesting innovations, but contrary to their promise, they are not silver bullets solving any problem. Another such buzzword is   AI   (or also referred to as Machine Learning, Deep Learning, Enforced Learning…​ - the difference between those terms put aside). Again this term is also seriously hyped, creating unrealistic expectations, but contrary to many other buzzwords, this is something I truly believe will have a much larger impact on the financial services industry than many other buzzwords. This opinion is backed by a study of McKinsey and PWC indicating that 72% of company leaders consider that AI will be the most competitive advantage of the future and that this technology will be the most disruptive force in the decades to come. Deep Learning (= DL) is a s