The Art and Science of Abstraction: Enhancing Productivity While Mitigating Risks

In 2019, I posted a blog about the use of abstractions in IT within the financial services sector ("Abstraction in Financial IT - How far can and should we go?" - https://bankloch.blogspot.com/2020/02/abstraction-in-financial-it-how-far-can.html). This continues to be a topic that fascinates me.

In IT, we tend to build layer upon layer of abstraction, which allows us to become continuously more productive as complexity is hidden away for the user. This means users can focus more on the functionality they want to deliver. The evolution of Low-code and No-code platforms, alongside AI advancements, will likely push this to an extreme, where people with no IT skills could simply converse with a chatbot to describe the application they need, which is then automatically created under the hood.

However, as I mentioned in my previous article, this continuous abstraction also comes with issues, as software is rarely optimized for usage. While a simple tool programmed 20 years ago required a few kilobytes of storage and memory, the same tool today might take megabytes. This is because abstractions come with a lot of overhead. With energy consumption in IT growing exponentially year after year, especially due to the rise of AI, this might become problematic in the future. We could expect that optimization tools (also based on AI) might enormously optimize the actual run-time code, while maintaining the ease and flexibility of the abstractions at design time.

Joel Spolsky described this principle in 2002 with the "Law of Leaky Abstractions". The law states that all abstractions are leaky, meaning that every abstraction will eventually hit its limitations. When this happens, software engineers need to delve into the black box of the abstraction and optimize for the specific use case at hand.

As we hide more and more complexity, abstractions are becoming also more declarative. This means engineers describe the expected result (the WHAT), rather than the instructions to achieve that result (the HOW). Declarative abstractions are easier for engineers to interact with as they align more closely with human thought processes. On the other hand, they result in much less direct control, which can pose issues (e.g. "the right answer delivered too late can be the wrong answer.").

Although declarative languages offer many advantages (readability, separation of concerns, etc.), they only work well if the problem fits the abstraction. If not, declarative languages will likely break down. This is true for any abstraction. An abstraction is created with one or more use cases in mind. As soon as you deviate from these use cases, abstractions become uncertain, and you will likely face significant issues.

This is an important constraint to consider, as software resides in a complex, continuously changing environment. This means that the used abstraction might fit perfectly with the initial requirements but not future ones. Due to the cost associated with software engineering, extensive refactoring is usually avoided when new requirements arise. As a result, new requirements are often "forced" into the abstraction, which was not initially designed for them. This leads to unmaintainable and poor-quality software.

To manage abstractions effectively in a dynamic environment, it is important to:

Evaluate Abstractions Regularly: Upon every new requirement, check if the existing abstractions are still fit for purpose. If not, it might be necessary to open up the abstraction and adapt it to the new requirements.
Document Constraints and Assumptions: Clearly document the constraints and assumptions behind each abstraction. When new requirements are defined, they should be checked against these constraints. If they no longer fit, it’s probably time for a refactoring.
Focus on Architecture and Encapsulation: Ensure that your software architecture supports encapsulation, which can increase resilience. This allows to contain potential impacts of abstraction limitations and makes it easier to adapt to new requirements without significant overhauls.

By following these practices, you can balance the benefits of abstraction with the need for optimization and maintainability.

Comments

Transforming the insurance sector to an Open API Ecosystem

1. Introduction "Open" has recently become a new buzzword in the financial services industry, i.e. open data, open APIs, Open Banking, Open Insurance …, but what does this new buzzword really mean? "Open" refers to the capability of companies to expose their services to the outside world, so that external partners or even competitors can use these services to bring added value to their customers. This trend is made possible by the technological evolution of open APIs (Application Programming Interfaces), which are the digital ports making this communication possible. Together companies, interconnected through open APIs, form a true API ecosystem , offering best-of-breed customer experience, by combining the digital services offered by multiple companies. In the technology sector this evolution has been ongoing for multiple years (think about the travelling sector, allowing you to book any hotel online). An excelle...

Are product silos in a bank inevitable?

Silo thinking is often frowned upon in the industry. It is often a synonym for bureaucratic processes and politics and in almost every article describing the threats of new innovative Fintech players on the banking industry, the strong bank product silos are put forward as one of the main blockages why incumbent banks are not able to (quickly) react to the changing customer expectations. Customers want solutions to their problems and do not want to be bothered about the internal organisation of their bank. Most banks are however organized by product domain (daily banking, investments and lending) and by customer segmentation (retail banking, private banking, SMEs and corporates). This division is reflected both at business and IT side and almost automatically leads to the creation of silos. It is however difficult to reorganize a bank without creating new silos or introducing other types of issues and inefficiencies. An organization is never ideal and needs to take a numbe...

RPA - The miracle solution for incumbent banks to bridge the automation gap with neo-banks?

Hypes and marketing buzz words are strongly present in the IT landscape. Often these are existing concepts, which have evolved technologically and are then renamed to a new term, as if it were a brand new technology or concept. If you want to understand and assess these new trends, it is important to reduce the concepts to their essence and compare them with existing technologies , e.g. Integration (middleware) software ensures that 2 separate applications or components can be integrated in an easy way. Of course, there is a huge evolution in the protocols, volumes of exchanged data, scalability, performance…, but in essence the problem remains the same. Nonetheless, there have been multiple terms for integration software such as ETL, ESB, EAI, SOA, Service Mesh… Data storage software ensures that data is stored in such a way that data is not lost and that there is some kind guaranteed consistency, maximum availability and scalability, easy retrieval...

IoT - Revolution or Evolution in the Financial Services Industry

1. The IoT hype We have all heard about the "Internet of Things" (IoT) as this revolutionary new technology, which will radically change our lives. But is it really such a revolution and will it really have an impact on the Financial Services Industry? To refresh our memory, the Internet of Things (IoT) refers to any object , which is able to collect data and communicate and share this information (like condition, geolocation…) over the internet . This communication will often occur between 2 objects (i.e. not involving any human), which is often referred to as Machine-to-Machine (M2M) communication. Well known examples are home thermostats, home security systems, fitness and health monitors, wearables… This all seems futuristic, but smartphones, tablets and smartwatches can also be considered as IoT devices. More importantly, beside these futuristic visions of IoT, the smartphone will most likely continue to be the cent...

PSD3: The Next Phase in Europe’s Payment Services Regulation

With the successful rollout of PSD2, the European Union (EU) continues to advance innovation in the payments domain through the anticipated introduction of the Payment Services Directive 3 (PSD3) . On June 28, 2023, the European Commission published a draft proposal for PSD3 and the Payment Services Regulation (PSR) . The finalized versions of this directive and associated regulation are expected to be available by late 2024, although some predictions suggest a more likely timeline of Q2 or Q3 2025. Given that member states are typically granted an 18-month transition period, PSD3 is expected to come into effect sometime in 2026. Notably, the Commission has introduced a regulation (PSR) alongside the PSD3 directive, ensuring more harmonization across member states as regulations are immediately effective and do not require national implementation, unlike directives. PSD3 shares the same objectives as PSD2, i.e. increasing competition in the payments landscape and en...

Trade-offs Are Inevitable in Software Delivery - Remember the CAP Theorem

In the world of financial services, the integrity of data systems is fundamentally reliant on non-functional requirements (NFRs) such as reliability and security. Despite their importance, NFRs often receive secondary consideration during project scoping, typically being reduced to a generic checklist aimed more at compliance than at genuine functionality. Regrettably, these initial NFRs are seldom met after delivery, which does not usually prevent deployment to production due to the vague and unrealistic nature of the original specifications. This common scenario results in significant end-user frustration as the system does not perform as expected, often being less stable or slower than anticipated. This situation underscores the need for better education on how to articulate and define NFRs , i.e. demanding only what is truly necessary and feasible within the given budget. Early and transparent discussions can lead to system architecture being tailored more close...

Low- and No-code platforms - Will IT developers soon be out of a job?

“ The future of coding is no coding at all ” - Chris Wanstrath (CEO at GitHub). Mid May I posted a blog on RPA (Robotic Process Automation - https://bankloch.blogspot.com/2020/05/rpa-miracle-solution-for-incumbent.html ) on how this technology, promises the world to companies. A very similar story is found with low- and no-code platforms, which also promise that business people, with limited to no knowledge of IT, can create complex business applications. These platforms originate , just as RPA tools, from the growing demand for IT developments , while IT cannot keep up with the available capacity. As a result, an enormous gap between IT teams and business demands is created, which is often filled by shadow-IT departments, which extend the IT workforce and create business tools in Excel, Access, WordPress… Unfortunately these tools built in shadow-IT departments arrive very soon at their limits, as they don’t support the required non-functional requirements (like h...

An overview of 1-year blogging

Last week I published my 60th post on my blog called Bankloch (a reference to "Banking" and my family name). The past year, I have published a blog on a weekly basis, providing my humble personal vision on the topics of Fintech, IT software delivery and mobility. This blogging has mainly been a personal enrichment , as it forced me to dive deep into a number of different topics, not only in researching for content, but also in trying to identify trends, innovations and patterns into these topics. Furthermore it allowed me to have several very interesting conversations and discussions with passionate colleagues in the financial industry and to get more insights into the wonderful world of blogging and more general of digital marketing, exploring subjects and tools like: Search Engine Optimization (SEO) LinkedIn post optimization Google Search Console Google AdWorks Google Blogger Thinker360 Finextra … Clearly it is not easy to get the necessary ...

The UPI Phenomenon: From Zero to 10 Billion

If there is one Indian innovation that has grabbed global headlines , it is undoubtedly the instant payment system UPI (Unified Payments Interface) . In August 2023, monthly UPI transactions exceeded an astounding 10 billion, marking a remarkable milestone for India’s payments ecosystem. No wonder that UPI has not only revolutionized transactions in India but has also gained international recognition for its remarkable growth. Launched in 2016 by the National Payments Corporation of India (NPCI) in collaboration with 21 member banks, UPI quickly became popular among consumers and businesses. In just a few years, it achieved remarkable milestones : By August 2023, UPI recorded an unprecedented 10.58 billion transactions , with an impressive 50% year-on-year growth. This volume represented approximately 190 billion euros . In July 2023, the UPI network connected 473 different banks . UPI is projected to achieve a staggering 1 ...

AI in Financial Services - A buzzword that is here to stay!

In a few of my most recent blogs I tried to demystify some of the buzzwords (like blockchain, Low- and No-Code platforms, RPA…), which are commonly used in the financial services industry. These buzzwords often entail interesting innovations, but contrary to their promise, they are not silver bullets solving any problem. Another such buzzword is AI (or also referred to as Machine Learning, Deep Learning, Enforced Learning… - the difference between those terms put aside). Again this term is also seriously hyped, creating unrealistic expectations, but contrary to many other buzzwords, this is something I truly believe will have a much larger impact on the financial services industry than many other buzzwords. This opinion is backed by a study of McKinsey and PWC indicating that 72% of company leaders consider that AI will be the most competitive advantage of the future and that this technology will be the most disruptive force in the decades to come. Deep Lea...

Bankloch

Search This Blog