Data Meshes in SMEs: A New View of Data

Revenues from data still often fail to materialize, even in SMEs. Data meshes could help to change that. “Data meshes are not a purely technological solution,” says Jan Krol of PROTOS Technologie. “Instead, they inject a new business view of data,” says Lars Francke of Stackable. How cloud native providers are draining data swamps.

Evaluating data, sharing it across the board, and implementing new digital business models – according to a recent study by the German Economic Institute (IW), in Germany this is still more science fiction than reality. “Companies tend to use data as a tool for internal optimization rather than as a source of revenue,” Barbara Engels and Jan Büchel of the Economic Research Institute state. While it is the case, for example, that every second company uses data to automate processes, “only 5 percent of all companies sell data.” The reason why revenues are missing out: “At present, it’s clear that many companies don’t know how they can integrate the use of data into their business model.” In order to have digital services become a far more regular core of the company’s own offer, new concepts such as data meshes can be a building block to the solution. This not only means that the company no longer collects information for its own sake, but also signifies that the company looks at the topic from the user’s perspective – and specifically focuses on how data products can be marketed and sold.

Data meshes: Profitable realization of data products in SMEs

Whether it’s sales figures, sensor values, wear or machine data – data meshes are the ideal way to store, process, and to commercially analyze any of these. “Data meshes are not a purely technological solution,” says Jan Krol, Cloud & Big Data Solutions Architect at PROTOS Technologie. “Instead, they inject a new business view of data,” says Lars Francke, CTO at Stackable, adding that this is “a view that clearly defines responsibilities.” In this way, data products should be able to be realized more successfully and profitably than ever before. To ensure that this does not remain as a theory for SMEs, cloud native service providers such as PROTOS Technologie and Stackable are using data meshes to drain their customers’ data swamps.

Data warehouses and data lakes: soak up, stockpile, and invest

“Data meshes solve many of the problems of centralized storage systems,” says Krol. “There is often little capital to be gained from data warehouses and data lakes,” adds Francke. The reason why: Information gets lost there too easily. For too long, data was regarded as a byproduct that companies simply gathered and stockpiled,” says Krol. The result: “Over the years, murky jungles have emerged that can hardly be managed, administered, or even used economically,” says Francke. If you still want to be successful, you will have to invest time, money, and effort. “Everything that is to be put into your own silo should be customized to the target format,” says Krol.

The success of the work is “relatively modest” according to Francke, who explains that “the sheer volume of bits and bytes makes many data warehouses intransparent.” Moreover, the organization calls their actual purpose into question. If, for example, accounting or marketing teams want to work with the data, central management can end up being a bottleneck in day-to-day business. Data lakes are no different to, for example, the collection of data being limited to specialist departments. Krol: “Complex data modeling for storage in the lake is usually not necessary.” But even here, digital value creation remains more of a pipedream. “If you have information accumulated and managed by one entity, you quickly come up against organizational limits,” says Francke, “regardless of whether it’s a large storage silo or a small lake.”

Data meshes: Decentralized, product-oriented, and with clear responsibilities

Things look different when it comes to data meshes. These are organized in a decentralized manner and focus on concrete data products for specific domains of use. In practical terms, this means that everything revolves around specific applications that, for example, the company’s specialist departments are responsible for helping to develop. “On the one hand, this ensures the practical relevance of the results,” says Krol. “On the other hand, everyone in a data mesh team understands what data must look like so that everything works in the end,” says Francke. In addition, today’s data products rely on standards and open source. “On that basis, the individual services can then be combined with each other,” says Krol. “This creates a modular and interoperable system where everything can be slotted together like building blocks in order to create new products,” adds Francke.

Connecting data via the cloud: Open, standardized and interoperable

Whether it’s via Amazon Web Services, Microsoft Azure or Google Cloud Platform – open source frameworks like Apache Spark form the bedrock for cluster computing. And open and free formats like Apache Parquet are a common tool for storing information. “Parquet data can be combined more flexibly than, say, relational tables,” says Krol. The only prerequisite: “Instead of mindlessly soaking up information as in the past, those responsible must think carefully about how to link it together logically and intelligently.” Harmonizing, blending, and smartly intermeshing information: “Data meshes also organize themselves via clouds and no longer via storage systems,” says Francke. The advantage: Those who manage the services only have to click once to make the data available, to assign rights, or to manage roles. The disadvantage: “Responsibility moves from central entities to the agile data mesh teams.” Companies must position themselves accordingly.

Cloud natives in SMEs: Resolving complexity, leveraging benefits

Clarifying licensing issues, regulating access rights, and providing metadata: “It’s only natural that everyone initially has a hard time with new responsibilities,” says Dr. Nils Kaufmann, under whose leadership approximately 20 cloud native providers such as PROTOS Technologie and Stackable have gathered in the EuroCloud Native (ECN) Cloud Native initiative since 2021. “Our members can also help SMEs with data meshes and the cloud,” Kaufmann says. And this is already possible for modest sums of money. If data warehouses still cost a fortune, the necessary resources are less expensive in the cloud. “Data meshes can even be worthwhile for small companies and small teams,” says Krol.

One thing is certain: “There is clearly still a lot of marketing behind this fairly new term,” says Francke. “Cloud natives initially need to make basic groundwork easier for their customers.” Nonetheless: “At a time when everything can be managed, mapped, and controlled in the cloud, it seems only logical to also interconnect your own information via the cloud,” says Kaufmann. “Ultimately, data meshes are a natural evolutionary step,” says Krol, “so as cloud infrastructures evolve, today’s data structures evolve with them.” Gaia-X is no different: what cloud native providers realize on a small scale for their clientele, the decentralized and connected data infrastructure is intended to accomplish on a large scale for Europe’s data economy. The IW study also finds that companies that use cloud technologies are more often “data economy ready” than others.

Nils Klute is Project and Communication Manager at EuroCloud Germany. He is responsible for content marketing activities on topics such as Gaia-X and AI, supports initiatives such as Service-Meister, EuroCloud Native, or systems integrators on their cloud journey. Prior to his start at eco in 2018, Nils worked as a corporate journalist for IT corporations (like SAP, T-Systems, and QSC at Cologne-based communication agency Palmer Hargreaves) and previously held public relations positions at market and economic research institutions.