A federated data governance approach: benefits and best practices

As we approach the AI era, the surging demand for data becomes a pivotal point in tech evolution. Success lies in meticulous data governance and shaping a strategy to balance governance with access in the imminent age of innovation. Enter the Federated Model for Modern Data Governance, gaining traction for its shift from central control. This innovative approach involves a cross-section of professionals overseeing data governance, redefining the tech landscape with a transformative and collaborative strategy.

Decoding Federated Governance Approach

In the realm of data governance, a federated approach entails a collaboration between centralized IT and domain-level teams that produce data within their domain. A domain may encompass various facets, such as a line of business, business function, or geography. Domains are defined by Bounded Context, which essentially means that each domain operates with a distinct role and relies on unique terms and rules to accomplish its objectives as efficiently as possible. These rules, terms, and objectives define the boundaries of a domain. For example, Otis Elevator Company sells lifts in the UK domain and elevators in the US domain. This approach ensures a tailored and efficient management of data within diverse operational contexts.

Sales and support domains also have their objectives, rules, and terms. As shown below, the customer and product have different roles in the sales and support contexts. This distinction implies that a customer in the sales domain may have a slightly different meaning than a customer in the support context. Effectively managing this overlap is a pivotal function of federated governance.

In a federated governance approach, the IT and domain-level teams have specific roles. Central IT manages high-level governance policies applicable across domains, enforcing standards to ensure seamless interoperability. On the other hand, domain teams focus on policies specific to the data they collect and manage within their domain.

Essentially, IT creates flexible frameworks that empower domains to manage their data efficiently, aligning with their unique needs while staying within the broader governance framework. This approach is the cornerstone for a modern data mesh architecture, fostering enhanced agility and facilitating broader access to data across the organization.

Navigating the benefits of the Federated Data Governance Approach

Adopting a federated governance approach to data management offers several benefits, particularly in enabling more distributed architectures.

By implementing federated data governance, organizations can establish distributed systems that are not only more agile and innovative but also more efficient and secure.

In contrast to the typical data governance strategies, where IT is responsible for ensuring data security, privacy, accuracy, availability, and usability, federated governance decentralizes this control. The conventional one-size-fits-all approach, though simpler to manage the complexity across domains while ensuring data integrity, can be overly restrictive, hindering legitimate users from accessing valuable data.

The risk-averse nature of IT often results in stringent policies, limiting the accessibility and usability of different datasets. Overly restrictive data governance may inadvertently lead to security vulnerabilities as users find workarounds to policies that do not meet their requirements. In an era where access to reliable data is imperative for organizations, those imposing limitations may find themselves at a competitive disadvantage.

Recognizing that different business functions and geographies have unique policies and regulations, federated governance places those who collect and understand the data best in a much better position to manage the governance rules. For instance, GDPR compliance is relevant only in the European Union, and varying businesses have varying data quality thresholds.

Even within data sets, some columns may contain PII data, while others may not. Federated governance empowers domain teams to manage governance rules based on their intricate knowledge of data nuances and specific business needs. This enables them to apply fine-grained access controls, allowing selective access to sensitive columns within a data set.

For example, in the insurance domain, authorities can mask sensitive columns within a table to grant sales and marketing access without exposing confidential information, facilitating more accurate market segmentations and targeted sales strategies. In financial domains, specific rules governing sensitive financial data can be selectively applied, avoiding unnecessary restrictions on less sensitive information and preserving access to valuable data.

Moreover, domain teams, possessing a deep understanding of data collection processes, also know where their data may have weaknesses or quality issues and, more importantly, why data sets have specific problems. This insight allows them to control how certain professionals use data, limiting access to the specific columns causing the issues.

In essence, a federated governance approach empowers organizations to optimize data access, security, and usability by decentralizing control and aligning governance with the unique requirements of diverse domains.

Challenges in Implementing Federated Data Governance

While federated data governance promises various benefits, implementing it can be tricky, requiring a well-defined strategy.

The initial challenge lies in clearly defining the roles and responsibilities of each domain. To avoid domain overlap and mitigate conflicts and redundancies, central IT must play a pivotal role in determining which domains are responsible for what datasets.

Maintaining the right balance between limited central governance and the autonomy of domains is crucial for success. Central governance must hold the responsibility of enforcing standards that ensure interoperability across domains. If groups implement different technology stacks that are not interoperable, there is a risk of creating new data silos, negating the purpose of breaking down existing ones.

Ensuring corporate-wide data quality standards is paramount for establishing trust in shared data. These requirements must be defined and enforced by the central IT department. A central technology framework and governance rules need to be established and rigorously enforced.

Federated Governance Platform

A central data mesh platform can be beneficial in supporting a federated data governance strategy. This platform serves as the technological foundation, setting the standard for interoperability. Managing governance controls independently but using similar technology and controls reinforces a federated framework. The platform becomes the mechanism for defining domains, ensuring cohesion and efficiency in the overall governance structure.

Without a platform data product, producers are on their own to ensure that governance rules are incorporated into their data products and SQL queries. Also, domain managers who are responsible for the security of their data may not have control over how it is accessed. A centralized access platform designed for a federated governance strategy can put control in the hands of the people who are responsible for the company’s data.

Layered access controls that easily enable different access rules at various layers of a data asset can enable greater flexibility and control. Permissions at the data source layer can be different than rules at the row level. This flexibility enables nuanced controls but also creates complexity. A solution needs to be well-organized and easy-to-understand in order to support this complexity.

Permissions

Organizing access privileges effectively is the first step to a strong solution. The Avrio Identity and Access Management (IAM) module enables control at various levels, including by data source, schema, table, column and row. Access permissions or privileges can be defined at each individual level allowing domain managers to define who can access data sources, schemas, and data tables. As permissions get more granular, users are able to control rules for more parts of the data set. As shown in the table below, users are only able to access data sources but, if they have the right permissions, they can get permission to delete table data.

When defining access privileges at the column and row level, a little more work is required. We need to define which rows and columns we want to create policies for. In the case of columns, we need to know the business terms that define data in each column. Alternatively, data tags can be used to note which columns the rules will apply to.

To apply rules to rows, it gets even more complicated. Each row is an individual record, and you need to define parameters to indicate which rows the rules will apply to. Take, for example, a situation where you would need to define logic, such as masking rows that have a value of 20, 30, or 40. This can get complicated and custom SQL scripts may be required to implement these types of more complex permissions. Maintaining a library of rules that domain managers can access can streamline managing such granular rules.

Governance Rules

While federated governance strategies push more control and responsibility for governance to the edge of the organization, some governance rules need to be uniform across the organization and centrally managed. Examples include rules that ensure that dates are formatted uniformly across departments and Personal Identifiable Information (PII) data such as email are consistently masked in certain applications. These rules need to be more rigid and controlled within the IT function. The Avrio IAM module restricts governance rule creation authority to central IT managers.

With many governance rules in place to effectively manage data access, inevitably there will be instances where rules conflict. The ability to rank rules to determine which one is applied first provides the mechanism to manage conflicting rules. Determining these priorities is another function that is best left to the IT department.

Groups

Creating and managing individual permissions can quickly become very complex. Defining certain groups and assigning privileges to each one reduces the layers of complexity. If an individual requests access to a specific data set, instead of creating new rules and permissions, they can be included in a group that already has access.

Permissions and rules can also be created based on the roles of different users. For instance, data product producers need capabilities different from those of data product users.

Access and governance rules are at the heart of safely providing access to valuable data to the people who need it. This responsibility must be shared across the organization, and providing the tools and infrastructure to support this strategy is central to its success.

To learn more about Avrio’s federated governance capabilities, schedule a Demo here