In a scenario where Artificial Intelligence and the intensive use of data are essential for competitiveness, information security and privacy become critical success factors.

A large company in the financial sector sought act digital to evolve its Data Intelligence platform in Databricks, ensuring advanced governance, compliance with LGPD and banking standards, without losing performance and scalability.

The partnership resulted in a Data Privacy by Design journey over Databricks, with unified governance of tens of terabytes of data and drastic reduction of operational and regulatory risks.

The challenge: balancing compliance, performance and continuity

The company maintained a highly complex analytical environment, with:

• More than 50 TB of data stored and processed in Databricks
• Thousands of tables distributed across business domains
• Sensitive data (PII) dispersed across multiple layers and pipelines

The main challenges were:

1. Regulatory compliance
   Ensure adherence to the LGPD and specific financial sector standards, with robust audit trails and consistent access controls.
2. Mitigating the risk of sensitive data exposure
   Reducing the risk surface in the storage and transit of personal data, preventing leaks and misuse.
3. Fragmented governance
   Absence of a single model for cataloging, classifying, and controlling assets, making it difficult to see the end-to-end data journey.
4. Limited operational efficiency
   Investigations of security incidents and errors in pipelines took days, impacting response agility and data reliability.

The solution: Data Platform with Privacy by Design at Databricks

To address the challenges, act digital structured a three-step approach, anchored in the concept of Privacy by Design and making the most of the capabilities of the Databricks platform, in combination with the Azure and Power BI ecosystem.

Discovery: Mapping and Classification of Sensitive Data

The first step was to build a complete view of the data assets:

• Mapping of PII (Personally Identifiable Information) throughout the environment.
• Classification of data by risk and criticality, defining protection priorities.
• Using cataloging and data lineage tools (such as Azure Purview) to track the data lifecycle.

With this, the company now knows exactly where the sensitive data is, who uses it and for what purposes.

Security and privacy architecture

With the consolidated inventory, a data architecture was designed in Databricks oriented to protection throughout the lifecycle:

• Encryption at rest and in transit, aligned with banking best practices.
• Data Anonymization and Masking at different layers of the data architecture.
• Modeling and consumption patterns that isolate sensitive layers and reduce the risk of exposure.

This architecture followed Well-Architected Framework principles, ensuring high availability, performance, and observability.

Controls and Governance: Shift Left in Data Security

To make governance sustainable, act digital implemented the concept of Shift Left Governance, taking security controls to the beginning of the data development and operation cycle. Among the practices adopted:

• Granular access controls:
  • ABAC (AttributeBased Access Control)
  • RBAC (RoleBased Access Control)
  • RLS (RowLevel Security)
  • CLS (ColumnLevel Security)
• Standardization of reusable security artifacts in Databricks (UDFs and modules)
• Integration with observability and auditing tools to monitor access, exceptions, and incidents.

As a result, any new data pipeline or product is born with built-in security policies, reducing rework and inconsistencies.

Scalability and standardization with Databricks and Power BI

Designed to scale along with business growth, the solution included:

• Standardized pipelines in Databricks, making it easy to create new flows without compromising privacy controls.
• Ready-to-consume analytical artifacts in Power BI, ensuring that business teams have access to insights without undue exposure of sensitive data.
• Single governance around thousands of tables, simplifying the management and evolution of the platform.

Results achieved

The Data Privacy by Design journey at Databricks has yielded tangible results across four key dimensions.

Proactive compliance

• +50 TB of data protected with native security on the platform.
• Enhanced compliance with LGPD and banking standards in hundreds of critical business processes.
• Centralized view of assets, allowing faster and more assertive audits.

The client no longer acts only in a reactive way to regulatory requirements and has started to have a proactive compliance posture, anticipating risks and demands from regulatory bodies.

Operational efficiency

• Reduced time to investigate errors and security incidents from days to hours.
• Standardization of artifacts and pipelines that decreased maintenance effort and reliance on tacit knowledge.
• Greater reliability in the platform, freeing the data team to focus on value-added initiatives, such as AI cases and advanced analytics.

Risk mitigation

• Protection of sensitive data throughout its lifecycle – at rest and in transit.
• Significantly reduced likelihood of data leakage and improper access
• Better balance between the need for analytical use and preservation of the privacy of the holders.

Strengthened governance

• Unified management of thousands of tables under a WellArchitected architecture model.
• Updated and integrated data catalog, with risk classification and clear access rules.
• Ability to expand the platform without loss of control, supporting new data products and AI use cases.

Conclusion: Data Privacy by Design as an Enabler of Data & AI Innovation

The initiative led by act digital demonstrated that data security and privacy are not barriers to innovation, but rather the building blocks for a sustainable Data & AI strategy.

By restructuring its Data Intelligence platform in Databricks with a Privacy by Design approach, the large company in the financial sector:

• Gained regulatory and reputational trust
• Increased operational agility to address incidents and regulatory changes
• It has laid a solid foundation to expand Analytics and Artificial Intelligence cases responsibly.

The partnership reinforces act digital's positioning as an AIfirst technology multinational, capable of combining strategic consulting, data architecture and technology execution to transform complex environments into secure, scalable and business value-driven platforms.