Peer Insights: AI Productivity Tools and Data Privacy

This is the first in our new series of “Peer Insights” articles, in which industry professionals discuss today’s most important cybersecurity challenges and related topics. In this article, we take a look at the data privacy risks that arise with the use of AI productivity tools like Microsoft Copilot.

How AI Tools Like Copilot Create Data Privacy Risks

AI assistants provide significant value, but they also have serious implications for data privacy.  According to Shehar Yar, CEO of Software House, “The fundamental concern with Copilot and similar AI assistants is that they process and learn from the data they interact with, which creates a new attack surface that most organizations have not fully accounted for in their security postures.” 

Mary Rundall, Senior Director of Product Marketing at Concentric AI, which makes a data governance platform, concurs. She said, “Microsoft Copilot adheres to access rules and can only see what those rules allow. As a result, it will only show sensitive data to authorized users. The problem is that access is often broader than it should be.”

Ash Sobhe is CEO of R6S, a company that “builds private Second Brains for executives and business owners who refuse to send their sensitive data to the cloud.” In his view, “The core problem with tools like Microsoft Copilot is that they are designed to be helpful by accessing everything. Copilot integrates across your email, documents, calendar, Teams conversations, and SharePoint. The AI does not know the difference between a document you were supposed to see and one that was shared with the wrong distribution list three years ago. It surfaces everything with equal confidence.”

The Underlying Problem of Awareness of Sensitive Data

Data privacy problems typically arise from a lack of awareness about the presence of sensitive data. As Mary Rundell put it, “Without the knowledge of what sensitive data an organization holds, where it’s stored, or who can access, what seems like a small oversight can turn into a runaway snowball that wipes out their data security policies along the way.” She added, “To make things worse, much sensitive information is mislabeled or not labeled at all. When labels are incorrect or missing, the access rules that depend on them fail.”

“The biggest data privacy concern with AI tools like Microsoft Copilot is that they're only as safe as the permissions structure underneath them,” said Edith Forestal, Founder & Cybersecurity Specialist at Forestal Security. “The pattern I see is the same everywhere: companies rush to enable Copilot for productivity gains without first auditing their data access controls.” Shehar Yar elaborated on this point, saying, “If an organization’s permission structures are messy, and most organizations' are, Copilot essentially becomes a tool that can surface confidential documents, financial data, or HR records to people who technically have access but were never meant to find them.”

Mitigating Data Privacy Risk with AI Tools

What should companies do if they want to mitigate data privacy risk with AI tools like Copilot? Mary Rundell recommends using data security governance tools that are powered by context-aware AI. As she said, “Focus on data discovery and categorization. Forget rules, regex, and trainable classifiers because context-aware AI doesn’t rely on them. Instead, these tools scan structured and unstructured data across cloud and on-prem environments to accurately identify what sensitive data organizations have, where it is stored, and who holds the keys.”

She further broke down risk mitigation practices to include: 

• Classification and access policies: New data is generated constantly, making manual labeling processes impractical. Context-aware AI can automatically assign labels and permissions to new data based on semantically similar existing data. This leads to more accurate classification with much less effort. Security teams need to ensure their chosen solution can actually remediate issues from the platform. Otherwise, they might end up depending on a patchwork of tools.
• Continuous risk monitoring: A single snapshot is helpful, but it degrades quickly. Businesses need ongoing monitoring to catch issues such as data in the wrong place, mislabeled data, or over-permissioned content. This way, they can respond quickly. Context-aware AI can also identify unusual user activity related to data that might signal a breach or insider attack, such as privilege escalation followed by a surge in encrypted or shared data records.
• Copilot user activity: Once security teams have discovered, labeled, and secured their data, they need a way to verify that their data governance is actually working. The solution should provide visibility into which data records Copilot has shared, who accessed them, and when. That way, they can be confident it is revealing sensitive information only to the people who are supposed to see it.
  

Edith Forestal put it this way: “My advice to any organization adopting AI tools is to treat the rollout as a security project first and a productivity project second. Lock down permissions, classify your sensitive data, implement DLP policies, and train your people on what's appropriate to share with AI before you flip the switch.” She also advises security managers to pay attention to data residency and third-party processing. She said, “When employees paste client information, medical records, or financial data into AI prompts, that data is being processed outside the user's device.”

Data security policies can also be effective in mitigating AI data privacy risk. NextPhone, which offers AI receptionists and customer service agents, has dealt with this challenge. Yanis Mellata, the company’s CEO & Founder, explained that the company builds data privacy policy into its product. As he shared, “Our AI agents only get access to the minimum data they need for each interaction. A receptionist bot doesn't need your full client database. It needs the booking calendar and a few FAQs. We deliberately wall off sensitive information so the AI literally can't leak what it doesn't have.”

Shehar Yar gives his clients three pieces of practical advice: First, audit Microsoft 365 permissions thoroughly before enabling Copilot. Second, implement sensitivity labels and data loss prevention policies that restrict what AI tools can process. Third, establish clear acceptable use policies that define what employees can and cannot input into AI assistants. 

Conclusion

AI tools like Microsoft Copilot, while a boon to productivity, can easily create data privacy risk exposure. The reason for this problem is mostly due to excessive permission access policies, as well as a lack of awareness of sensitive data stored in the IT estate. It is possible to mitigate this risk, however. With the right tooling, security managers can discover sensitive data that should be kept away from AI software. They can audit Copilot access permissions and establish policies to minimize the chance of the AI tool accessing data that it shouldn’t see and surfacing it to other users.