Copilot Oversharing Tests Should Come Before Agent Rollout

Tools & Technical Tutorials

27 September 2026 | By Ashley Marshall

Most Copilot risk is not exotic. It is old permissions, broad connectors and missing evidence made visible by a much faster interface.

Copilot changes the blast radius of old permissions

The practical risk with Microsoft 365 Copilot and Copilot Studio agents is not that they invent a new class of confidential document. It is that they make old permission mistakes searchable, summarised and reusable at speed. A folder that too many people could technically open last year becomes a folder an assistant can now read, cite and blend into a board pack, customer response or sales proposal. Microsoft has been direct about this in its own guidance on oversharing, warning that Copilot and agents can summarise and surface data across contexts, so enforced access controls are critical. That should move the conversation away from "can we switch it on?" and towards "what evidence proves our permission model is ready?"

For UK leaders, this is a business operating issue as much as an IT issue. SharePoint sites, Teams channels, OneDrive links, legacy groups and third-party connectors all become part of the retrieval surface. If the organisation has treated access reviews as annual hygiene, Copilot will expose the gap between policy and reality. The first control is therefore not a prompt policy. It is an oversharing test pack that samples sensitive locations, checks inherited permissions, reviews external sharing, and records which agent or Copilot surface can reach which class of content.

What this means in practice is simple. Before a wider rollout, pick ten sensitive business processes and ask the system to find information a normal user should not need for that role. Include finance forecasts, HR case notes, commercial contracts, customer exports and merger planning folders. If the assistant can reach content because a legacy group or old link still grants access, the rollout is not ready. The fix is not to ban Copilot. The fix is to clean the permission model before agents make it operationally visible.

Agent connectors need their own evidence trail

The second risk is connectors. Microsoft explains in its Copilot extensibility documentation that when organisations integrate business workflows as agents, external data can stay within the app and does not flow into Microsoft Graph or train Microsoft 365 Copilot. That is useful, but it is not the end of the control question. The same documentation also explains that Copilot can generate a search query to send to the agent on the user's behalf, based on the prompt, conversation history and Microsoft 365 data the user can access. In plain English, a connector can become a bridge between internal context and an external business system.

That bridge needs evidence. A buyer should be able to see which connectors exist, who approved them, which identities they use, which data scopes they request, what logs they produce, and how they are revoked. Without that inventory, governance teams are left with a comforting architecture diagram and no operational control. The old SaaS question was "who has access to this app?" The agentic version is "which assistant can ask this app a question, using whose authority, with what surrounding context?" That is a sharper question and it needs a sharper answer.

A sensible starter pack is a connector register with five fields that procurement and security can understand: business owner, data owner, permission scope, logging location and failure mode. If the connector can write as well as read, add approval rules and rollback steps. If it touches customer data, add the lawful basis and data processing notes. If it can reach commercial data, add a sample query test showing that role boundaries hold. This does not slow the project down. It gives leaders a way to say yes to useful agents without pretending every connector is low risk.

The UK regulatory signal is accountability, not paperwork

The Information Commissioner's Office has been consistent on AI and data protection: accountability, transparency and explainability matter when personal data is involved. Its AI and data protection guidance points organisations back to established UK GDPR duties, while its 2026 work on automated decision-making and agentic AI keeps the focus on real accountability and redress. That matters for Copilot agents because they often sit in the messy middle between search, recommendation, workflow automation and decision support. A sales assistant that summarises account history is not the same as a hiring decision engine, but both may process personal data and both need a clear record of what happened.

The mistake is to treat this as a legal wrapper added after technical rollout. If an assistant can read HR documents, customer complaints, health information, payment history or disciplinary records, the evidence needs to exist before staff start using it. That evidence should include purpose, data categories, access rules, retention assumptions, human review points and a route for people to challenge or correct outputs. None of this requires a 90-page framework for every small use case. It does require enough structure to answer a regulator, a client or an employee if a poor answer causes harm.

The counterargument is that Microsoft 365 already has audit logs, sensitivity labels and Purview controls, so the organisation is covered. Those tools help, but they do not decide whether a particular agent should exist, which roles should use it, or whether the underlying SharePoint permissions are appropriate. The business still owns the decision to deploy. In practice, the minimum evidence pack should connect the technical controls to a named risk owner, a named data owner and a test result that shows the agent behaves inside the intended boundary.

NCSC guidance points towards small, controlled starts

The National Cyber Security Centre's recent work on agentic AI is valuable because it avoids the fantasy that autonomous systems are either magic or unusable. Its June 2026 blog urged organisations to start small, use agents for low-risk tasks first, and apply established cyber security controls from the outset. Its August 2026 guidance on managing the cyber risk of agentic AI went further, describing interim practical advice while formal guidance is developed. The useful takeaway for Copilot rollouts is that autonomy should be earned through evidence, not granted because a vendor feature is generally available.

That changes the rollout sequence. A common implementation path starts with licences, training, champions and a few high-visibility use cases. A better path starts with bounded pilots that deliberately test where the system should fail. Can the assistant retrieve board papers for someone outside the board group? Can it summarise a customer complaint without exposing unrelated personal data? Can a connector answer from a CRM object the user cannot access directly? Can the agent take an action without a clear approval event? These questions are uncomfortable, which is exactly why they belong before launch.

What this means in practice is a simple release gate. No Copilot agent or connector moves from pilot to production until it has a scope statement, permission test results, sample prompts, logging checks, rollback instructions and a named owner. That is not heavyweight bureaucracy. It is the same principle as change control for a finance workflow or firewall rule. If an assistant can see, summarise or act across business systems, it deserves the same operational discipline as any other system that can affect customers, employees or revenue.

Oversharing tests should be written like acceptance criteria

The fastest way to make this practical is to write oversharing tests as acceptance criteria, not as a vague security review. For each agent or Copilot surface, define the roles, content sources, allowed tasks, blocked tasks and expected evidence. A finance analyst may be allowed to summarise current month management accounts, but not HR grievance files. A service manager may be allowed to search support tickets, but not payroll exports. A sales manager may be allowed to ask for account risks, but not retrieve another team's commission spreadsheet. These examples sound obvious until a legacy group, old link or broad connector scope makes them false.

The test pack should include positive tests and negative tests. Positive tests prove the assistant can do the work that justifies the investment. Negative tests prove it cannot cross boundaries just because the prompt is clever. Include prompt injection attempts, role switching attempts, broad search requests, ambiguous personal data queries and requests that mix internal and external sources. Record screenshots or logs, the account used, the time of the test and the result. If the answer depends on configuration that may drift, record the control that will detect drift later.

This is where many businesses discover the real benefit of the exercise. The oversharing tests often reveal old collaboration habits that needed cleaning up anyway. A Copilot readiness project becomes a permission hygiene project, a data ownership project and a better information architecture project. The agent rollout still matters, but the evidence created around it becomes reusable. It can support supplier assurance, ISO work, client questionnaires, board reporting and future automation decisions.

The leadership decision is when to pause

The hardest part of Copilot governance is not choosing a control. It is deciding when the evidence is poor enough to pause. Leaders often worry that a pause will make the organisation look slow or anti-innovation. The opposite is usually true. A short pause after a failed oversharing test shows that the business understands the difference between productive adoption and uncontrolled exposure. Staff also notice the signal. If leadership waves through an assistant that can reach content it should not, people learn that speed beats judgement. If leadership fixes the boundary before rollout, people learn that AI adoption has standards.

The pause rule should be written before the pilot starts. For example, pause wider rollout if any test account can retrieve special category data outside its role, if an agent can write to a system without approval, if logs cannot identify which user triggered a query, if a connector owner cannot be named, or if external sharing settings are inconsistent across sensitive sites. These are not theoretical edge cases. They are practical signs that the organisation cannot yet explain what the assistant can see or do.

The common misconception is that this level of control is only for banks, law firms or public sector bodies. In reality, most UK SMEs have enough sensitive data to make oversharing painful: salaries, customer lists, disputes, quotes, acquisition plans, medical notes, safeguarding records, contract margins and board conversations. Copilot and agents can be genuinely useful, but they raise the standard of permission hygiene. The businesses that get value fastest will not be the ones that ignore the risk. They will be the ones that turn the risk into a clear release checklist.

Sources used: NCSC guidance on managing the cyber risk of agentic AI, NCSC advice on careful adoption of agentic AI, ICO guidance on AI and data protection, Microsoft Learn on Copilot extensibility, data, privacy and security, and Microsoft guidance on mitigating oversharing for Copilot and agents.

Frequently Asked Questions

What is a Copilot oversharing test?

It is a controlled check that asks Copilot or an agent to retrieve information from sensitive locations using realistic user roles. The aim is to prove both what the assistant can access and what it cannot access.

Is this only a Microsoft 365 issue?

No. Microsoft 365 is a common starting point because Copilot sits over SharePoint, Teams, OneDrive and Graph data, but the same pattern applies to any AI assistant with connectors into business systems.

Do sensitivity labels solve the problem?

They help, but they do not replace permission design, connector ownership or release testing. Labels are one control in the evidence pack, not the whole governance model.

Who should own Copilot agent approval?

Each agent should have a business owner, a data owner and a technical owner. Security or IT can run the control process, but the business must own the use case and the risk.

How often should oversharing tests be repeated?

Repeat them before launch, after major permission changes, after new connectors are added, and on a regular review cycle for sensitive processes. Quarterly is a sensible starting point for higher-risk agents.

What should stop a rollout?

Pause if an agent can reach data outside the intended role, write to systems without approval, lacks usable logs, has no named owner, or depends on a connector nobody can explain.

Can small businesses keep this lightweight?

Yes. A small business can start with a one-page agent register, a ten-prompt test pack and screenshots of results. The point is usable evidence, not a large governance document.

Where do regulations come into this?

If personal data is involved, UK GDPR accountability, transparency and access control duties still apply. Copilot does not remove the organisation's responsibility for deciding what staff and systems can access.