Microsoft is pushing the boundaries of AI-powered automation with its latest enhancement to Copilot Studio. The new "computer use" capability, currently in early access research preview as of April 2025, allows Copilot Studio agents to interact directly with websites and desktop applications, mirroring human interaction by clicking buttons, selecting menus, and entering text. This represents a significant leap forward in Microsoft's vision for intelligent automation, effectively enabling Copilot Studio agents to treat graphical user interfaces (GUIs) as tools.
The key benefit of "computer use" lies in its ability to automate tasks even when direct API connections are unavailable. As Charles Lamanna, Corporate Vice President for Business and Industry at Microsoft Copilot, stated, "If a person can use the app, the agent can too." This opens up a vast range of automation possibilities, particularly for processes involving legacy systems or external websites lacking robust API support.
Several practical use cases highlight the potential of this technology:
- Automated Data Entry: Agents can automate data entry across systems that lack direct integrations, eliminating manual effort and minimizing errors.
- Market Research: Marketing teams can automate the collection of market data from various online sources for analysis, providing valuable insights without manual intervention.
- Invoice Processing: Finance departments can streamline invoice processing by automating data extraction and input into accounting systems, reducing manual errors and improving efficiency.
Beyond simply mimicking human actions, "computer use" incorporates advanced AI capabilities to ensure resilience and adaptability. The agents can automatically adjust to changes in application and website interfaces, using built-in reasoning to fix issues in real-time. This self-healing capability minimizes disruptions and reduces the need for constant manual adjustments, a common pain point in traditional Robotic Process Automation (RPA).
Microsoft emphasizes the security and governance aspects of "computer use." The feature is built upon Copilot Studio's existing robust security measures and governance frameworks, ensuring compliance with organizational and industry standards. Furthermore, "computer use" runs on Microsoft-hosted infrastructure, eliminating the need for organizations to manage their own servers and keeping enterprise data within Microsoft Cloud boundaries. Microsoft also states that enterprise data is not used to train the Frontier model.
The introduction of "computer use" positions Copilot Studio as a modern alternative to traditional RPA solutions. Unlike traditional RPA, which often requires extensive coding and scripting, Copilot Studio allows users to describe automation tasks using natural language. The platform even provides side-by-side video previews, showing how the agent interprets and executes UI actions, enhancing transparency and control. Moreover, full audit trails, including screenshots and reasoning steps, are available for every action performed by the agent.
The "computer use" feature is currently available as an early access research preview. Microsoft plans to share more details and demonstrations at Microsoft Build in May 2025. The company is actively seeking feedback from users interested in exploring the new capability and encourages them to participate in the preview program. With its ease of use, adaptability, and robust security features, "computer use" in Copilot Studio promises to revolutionize enterprise automation, empowering organizations to streamline processes, enhance productivity, and drive innovation.
