Anthropic, a company rapidly emerging as a competitor to OpenAI, has launched an exciting update to its Claude AI model that promises to transform how we interact with computers. With this release, Claude 3.5 Sonnet introduces a Next-gen improvement called “computer use,” allowing the AI to autonomously control your PC by mimicking human interaction with software interfaces.
How Claude 3.5 Sonnet Controls Your Computer
This new feature, now in public beta, lets Claude 3.5 Sonnet perform actions such as moving the cursor, clicking buttons, and typing text—just as a human would. Unlike traditional AI models that primarily operate within text-based environments, Claude's computer use allows it to visually perceive and interact with your screen. This opens up vast possibilities, from automating routine tasks to executing multi-step processes that would usually take a person significant time to complete.
For instance, companies like Canva and DoorDash are already experimenting with the technology, using Claude to handle tasks that can require hundreds of steps. Whether it’s managing complex design workflows or automating administrative processes, Claude is showing promising results in early testing.
Performance and Limitations
Despite its advancements, Anthropic acknowledges that the technology is still in its experimental stages. While Claude is currently leading in its category of AI models designed to control computers, it still has a long way to go before it can match human-level performance. For example, in the OSWorld benchmark, which measures an AI’s ability to use a computer like a human, Claude scores about 14.9%, compared to the human benchmark of 75%.
This may seem like a low figure, but it still outperforms competing models such as OpenAI’s GPT-4, which scored around 7.7% on the same test. As the technology improves, we can expect these numbers to increase, making the prospect of fully automated computer use more viable.
Potential Use Cases and Risks
The potential for this technology is immense. By allowing an AI to take control of basic computer tasks, businesses, and individuals could save hours on repetitive actions like filling out forms, organizing files, or even troubleshooting software issues. Some early adopters are already using it for coding, design, and customer service processes.
However, as with any powerful technology, there are risks involved. Anthropic has set strict limitations on what Claude can do. For instance, it is not allowed to make purchases or access sensitive information without additional security safeguards. These restrictions are part of Anthropic’s broader commitment to “AI safety,” ensuring that users retain control over their machines.
The Future of AI-Controlled Devices
The launch of Claude’s computer use feature marks a significant step forward in making AI assistants more integrated into our daily lives. As the technology matures, we can see Claude and similar models handling more advanced tasks—from managing personal schedules to running complex operations autonomously. Companies like Microsoft and Amazon are already exploring similar capabilities for their AI models, indicating that AI-powered PCs could become the norm in the coming years.
For now, Anthropic's Claude offers an interesting look into the future of human-computer teamwork, where digital helpers can not only answer questions but also perform important tasks for you directly.
BONUS: Get our $200 “AI Mastery Toolkit” FREE when you sign up!
