Where Does the Execution Power of Large Models Come From?
Last month, Anthropic released the upgraded Claude 3.5 Sonnet. This latest version can mimic human-computer interactions by moving cursors, clicking specific locations, and typing through a virtual keyboard based on user instructions.
In simple terms, Claude 3.5 Sonnet can now use a computer like a human!
Enhanced with the Computer Use feature, Claude 3.5 Sonnet becomes the first AI model capable of controlling a computer as a graphical user interface (GUI) agent during public testing.
As an early beta version, Anthropic noted during its release that the model's abilities in real-world complex environments remain uncertain.
Recently, a research team from the Show Lab at the National University of Singapore (NUS) designed and conducted a series of carefully curated tasks to explore the depths of the Computer Use feature.
The results? Beyond the "AI can now work independently" showcased in the demo, it turns out:
AI can also play games independently! Don’t believe it? Check out the VCR where Claude takes over Honkai: Star Rail gameplay:
Tasks such as finding dungeons and clicking teleport are now effortlessly handled by Claude’s "invisible hands."
