AI Disruption

AI Disruption

Share this post

AI Disruption
AI Disruption
Claude Operates Computers: Auto-Farming Upgrade Materials in Honkai: Star Rail

Claude Operates Computers: Auto-Farming Upgrade Materials in Honkai: Star Rail

Discover how Claude 3.5 Sonnet automates GUI tasks, plays games, and revolutionizes human-computer interaction with advanced visual capabilities.

Meng Li's avatar
Meng Li
Nov 20, 2024
∙ Paid
1

Share this post

AI Disruption
AI Disruption
Claude Operates Computers: Auto-Farming Upgrade Materials in Honkai: Star Rail
1
Share

Where Does the Execution Power of Large Models Come From?

Last month, Anthropic released the upgraded Claude 3.5 Sonnet. This latest version can mimic human-computer interactions by moving cursors, clicking specific locations, and typing through a virtual keyboard based on user instructions.

In simple terms, Claude 3.5 Sonnet can now use a computer like a human!

Enhanced with the Computer Use feature, Claude 3.5 Sonnet becomes the first AI model capable of controlling a computer as a graphical user interface (GUI) agent during public testing.

As an early beta version, Anthropic noted during its release that the model's abilities in real-world complex environments remain uncertain.

Recently, a research team from the Show Lab at the National University of Singapore (NUS) designed and conducted a series of carefully curated tasks to explore the depths of the Computer Use feature.

The results? Beyond the "AI can now work independently" showcased in the demo, it turns out:

AI can also play games independently! Don’t believe it? Check out the VCR where Claude takes over Honkai: Star Rail gameplay:

图片

Tasks such as finding dungeons and clicking teleport are now effortlessly handled by Claude’s "invisible hands."

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Meng Li
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share