According to the manager, AI apps for Windows 11 experienced a real boom after Microsoft released Windows ML last fall. It is a platform that helps developers create offline AI applications without having to access cloud models. Windows ML connects applications, localized AI models, and hardware such as GPUs and neural processors.
Agents with an open ear
Windows ML is part of Microsoft’s “Foundry” product portfolio. This includes Foundry Local, which runs open source models on Windows devices. This product range also includes Windows AI APIs, which automatically summarize conversations, recognize language and upscale videos.
Microsoft is also using AI agents to change the way users interact with Windows 11: For example, they can describe a task in natural language. An agent running continuously in the background then takes over and does the work.
“Windows is evolving into a platform where natural language can trigger concrete system actions,” said Samantha Song, product manager for Windows at Microsoft. She demonstrated how users can simply say or type how they want to customize colors, background images, or menus. The agent then makes the appropriate change. “No manual configuration of designs, settings or lighting is required. The system views the process as a single, coherent action,” explained the product manager.
For this to work, developers would have to create a so-called “skills file” that defines the behavior of the agent. This function can then be reused as often as desired, says Song. “At an enterprise level, a scenario is conceivable in which a user switches to a secure financial mode and the system automatically adapts apps, access areas and the environment accordingly,” she adds.
More use cases for agents
At Build, Microsoft also demonstrated how OpenClaw can be used to create personalized agents to execute Windows functions.
