5 Things You Didn’t Know Your PC Could Do (Copilot Voice & Vision)

Microsoft Copilot introduces advanced voice and vision AI capabilities to Windows PCs, allowing users to interact naturally and perform complex tasks by seeing and understanding screen content.

• Copilot Voice offers natural language commands for effortless PC interaction.

• Copilot Vision understands screen content, providing context-aware assistance.

• Upcoming features include deep integrations and automated PC tasks.

Modern Windows PCs, powered by Microsoft Copilot, offer five innovative features that enhance user interaction and productivity. These include natural voice commands, screen-aware vision capabilities for context-driven assistance, step-by-step guidance in applications, and seamless integration with personal accounts. Upcoming developments will further embed Copilot into the taskbar and introduce automated task execution, transforming the PC into a more intelligent and responsive personal assistant.

Copilot Voice Interaction

• 00:00:28 Copilot Voice enables natural language interaction with the PC, eliminating the need for typed commands. Users can activate it via a dedicated Copilot key, Windows + C, or by using the wake phrase 'Hey Copilot' after enabling it in settings. This feature feels intuitive, akin to asking a knowledgeable friend for information, such as recipe instructions, making the PC a more conversational and accessible tool.

Copilot Vision Capabilities

• 00:01:41 Copilot Vision allows the PC to 'see' and interpret content displayed on the screen, whether it's an entire desktop or a specific application. By clicking the 'glasses' icon in the Copilot app, users can share their screen and ask questions about displayed images, documents, or profiles. For example, it can identify landmarks in photos or offer professional advice on a LinkedIn profile, effectively acting as a contextual career coach.

Guided Task Execution

• 00:03:22 Copilot Vision, enhanced with a 'highlight' feature, provides step-by-step guidance within applications, such as Microsoft Excel. When asked how to perform a specific task, like conditional formatting, Copilot can not only provide instructions but also visually indicate where to click on the screen. This capability helps users navigate complex software functions with ease, ensuring correct execution of multi-step processes.

Copilot Connectors & Future

• 00:04:04 Copilot Connectors integrate the AI with other personal accounts like Outlook, Gmail, Google Drive, and Google Calendar, allowing Copilot to search across these services for information. This enables tasks such as finding calendar events or generating documents based on connected data. Future updates will bring Copilot directly to the taskbar, enable text-based vision queries for discreet use, and introduce 'Copilot actions' for automated tasks like photo reorientation or data extraction.