- Регистрация
- 17 Февраль 2018
- Сообщения
- 38 906
- Лучшие ответы
- 0
- Реакции
- 0
- Баллы
- 2 093
Offline
With Recall a bust, it's time to move on. Here's the next big selling point for Microsoft's AI laptops.
Image: Microsoft
“Click To Do” is Microsoft’s latest AI feature and selling point for Copilot+ PCs. Now that Windows Recall has taken a backseat after so much privacy criticism, Microsoft is turning the page—and this time it’s all about a special shortcut for accessing contextual AI actions.
To use Click To Do, you just hold down the Windows key on your keyboard and click once with the left mouse button. You can also press Windows key + Q if you’d rather use a keyboard shortcut. (If nothing happens, that means you aren’t on a Copilot+ PC.)
When Click To Do is activated, an outline will appear around your screen and Windows will highlight all text and images on your screen, making them selectable and allowing you to perform actions with them. Not only that, but Microsoft keeps adding more actions to this menu!
Click To Do started with Windows Recall
Funnily enough, Click To Do began its life as a feature built into Windows Recall. It let you take actions on text that appeared in the snapshots that Recall automatically took of your screen. However, after Microsoft pivoted from Recall, Click To Do became its own standalone feature.
As far as privacy goes, Click To Do feels like the anti-Recall: it doesn’t do anything in the background, and you must choose to use it.
Chris Hoffman / Foundry
When you activate Click To Do, it takes a screenshot and lets you interact with it. When you select actions like “Summarize Text,” all of it happens right on your PC with your PC’s neural processing unit.
But while most actions happen right on your PC, there are some actions—like “Search the web,” “Visual search with Bing,” and “Ask Copilot”—that will send data to Microsoft’s servers for processing. Fortunately, nothing leaves your PC unless you intentionally use such actions.
Click To Do lets you feed text to AI models
Click To Do uses optical character recognition (OCR) technology to scan your current screen and make text selectable. Basically, it’s taking a screenshot and letting you interact with elements within it.
For example, if you select an email address, the Click To Do menu presents “Send email” to compose an email. If you select a website URL, you can choose “Open website” to launch it in your default web browser. (Thankfully, this doesn’t always use Microsoft Edge!)
If you select over 10 words of text, things get more interesting. You’ll get a variety of actions that use the Phi Silica language model that runs on your Copilot+ PC’s NPU to perform actions like summarizing text, creating a bulleted list, or rewriting the text in different tones.
Chris Hoffman / Foundry
This is one of Microsoft’s first attempts at integrating NPU-powered text actions on a Copilot+ PC. Because it’s based on a screenshot, you can only send so much text to the language model at once.
That should improve the experience, because let’s be honest: those local language models that run on your Windows PC are nowhere near as powerful as a cloud-based large language models like ChatGPT. (If you’re using a chatbot app in your browser, you’ll wonder why you’re bothering to use a worse app that runs entirely on your PC. That’s a core problem with so many Copilot+ PC local AI features.)
Chris Hoffman / Foundry
There’s also an “Ask Copilot” action that will send your selected text to Microsoft’s Copilot AI chatbot, and a “Draft with Copilot in Word” action to start working on a Word document with Microsoft’s AI.
You can send information to a chatbot in the cloud here, too—but only if you’re using Microsoft’s Copilot (for home users) or Microsoft 365 Copilot (for businesses). Yeah, I know, it can be confusing.
Click To Do exposes AI image tools, too
Click To Do isn’t just for doing things with text—it also takes actions with images. This is where Microsoft’s other goals for Click To Do start to become clear. When you click on an image, you can select actions like “Blur background with Photos,” “Erase objects with Photos,” or “Remove background with Paint.”
Chris Hoffman / Foundry
These are all AI-powered image actions that would normally be scattered throughout Windows across different applications, but Microsoft is exposing them here through the Click To Do interface. And there’s also an “Ask Copilot” action here, so you can send an image to Microsoft’s Copilot AI and start a conversation about it there.
There’s still no shortcut for the impressive Super Resolution feature, though. That would be really useful!
You can disable Click To Do if you want
If you don’t want Click To Do for whatever reason—and let’s be honest, that “hold the Windows key and click” shortcut could easily get in the way when playing some PC games—you can turn it off.
Chris Hoffman / Foundry
To do so, head to Settings > Privacy & security > Click to Do. Flip the switch here to turn it off. (While Click To Do is activated by default, it’s only available on Copilot+ PCs, so you won’t see it on your average Windows 11 PC.)
What’s coming in the future?
Microsoft has been spending a lot of time adding feature after feature to Click To Do. For example, Click To Do will soon have an integrated Copilot prompt box where you can select text/images, type a prompt, and then click “Ask Copilot” to send the selected content to Copilot along with the prompt you typed.
You will also be able to select a mix of text and images, describe an image with an on-device large language model, and send text to the Microsoft Reaching Coach app. Click To Do will also soon be able to detect tables on your screen so you can send them right to Microsoft Excel.
This mix of features shows what Microsoft wants Click To Do to become: the “one click to access AI anywhere in Windows” action.
Microsoft hasn’t added file integration to Click To Do yet, but lots of actions for right-clicking files and sending them to Copilot and other AI tools are popping up in File Explorer’s context menu.
As for me? I’d rather copy and paste
After using so many half-baked Copilot+ PC features, it’s nice to see something with long-term potential. However, even though I use a Surface Laptop when I’m away from my powerful desktop PC, I must confess: I never actually use Click To Do, just like I never use Recall.
Click To Do is an interesting idea, but I still find myself copy-and-pasting text and images instead. I can copy-paste text and images into any app, with or without AI tools. And Windows 11’s built-in Snipping Tool is already great for extracting text from screenshots and capturing images for me to send to other apps or whatever else.
Maybe this will change in the future. Maybe not. But one thing’s for sure: Microsoft’s move away from Recall towards Click To Do is a smart one, and if it ends up paying off, it will pay off big.
Further reading: Should you buy a Copilot+ PC? What to know
Author: Chris Hoffman, Contributor, PCWorld
Chris Hoffman is the author of The Windows Readme, a newsletter that brings Windows PC tips, tricks, and experiments to more than 10,000 email inboxes each week. He's also the former editor-in-chief of How-To Geek and a veteran tech journalist whose work has appeared in The New York Times, PCMag, Reader's Digest, and other publications.
Recent stories by Chris Hoffman:
Image: Microsoft
“Click To Do” is Microsoft’s latest AI feature and selling point for Copilot+ PCs. Now that Windows Recall has taken a backseat after so much privacy criticism, Microsoft is turning the page—and this time it’s all about a special shortcut for accessing contextual AI actions.
To use Click To Do, you just hold down the Windows key on your keyboard and click once with the left mouse button. You can also press Windows key + Q if you’d rather use a keyboard shortcut. (If nothing happens, that means you aren’t on a Copilot+ PC.)
When Click To Do is activated, an outline will appear around your screen and Windows will highlight all text and images on your screen, making them selectable and allowing you to perform actions with them. Not only that, but Microsoft keeps adding more actions to this menu!
Click To Do started with Windows Recall
Funnily enough, Click To Do began its life as a feature built into Windows Recall. It let you take actions on text that appeared in the snapshots that Recall automatically took of your screen. However, after Microsoft pivoted from Recall, Click To Do became its own standalone feature.
As far as privacy goes, Click To Do feels like the anti-Recall: it doesn’t do anything in the background, and you must choose to use it.
Chris Hoffman / Foundry
When you activate Click To Do, it takes a screenshot and lets you interact with it. When you select actions like “Summarize Text,” all of it happens right on your PC with your PC’s neural processing unit.
But while most actions happen right on your PC, there are some actions—like “Search the web,” “Visual search with Bing,” and “Ask Copilot”—that will send data to Microsoft’s servers for processing. Fortunately, nothing leaves your PC unless you intentionally use such actions.
Click To Do lets you feed text to AI models
Click To Do uses optical character recognition (OCR) technology to scan your current screen and make text selectable. Basically, it’s taking a screenshot and letting you interact with elements within it.
For example, if you select an email address, the Click To Do menu presents “Send email” to compose an email. If you select a website URL, you can choose “Open website” to launch it in your default web browser. (Thankfully, this doesn’t always use Microsoft Edge!)
If you select over 10 words of text, things get more interesting. You’ll get a variety of actions that use the Phi Silica language model that runs on your Copilot+ PC’s NPU to perform actions like summarizing text, creating a bulleted list, or rewriting the text in different tones.
Chris Hoffman / Foundry
This is one of Microsoft’s first attempts at integrating NPU-powered text actions on a Copilot+ PC. Because it’s based on a screenshot, you can only send so much text to the language model at once.
That should improve the experience, because let’s be honest: those local language models that run on your Windows PC are nowhere near as powerful as a cloud-based large language models like ChatGPT. (If you’re using a chatbot app in your browser, you’ll wonder why you’re bothering to use a worse app that runs entirely on your PC. That’s a core problem with so many Copilot+ PC local AI features.)
Chris Hoffman / Foundry
There’s also an “Ask Copilot” action that will send your selected text to Microsoft’s Copilot AI chatbot, and a “Draft with Copilot in Word” action to start working on a Word document with Microsoft’s AI.
You can send information to a chatbot in the cloud here, too—but only if you’re using Microsoft’s Copilot (for home users) or Microsoft 365 Copilot (for businesses). Yeah, I know, it can be confusing.
Click To Do exposes AI image tools, too
Click To Do isn’t just for doing things with text—it also takes actions with images. This is where Microsoft’s other goals for Click To Do start to become clear. When you click on an image, you can select actions like “Blur background with Photos,” “Erase objects with Photos,” or “Remove background with Paint.”
Chris Hoffman / Foundry
These are all AI-powered image actions that would normally be scattered throughout Windows across different applications, but Microsoft is exposing them here through the Click To Do interface. And there’s also an “Ask Copilot” action here, so you can send an image to Microsoft’s Copilot AI and start a conversation about it there.
There’s still no shortcut for the impressive Super Resolution feature, though. That would be really useful!
You can disable Click To Do if you want
If you don’t want Click To Do for whatever reason—and let’s be honest, that “hold the Windows key and click” shortcut could easily get in the way when playing some PC games—you can turn it off.
Chris Hoffman / Foundry
To do so, head to Settings > Privacy & security > Click to Do. Flip the switch here to turn it off. (While Click To Do is activated by default, it’s only available on Copilot+ PCs, so you won’t see it on your average Windows 11 PC.)
What’s coming in the future?
Microsoft has been spending a lot of time adding feature after feature to Click To Do. For example, Click To Do will soon have an integrated Copilot prompt box where you can select text/images, type a prompt, and then click “Ask Copilot” to send the selected content to Copilot along with the prompt you typed.
You will also be able to select a mix of text and images, describe an image with an on-device large language model, and send text to the Microsoft Reaching Coach app. Click To Do will also soon be able to detect tables on your screen so you can send them right to Microsoft Excel.
This mix of features shows what Microsoft wants Click To Do to become: the “one click to access AI anywhere in Windows” action.
Microsoft hasn’t added file integration to Click To Do yet, but lots of actions for right-clicking files and sending them to Copilot and other AI tools are popping up in File Explorer’s context menu.
As for me? I’d rather copy and paste
After using so many half-baked Copilot+ PC features, it’s nice to see something with long-term potential. However, even though I use a Surface Laptop when I’m away from my powerful desktop PC, I must confess: I never actually use Click To Do, just like I never use Recall.
Click To Do is an interesting idea, but I still find myself copy-and-pasting text and images instead. I can copy-paste text and images into any app, with or without AI tools. And Windows 11’s built-in Snipping Tool is already great for extracting text from screenshots and capturing images for me to send to other apps or whatever else.
Maybe this will change in the future. Maybe not. But one thing’s for sure: Microsoft’s move away from Recall towards Click To Do is a smart one, and if it ends up paying off, it will pay off big.
Further reading: Should you buy a Copilot+ PC? What to know
Author: Chris Hoffman, Contributor, PCWorld
Chris Hoffman is the author of The Windows Readme, a newsletter that brings Windows PC tips, tricks, and experiments to more than 10,000 email inboxes each week. He's also the former editor-in-chief of How-To Geek and a veteran tech journalist whose work has appeared in The New York Times, PCMag, Reader's Digest, and other publications.
Recent stories by Chris Hoffman:
