There’s so much noise in outbound sales today: generic cold emails, calls, and a ton of spam.
As a go-to-market team, your biggest challenge is figuring out how to target and personalize your outreach as well as possible.
The trick lies in how great you can make your data enrichment—i.e., your research on companies and people. To really find the right companies to target, you need to ask questions that can’t just be answered by a canned data enrichment tool.
That means asking questions like:
Is this company hiring?
Who are this company’s competitors?
Is this company PLG or sales-led?
Does this company have SOCII compliance?
…and more
Traditionally, GTM functions have attempted to do this kind of research in two ways.
Option A: Hire a gazillion SDRs to do manual research. If you do this, it’s impossible to make the numbers work. You’ll never find enough qualified leads and to convert them at the rate you want.
Option B: Give up and skip the deep research. Just spray and pray method, sending blanket, impersonal messaging into the void. If you do this—especially during a time of tightening email provider rules—it’s just a matter of time until you’re blocked by spam.
Luckily, there’s now an Option C—and that is, use an AI web scraper to do your manual research for you. With Claygent, Clay’s AI web scraping agent, you can start with a list of domains, ask any question, and watch as the answers filter in: no manual work needed.
This post dives into web scraping for GTM workflows, showing you how you can use it to find relevant information about companies and employees that might be a good fit for your product.
Web scraping is…probably exactly what you think it is
First off, let’s briefly define web scraping. It’s probably exactly what you think it is. I like Upwork’s simple and clear definition, which says web scraping is, “automating the collection of structured data sets from the internet.” And although the term itself may sound a bit covert, it’s just the ability to capture publicly available information, at the HTML code level on a website.
Any number of industries—including those of us specializing in lead generation—can benefit from access to real-time research data from the internet. For GTM teams, this helps you better identify your ICP at the right target accounts.
No-code technology makes it possible for anyone to use web scraping
Before Clay, web scraping wasn’t readily accessible to most people. Unless you were really technical and could build a customized web scraper, or you had a big budget and could pay a full stack developer to do it for you, you just couldn’t access the power of web scraping.
New tools like Apify (which Clay integrates with!) introduced some out-of-the-box web scrapers. Preconfigured tools, however, can miss key use cases and don’t have the flexibility to provide answers to questions that GTM professionals need to solve to reach the right people.
When Clay introduced the AI web scraper, Claygent, in 2023, it opened the world of web scraping up to non-technical users. Claygent is among the most powerful web scrapers out there, finding answers to simple yes and no questions and more complex questions within seconds, and you don’t need to be able to write a single line of code to use it.
Let’s take a look at how you can use the Claygent to scrape the web for real-time information that can help you target and write personalized messaging at unprecedented scale.
How the Claygent helps you find key information about your prospects
Start with a list of domains of your target accounts in Clay.
💡Pro Tip💡: If you don’t already know their web addresses, you can use the Domain Enrichment Tool in Clay to identify their website domain. Then, you’re ready to get started!
When you’re ready to try Claygent, click on Add Enrichment.
Then, begin your query in one of two ways.
The first is that you can use Claygent (AI Web Scraper) Templates to get answers to frequently asked questions within seconds.
These preloaded templates allow you to query for a variety of information types that will better inform who you should be reaching out to and why. They include categories like email classification type, a company’s competitors, and an employee’s time spent at a company, for example.
The other way to use the Claygent is to write your prompt as depicted in the image. You can use plainspoken, natural language to ask your question—and AI will take care of the rest.
The web scraper allows you to ask all types of questions. Let’s look at a few questions that may naturally come up as you’re determining who you should be reaching out to and how you can use Claygent to get good results. You can also watch our lesson in Clay University to learn more about how to write effective queries.
For instance, if you want to know if a company is hiring, here’s how you would interact with Claygent:
You would ask Claygent: Is [company name] hiring?
You would set the prompt so that you receive responses that are either “true” or “false.”
When you do receive responses, you can view “reasoning” for those answers. This will show you why Claygent returned the results that it did. You can even use the Claygent to route to the sources it used if you want to double check them or dive deeper.
You can also ask the Claygent to web scrape more complex questions that do not have a yes or no answer. For example, let’s say you want to determine if a given company has a pricing page on their site. Here’s how you can find out.
You would ask the Claygent (AI Web Scraper) three questions:
- Does [company name] have a pricing page on their site?
- If so, respond with their URL. If you cannot find a URL, respond “none.”
- Only respond with a pricing page from their website. This is their website: [website].
As you can see, the results will either return a URL or “none” to indicate that the website does not contain a pricing page.
Here are some more example prompts to get you started:
Is [company name] B2B or B2C?
Who are [company name]’s main competitors? List up to 5 company names.
What is the [company name]’s podcast? Provide the name and web address for the company’s podcast.
What is the [company name]’s YouTube Channel? Provide the handle for any YouTube Channel belonging to the company.
Has [company name] ever acquired another company? Answer in true or false.
Does [company name]] allow remote work? Answer in true or false.
Name one or two happy customers featured on [company name]’s website. (This will scrape the company’s domain to find any enterprise deals, news about successful partnerships, or information from a page that displays a list of customers.)
Does [company name] have a free trial? You can then clarify the prompt by saying, “if the website has language on it like “get started” or “sign up” or “free trial”, the answer is yes.
Scrape anything with Claygent (AI Web Scraper)
As you can probably tell, the possible use cases for Claygent are multitudinous. With just one or two lines of natural language, you can easily automate research that once was either impossible or impossible to justify because it would take too long.
Luckily, the future is here and it’s rooted in personalized automation at scale. You can sign up for Clay here and get started using the Claygent immediately! You can also join our Slack Community and post questions, ideas, and your own innovative solutions!
There’s so much noise in outbound sales today: generic cold emails, calls, and a ton of spam.
As a go-to-market team, your biggest challenge is figuring out how to target and personalize your outreach as well as possible.
The trick lies in how great you can make your data enrichment—i.e., your research on companies and people. To really find the right companies to target, you need to ask questions that can’t just be answered by a canned data enrichment tool.
That means asking questions like:
Is this company hiring?
Who are this company’s competitors?
Is this company PLG or sales-led?
Does this company have SOCII compliance?
…and more
Traditionally, GTM functions have attempted to do this kind of research in two ways.
Option A: Hire a gazillion SDRs to do manual research. If you do this, it’s impossible to make the numbers work. You’ll never find enough qualified leads and to convert them at the rate you want.
Option B: Give up and skip the deep research. Just spray and pray method, sending blanket, impersonal messaging into the void. If you do this—especially during a time of tightening email provider rules—it’s just a matter of time until you’re blocked by spam.
Luckily, there’s now an Option C—and that is, use an AI web scraper to do your manual research for you. With Claygent, Clay’s AI web scraping agent, you can start with a list of domains, ask any question, and watch as the answers filter in: no manual work needed.
This post dives into web scraping for GTM workflows, showing you how you can use it to find relevant information about companies and employees that might be a good fit for your product.
Web scraping is…probably exactly what you think it is
First off, let’s briefly define web scraping. It’s probably exactly what you think it is. I like Upwork’s simple and clear definition, which says web scraping is, “automating the collection of structured data sets from the internet.” And although the term itself may sound a bit covert, it’s just the ability to capture publicly available information, at the HTML code level on a website.
Any number of industries—including those of us specializing in lead generation—can benefit from access to real-time research data from the internet. For GTM teams, this helps you better identify your ICP at the right target accounts.
No-code technology makes it possible for anyone to use web scraping
Before Clay, web scraping wasn’t readily accessible to most people. Unless you were really technical and could build a customized web scraper, or you had a big budget and could pay a full stack developer to do it for you, you just couldn’t access the power of web scraping.
New tools like Apify (which Clay integrates with!) introduced some out-of-the-box web scrapers. Preconfigured tools, however, can miss key use cases and don’t have the flexibility to provide answers to questions that GTM professionals need to solve to reach the right people.
When Clay introduced the AI web scraper, Claygent, in 2023, it opened the world of web scraping up to non-technical users. Claygent is among the most powerful web scrapers out there, finding answers to simple yes and no questions and more complex questions within seconds, and you don’t need to be able to write a single line of code to use it.
Let’s take a look at how you can use the Claygent to scrape the web for real-time information that can help you target and write personalized messaging at unprecedented scale.
How the Claygent helps you find key information about your prospects
Start with a list of domains of your target accounts in Clay.
💡Pro Tip💡: If you don’t already know their web addresses, you can use the Domain Enrichment Tool in Clay to identify their website domain. Then, you’re ready to get started!
When you’re ready to try Claygent, click on Add Enrichment.
Then, begin your query in one of two ways.
The first is that you can use Claygent (AI Web Scraper) Templates to get answers to frequently asked questions within seconds.
These preloaded templates allow you to query for a variety of information types that will better inform who you should be reaching out to and why. They include categories like email classification type, a company’s competitors, and an employee’s time spent at a company, for example.
The other way to use the Claygent is to write your prompt as depicted in the image. You can use plainspoken, natural language to ask your question—and AI will take care of the rest.
The web scraper allows you to ask all types of questions. Let’s look at a few questions that may naturally come up as you’re determining who you should be reaching out to and how you can use Claygent to get good results. You can also watch our lesson in Clay University to learn more about how to write effective queries.
For instance, if you want to know if a company is hiring, here’s how you would interact with Claygent:
You would ask Claygent: Is [company name] hiring?
You would set the prompt so that you receive responses that are either “true” or “false.”
When you do receive responses, you can view “reasoning” for those answers. This will show you why Claygent returned the results that it did. You can even use the Claygent to route to the sources it used if you want to double check them or dive deeper.
You can also ask the Claygent to web scrape more complex questions that do not have a yes or no answer. For example, let’s say you want to determine if a given company has a pricing page on their site. Here’s how you can find out.
You would ask the Claygent (AI Web Scraper) three questions:
- Does [company name] have a pricing page on their site?
- If so, respond with their URL. If you cannot find a URL, respond “none.”
- Only respond with a pricing page from their website. This is their website: [website].
As you can see, the results will either return a URL or “none” to indicate that the website does not contain a pricing page.
Here are some more example prompts to get you started:
Is [company name] B2B or B2C?
Who are [company name]’s main competitors? List up to 5 company names.
What is the [company name]’s podcast? Provide the name and web address for the company’s podcast.
What is the [company name]’s YouTube Channel? Provide the handle for any YouTube Channel belonging to the company.
Has [company name] ever acquired another company? Answer in true or false.
Does [company name]] allow remote work? Answer in true or false.
Name one or two happy customers featured on [company name]’s website. (This will scrape the company’s domain to find any enterprise deals, news about successful partnerships, or information from a page that displays a list of customers.)
Does [company name] have a free trial? You can then clarify the prompt by saying, “if the website has language on it like “get started” or “sign up” or “free trial”, the answer is yes.
Scrape anything with Claygent (AI Web Scraper)
As you can probably tell, the possible use cases for Claygent are multitudinous. With just one or two lines of natural language, you can easily automate research that once was either impossible or impossible to justify because it would take too long.
Luckily, the future is here and it’s rooted in personalized automation at scale. You can sign up for Clay here and get started using the Claygent immediately! You can also join our Slack Community and post questions, ideas, and your own innovative solutions!
.jpg)
.jpg)
.avif)
