Checklist for auditing ai lead scoring fit
Learn how to manually audit ai lead scores to ensure your icp is accurate. Use this process to spot-check samples and refine your customer descriptions.
Get startedShort answer
To tell if ai lead scoring is accurate, you must manually audit a random sample of high, medium, and low scores. Compare the ai's reasoning against your actual ideal customer profile (icp) to see where it is overvaluing irrelevant keywords or missing subtle disqualifiers, then refine your icp description to close those gaps.
Define your perfect fit benchmarks
You can't tell if a score is wrong if you don't know what right looks like. Ai scoring often fails when human instructions are too broad. If you tell an ai to find growth marketers, it might include growth coaches, freelancers, or business consultants because they all use the word growth in their headlines.
Before looking at the data, write down your hard boundaries. These are the filters that should result in an automatic zero or a very low score, no matter what other positive signals are present. If a lead is a freelancer but has the perfect title, they are still a zero if you only sell to in-house teams.
Most founders make the mistake of describing the person they want to talk to rather than the business they want to serve. A person can have the right title but work at a company that is too small to afford your product or too large to move quickly. Your benchmarks must include both the persona and the company attributes.
When you define these benchmarks, avoid adjectives. Words like successful, experienced, or innovative are useless to an ai. They don't have a concrete meaning in a linkedin profile. Instead, use measurable data like employee count, specific software mentioned in their tech stack, or exact department names.
Vague benchmarks
- Target mid-sized companies
- Looking for marketing leaders
- People interested in efficiency
Concrete benchmarks
- Company size must be 50 to 200 employees
- Must have a title containing VP or Head of Growth
- Must be in the SaaS or Fintech industry
A hard disqualifier is a trait that makes a lead a zero, even if every other part of their profile is perfect.
Pull a diverse audit sample
Checking only your best leads creates confirmation bias. You'll see a few winners and assume the system is perfect, while missing the fact that a large part of your pipeline is actually junk. You need to look at the edges of your scoring distribution to find where the ai is guessing.
Export a list of 30 to 50 recent leads. This is a small enough sample to audit in an hour but large enough to spot patterns in how the ai is misinterpreting your icp. If you only look at 5 leads, you might just be seeing a fluke. If you look at 50, you'll see the logic errors.
The most dangerous errors are the false positives, where a lead gets a 90 but is a total mismatch. These waste your time and can damage your linkedin account reputation if you send irrelevant messages. However, false negatives are also a problem because they represent missed revenue from perfect fits who just use odd phrasing.
Organize your sample by score buckets. Do not just look at the top of the list. You need to know why a lead was scored a 40 and why another was scored a 60. This reveals the sensitivity of the scoring engine and tells you if your fit line is set too high or too low.
- 1
Isolate the sure things
Pick 10 leads with the highest scores. Verify if they actually meet all your non-negotiable requirements including company size and industry.
- 2
Analyze the maybes
Pick 10 leads with mid-range scores. Determine if they are actually high-quality leads the ai underestimated or mediocre leads it overvalued.
- 3
Review the no-gos
Pick 10 leads that the ai rejected or scored lowest. Check if any of these were actually perfect fits that used unusual terminology for their role.
- 4
Document the gap
For every lead where your score differs from the ai score, write one sentence explaining why. This becomes the basis for your prompt updates.
The manual scoring test
Now you compare the ai's logic to your own. The goal is to find the gap between keyword matching and actual intent. Ai often gets tricked by people who talk about a topic in their about section but don't actually hold the role you need for a sale.
Look for patterns. If you see five leads scored 90+ who are all consultants instead of in-house executives, you have a keyword problem. The ai is seeing the word growth and ignoring the word consultant. This is a systemic error that will repeat until you change the instructions.
Check for title synonyms. You might be looking for a VP of Sales, but the ai might be scoring a Head of Partnerships low even though they have the same buying power in your specific niche. If you find a pattern of missing good leads, you need to expand your vocabulary.
Pay attention to company descriptions. Sometimes a lead has the perfect title but the company description mentions they are a recruiting agency. If the ai doesn't catch that they are a service provider rather than a software company, your scoring is too focused on the person and not enough on the entity.
Lead A
- Lead Profile
- Growth Coach, 1-person biz
- Ai Score
- 95
- Your Score
- 10
- Reason for Difference
- Keyword match on growth, missed company size
Lead B
- Lead Profile
- VP Demand Gen, 150 employees
- Ai Score
- 40
- Your Score
- 90
- Reason for Difference
- Missed title synonym for Growth
Lead C
- Lead Profile
- CEO, 10 employees
- Ai Score
- 80
- Your Score
- 20
- Reason for Difference
- Overvalued CEO title, ignored size limit
Adjusting your icp description
Once you find the patterns, update your icp description. Don't just add more keywords; add constraints. If the ai is bringing in freelancers, tell it explicitly to ignore them. Constraints are more powerful than keywords for cleaning up a lead list.
If you are missing good leads, expand the vocabulary. People in different regions or industries use different words for the same job. A Head of Demand Generation in New York might be a Growth Lead in London or a Marketing Director in a traditional industry.
Test your updates in small batches. Change one part of your description at a time. If you change the title requirements and the company size limits at once, you won't know which change fixed the problem or which one accidentally blocked good leads.
Use a negative list in your description. List the exact words that should trigger a low score. This includes common words found in the profiles of people who look like your customers but aren't, such as aspiring, student, or seeking opportunities.
Example
- You
My ai is scoring every Growth Coach as a 90. I need to fix the prompt.
- Suggested reply
Update the description to: Target VPs of Growth at B2B SaaS companies. Exclude anyone with Coach, Consultant, or Freelancer in their title or bio.
- You
Now it's missing the VPs of Demand Gen. I'll add those as an approved synonym.
Avoid using words like experienced or successful in your description. These are subjective and confuse the ai.
Building a long term audit loop
An icp is not a static document. As you start conversations, you'll realize that the people you thought were perfect fits are actually not the ones who buy. You might find that VPs of Growth are too busy, but Directors of Growth have all the budget and authority.
Create a feedback loop between your inbox and your lead scoring. Every time you get a reply that is a total mismatch, go back to the lead's profile and ask why the ai gave them a high score. Use that specific profile as a negative example to refine your constraints.
The goal is to reach a state where 90% of your high-scoring leads are people you would actually be happy to get on a call with. If you are still spending more than ten minutes a day filtering out junk, your description is still too broad.
Keep a log of the changes you make to your icp. If you find that your reply rates drop after a specific update, you can easily revert to the previous version. This prevents you from over-optimizing for fit and accidentally killing your volume.
Paste this job
ICP Update Log Template: Date: [Date] Issue: [e.g., Too many freelancers] Change: [e.g., Added 'Exclude consultants' to constraints] Result: [e.g., High scores dropped by 20%, but lead quality improved]
Doing this with Kinlead
Kinlead automates this scoring process by matching people on linkedin against the customer profile you describe or your website. It scores each lead from 0 to 100 and keeps only those above the fit line you set, so you don't have to manually sift through lists.
To refine your accuracy, you can update your profile description and re-run the search to see how the scores shift. This lets you iterate on your icp without manual searching. Leads are kept in lead groups, allowing you to organize different test segments for your audit.
Because Kinlead uses linkedin's standard search, you can find these leads without a sales navigator seat. You can then use the lead scoring and fit lines feature to ensure only the highest quality prospects enter your campaigns.
Frequently asked questions
Related
Get started with Kinlead
Prefer markdown? Checklist for auditing ai lead scoring fit.md