# Checklist for A/B testing AI outreach
> Learn how to A/B test LinkedIn messages by splitting lead groups and testing single variables to find the script that generates the most replies.
- HTML: https://getkinlead.com/checklist-for-ai-outreach-a-b-testing
- Markdown: https://getkinlead.com/checklist-for-ai-outreach-a-b-testing.md

**Short answer:** To A/B test LinkedIn messages, split a target audience into two identical lead groups. Send one version of your script (the control) to group A and a version with one changed variable (the variant) to group B to see which gets more replies.

## Pick one variable to test

If you change three things at once and your reply rate goes up, you don't know which change caused the result. You have to isolate a single variable. The messages in Group A and Group B should be identical except for the one specific element you're testing.

Most founders start with the hook since it's the first thing a prospect sees. If the hook fails, the rest of the message is ignored. Once you find a winning hook, move to testing the offer or the call to action.

Common variables include the opening line, the value proposition, the length of the message, or the specific request at the end. Testing the tone (formal versus casual) is also a common starting point for teams moving into a new industry.

**Testing too many variables**

- Group A: Formal tone, long pitch, asks for a 30 minute call.
- Group B: Casual tone, short pitch, offers a free PDF, asks for a quick chat.

**Testing one variable (The offer)**

- Group A: 'Would you be open to a 15 minute demo to see how this works?'
- Group B: 'Would you be open to a free 5 minute audit of your current process?'

> Never test a new hook and a new offer in the same batch. You'll end up with a winner you can't replicate because you don't know why it worked.

## Split your leads into equal groups

A test only works if the two groups are similar. If Group A has CEOs and Group B has Managers, the results show who is more likely to reply, not which script is better. This is called selection bias and it ruins your data.

You need a random distribution to avoid this. This ensures the performance difference comes from the message and not the people receiving it. If you have 200 leads, you should have 100 in each group.

If your list is small, you might be tempted to just send to whoever you find first. This is a mistake. The first 50 people you find might be from a specific region or industry that happens to be more responsive, which will skew your results for the second 50.

1. **Export your target list.** Pull your full list of target prospects into a CSV file so you have a master record of everyone in the test.
2. **Randomly assign leads.** Split the list 50/50. Assign a random number to each row in a spreadsheet and sort them to do this.
3. **Verify the mix.** Check that both groups have a similar ratio of company sizes and seniority levels.
4. **Lock the groups.** Tag these leads as Group A and Group B. Make sure no lead is in both groups so they don't see both versions.

## Set up the control and the variant

The control is your baseline. This is usually the script you've been using or a standard approach that gets a known level of response. The variant is your hypothesis where you apply the change.

Consistency matters as much as the copy. If you send Group A's messages on Tuesday morning and Group B's on Friday afternoon, the timing might skew your data. People are generally more responsive on certain days of the week.

Your variant should be based on a specific theory. For example, if you think your current request for a call is too aggressive, your variant should test a lower-friction request, like asking for permission to send a video. This gives you a clear reason why the variant might outperform the control.

| | Element | Control (Group A) | Variant (Group B) |
| --- | --- | --- | --- |
| Hook | I saw your post about X. | I saw your post about X. |
| The Variable (Offer) | Can I send a 2 minute video? | Can I send a 1 page PDF? |
| Closing | Best, Sarah | Best, Sarah |
| Send Window | 9am to 11am EST | 9am to 11am EST |

## Run the test and track replies

Don't stop the test after ten messages. You need enough data to be sure the result isn't a fluke. Send to both groups at the same time and let them run until you hit your target sample size.

Focus on high-intent replies. A 'thanks' or 'not interested' is a response, but it isn't a conversion. You want the version that starts a real business conversation. This means the person is asking about your product or agreeing to a meeting.

Keep a simple tally of the results. Do not overcomplicate the tracking. A basic spreadsheet with the number of messages sent and the number of positive replies for each group is all you need to determine the winner.

1. **Launch simultaneously.** Start both campaigns on the same day to keep external market conditions identical.
2. **Set a minimum sample.** Decide on a number of sends per group (like 50 or 100) before you analyze the results.
3. **Tally positive replies.** Count only the leads who asked a question, agreed to a call, or showed genuine interest in the offer.
4. **Declare the winner.** The version with the higher percentage of positive replies becomes your new control for the next test.

- **draft:** Hi Mark, saw your post on scaling agencies. I've built a tool that automates the lead scoring part of that. Would you be open to a 15 minute demo?
- **them:** I'm busy this week, but I'd be open to seeing a quick recording of it first.
- **you:** Totally get it. I'll send over a 2 minute Loom video now. Does that work?

## Refining the process for long term growth

Once you have a winning script, it doesn't stay the winner forever. Market fatigue happens, and your target audience may stop responding to a specific phrase or offer after a few months.

The goal is to create a cycle of continuous improvement. Take your winner, make it the new control, and then introduce a new variant. This allows you to incrementally increase your reply rate over time.

You can also test different segments of your audience. A hook that works for a founder might not work for a VP of Sales. If you notice a drop in performance, try splitting your tests by job title to see if different personas require different messaging.

```
Hypothesis: Changing the CTA from 'Book a call' to 'Ask for a resource' will increase positive replies by 10% because it lowers the initial commitment for the prospect.
```

## Doing this with Kinlead

You can automate this by creating two separate [lead groups](/features/lead-groups-and-scoring) in Kinlead and pointing a different [campaign](/features/campaigns-and-send-windows) at each. Use the AI message generator to create your control and variant scripts based on your own sales playbook.

To track quality, set the [AI reply mode](/features/ai-linkedin-outreach) to hand-off at the first reply. This lets you manually verify if the lead is actually interested. All replies land in one [unified inbox](/features/unified-inbox), where you can see which campaign generates more intent-driven conversations.

Kinlead sends at a human pace, sending at most one action every 5 minutes inside the daily limits and send windows you set. This ensures your A/B tests aren't skewed by LinkedIn's automated filters or account restrictions. If you want to see how this works in a live environment, you can book a [demo](/demo).

## Related

- [Book a demo](https://getkinlead.com/demo.md)
- [AI LinkedIn outreach](https://getkinlead.com/features/ai-linkedin-outreach.md)
- [Lead groups and scoring](https://getkinlead.com/features/lead-groups-and-scoring.md)

## Common questions

**Can I test three different versions at once?**

You can, but it requires a larger total list of leads. Split your audience into three equal groups (A, B, and C) and keep the same constraints on variables.

**How many replies do I need before a winner is decided?**

Wait until you have sent the same number of messages to both groups. Look for a clear gap in the percentage of positive replies.

**Should I test different connection request notes or only the first message?**

Start with the connection note. If people don't accept your invite, they'll never see your first message. Once your acceptance rate is stable, test the follow-up messages.

**What happens if I change the offer and the hook at the same time?**

You won't know which change worked. If the reply rate increases, you won't know if it was the better hook or the better offer, so you can't optimize further.

[Create a Kinlead account](https://getkinlead.com/signup) or [book a call](https://getkinlead.com/demo) to talk through pricing and setup.
