Marketing experiment log: how to document tests and trust the results

What is a marketing experiment log?

A marketing experiment log is a running record of every test your team runs: the hypothesis, the exact change, where it ran, the start and end dates, the metric you judged it on and the result. It answers "did we already try that?" and "why did we roll that out?" months later.

Most teams keep test results somewhere: a slide, a Slack thread, the Google Ads Experiments page. The problem is that these records are scattered and often lose the context that matters most, such as what else changed while the test was running. A log puts every test in one place, in the same format, with dates you can line up against your results.

An experiment log is not the same as a general marketing change log, but the two belong together. The change log records every edit to campaigns, the website, tracking and the CRM. The experiment log records the subset of changes you made on purpose to learn something. When both share one timeline, you can see which routine changes overlapped each test.

Keep one for three reasons:

• Avoid repeat tests. New team members and new agencies often rerun tests that already failed because nobody wrote the result down.
• Defend decisions. When a client asks why the landing page looks the way it does, you can show the test, the dates and the numbers.
• Judge results fairly. A test is only as clean as the period it ran in. The log tells you whether that period was quiet.

What should every experiment entry include?

Every entry should hold enough detail that someone who wasn't there can repeat the test or trust its result. Optimizely's experiment plan guidance groups this into why you run it (the hypothesis), when and where the variation runs, who sees it, what the variation changes and how success is measured. Add the result and the decision, and you have a complete record.

The last field is the one most templates leave out, and the one that decides whether a result can be trusted. Fill it in from your change log at the end of the test, not from memory. If you want a ready structure, start from a marketing change log template and add columns for the hypothesis, method and decision.

How do I log Google Ads and Meta Ads experiments?

Log every platform experiment the day you schedule it, then again when it ends. The ad platforms store test settings, but they don't record your hypothesis, the decision you took or the changes made around the test.

Google Ads experiments

In Google Ads, custom experiments live under Campaigns → Experiments, where you select the plus button and choose Custom. A few rules from Google's help pages are worth copying into your log, because they change how you read the result:

• Search campaigns can use a cookie-based or a search-based split; Display campaigns always use cookie splits. Write down which one you chose.
• The traffic split can't be changed after setup, so the split in your log stays valid for the whole test.
• Google recommends running an experiment for at least 4–6 weeks, longer if conversions are delayed, and excludes the first 7 days from experiment results to account for ramp-up.
• Google says changes made to the original campaign or the experiment while it runs can make results harder to interpret. Each such change belongs in the "overlapping changes" field.
• Experiments ended manually aren't auto-applied; only those that reach their defined end date are. Log the actual end date and how the test ended.

For a worked example of testing a page this way, see how to measure a landing page change.

Meta Ads tests

Meta Ads Manager has its own A/B tests. Log them with the same fields: the variable you tested, the audience, the dates, the metric you picked and the result Meta reported. Add any budget, audience or creative edits made to the tested ad sets during the test window, because Meta's report can't separate those from your variable.

Before/after tests on the website

Many tests have no split at all: a new page goes live and you compare weeks. These need the log most, because nothing in the test design protects you from other changes. Write down the exact go-live time and treat every overlapping change as a threat to the result.

Why do experiment results get mixed up with other changes?

Because a test never runs in a vacuum. During the weeks a test is live, someone raises a budget, Smart Bidding adjusts, a newsletter goes out, a tag is republished or a competitor runs a sale. Any of these can move the metric you're watching.

Split tests handle part of this: if both arms get the same kind of traffic at the same time, a seasonal lift hits both. But a change that affects one arm more than the other, or a tracking change that breaks one conversion action, still distorts the result. Before/after tests are exposed to everything.

The common sources of contamination:

• Edits inside the test. Budget, bid target or audience changes made to the tested campaigns.
• Tracking changes. A new Google Tag Manager version, a consent banner update or a changed conversion action can move counts without any change in behaviour.
• Other channels. Email, social or offline promotions that send different visitors during the test.
• External events. Holidays, sales periods and platform outages.

Annotations in reports help but don't replace the log. GA4 annotations, for example, allow a 60-character title and a 150-character description, with up to 1,000 per property, so they work as markers on a chart, not as a record of a hypothesis and its result. Use them to point to the log entry; the GA4 annotations guide shows how to set them up.

How do I prove which change caused an experiment's result?

Put the test and every other change from the same period on one timeline, rule out tracking first, then check whether any overlapping change could explain the difference on its own. If none could, the tested change is your best explanation; if one could, the result is inconclusive and the test should be rerun.

1. Confirm tracking held. Check that conversion actions, tags and consent settings didn't change during the test window.
2. List overlapping changes. Pull every edit from the test dates and the week before: your team's changes, platform automations, site releases and promotions.
3. Ask which arm each change hit. In a split test, a change that hit both arms equally is less of a threat than one that hit only one arm.
4. Compare timing. If the metric moved on the day of an overlapping change rather than at launch, that change is the more likely cause.
5. Write the verdict. Proven, likely or inconclusive, with the reason.

With every change on one timeline, you would see at a glance that a budget increase landed in the middle of your page test, before you report the lift as the page's win. Changeline keeps that timeline per client: Google Ads edits are imported automatically, and the team logs site, tracking, email and test changes next to them in a shared change timeline.

Example. A SaaS client asks: "Your new pricing page lifted sign-ups 18% last week. Can we credit the page?" The team lines up the changes from that week. The page went live on Monday, but on Wednesday the Meta Ads budget was raised 25% and on Thursday a promo newsletter went out. Sign-ups jumped on Friday, not Monday. The verdict: inconclusive. The team reran the page as a 50/50 split test, which showed a 6% lift. (Illustrative names and numbers.)

How do I keep an experiment log useful over time?

Make logging part of launching a test, review the log on a fixed rhythm and keep old entries even when the test failed. A log is only useful if it is complete and easy to search.

1. Write the hypothesis
2. Log dates and method
3. Hold other changes
4. Record the verdict

• No entry, no launch. A test without a written hypothesis and success metric doesn't start.
• One naming pattern. Use the same name in the log, the ad platform and your reports, for example 2026-11 · Search · tCPA 40 vs 32.
• Keep the failures. Failed and inconclusive tests stop the next person from repeating them.
• Review monthly. Read the month's tests next to the month's results. It also makes explaining a drop to a client much faster.
• Hold changes during a test. If an urgent edit has to happen, log it, then mark the test as affected.

For a wider view of judging whether a change worked, see how to measure the impact of marketing changes.