Run an A/B test end to end

This guide follows one test from idea to result. Every test lives on the Board and has four tabs: Design, Launch, Results, and Relations.

If you want the test to run on GrowthBook, Intelligems, or Convert instead of Spar, follow this guide through step 4, then switch to Send a test to your experimentation platform.

1. Create the test

From an opportunity:

1. Open an accepted opportunity from the Active tab of Opportunities.
2. Under Ideas and tests, click Review on the idea you want to try.
3. In the idea panel, click Create test. A toast confirms it with an Open test link.

Or start from the Board: click Create test, describe the test, and pick where it runs under Run this test on.

New tests appear on the Board in Generating or Setup while Spar prepares them, then move to Backlog.

2. Pick a starting design

Open the test. It starts on the Design tab.

1. Check the test details on the right: purpose, hypothesis, and target page.
2. If you want to steer the designs, describe the change in Proposed change and click Generate designs.
3. Spar creates three starting designs. This usually takes a couple of minutes.
4. Click a design to see it in Selected design, then click Build this design at the bottom.

Spar builds the design into a live preview of your storefront and opens the variation editor when it's done. If the build doesn't finish, your selected design is saved and you can click Try again.

3. Fine-tune the variation

The variation editor shows a live preview of the page with your change.

• Ask for changes in chat. Describe what to adjust. Use the select tool to point at a specific element on the page first.
• Edit content directly. The Content items list holds the text, images, and other content the variation uses. Edit a field and click Save changes.
• Upload an image. Image fields take either a link or a file. Paste a URL into Enter image URL, or click the upload button next to it and pick a JPEG, PNG, WebP, GIF, or SVG file up to 5MB.
• Approve the preview. If the editor shows Review required, check the preview and click Approve desktop preview (and the mobile one if it asks), then Continue to launch.

4. Rename or add variations

Back on the Design tab, each variation has a card.

• Rename a variation: open the card's ... menu, choose Rename variation, type the new Variation name, and click Save. You can do the same from the ... menu next to a variation in the editor's sidebar.
• Add another variation: click the Add variation card to start a new design from the test details.
• Share a preview: ... > Share preview link copies a link to the live preview.
• A card's main button tells you what's left: Finish design, View progress, or Edit.

5. Set up the launch

Open the Launch tab (or click Launch test in the test header, which takes you there).

Across the top:

• Primary objective: the metric that decides the test.
• Also measure: extra KPIs to track alongside it.
• Scheduled end: when the test stops, such as After 14 days, Until you end it, or a custom number of days.
• Visitor observations: an optional cap on how many visitors the test sees. No cap is the default.
• Smart-end tests: when on, Spar stops the test early if a clear winner is found.

In the Variations card:

• Drag the split slider, or type a percentage for each variation, to set the traffic split. Control is the page as-is. Every variation needs a share above 0%.
• Turn on Bandit to let Spar shift traffic toward the better performer instead of using a fixed split. It isn't available for every objective (Bounce Rate, for example) and can only be changed while the test is in Backlog.
• Click the eye icon on a row to preview that variation live.

In the Target page card:

• Pick the page from the list, or click Enter a URL instead.
• Turn on Run on all product pages in this template (the wording follows the page type) to run on every page of that type.
• URL targeting: All, or Include / Exclude with one URL per line, matched against the page path. Excluding cart and checkout from a site-wide test is a common use.
• Device targeting: All devices, Desktop, or Mobile.
• Visitor targeting: All, New, or Returning visitors.

The Experiment platform card has Run this test on. Leave it on Default (Spar) to run the test with Spar.

Click Save changes when you're done.

6. Clear the blockers and launch

The bar at the bottom of the Launch tab tells you whether the test can go live. If it says something like 1 blocker to launch, click Review blockers.

Site install not confirmed only warns you. You can still launch, and the button changes to Launch anyway, but the test may not run until the script is installed. On a Shopify store connected through the Spar app, the check looks at the app embed instead. If the embed is turned off it does block, and Enable app embed opens your theme editor so you can turn it on.

When the bar reads Ready to launch (or Ready to launch with 1 warning), click Launch test (or Launch anyway). Spar starts the test and takes you to the Results tab.

7. Watch the results

Once the test is running, the Results tab opens up and shows:

• Overall performance for each variation against Control.
• Current leader while the test runs, and Result after it ends.
• Charts of chance to win and relative change over time.
• Where the lift comes from, a breakdown by segment.

At the top, a status message tells you where things stand, for example Waiting for data, Collecting more data, or Clear winner identified.

8. End the test

A test ends when it reaches its Scheduled end or Visitor observations cap, or earlier when Smart-end tests finds a clear winner. If you've turned on End tests automatically once a winner reaches the confidence above in Settings > Workflow, Spar ends it as soon as a variation reaches the confidence level you set there.

To stop it yourself, drag its card into Completed on the Board. To take a break instead, click Pause test in the test header and resume it later from the Launch tab.

If a test isn't getting results

• Waiting for data: the test is running but no visitors have seen it. Check that the Spar script is installed (Settings > Installation) and that the target page and targeting rules match real traffic.
• No clear winner: the variations are performing about the same. Let it run longer, or rethink the variation.
• Not enough traffic: tests on low-traffic pages take a long time to reach a result. If the change is low-risk, ship it as a fix instead: while the test is still in Backlog, open its ... menu and choose Convert to fix.