Skip to main content

How to Configure Your Price Test Statistical Settings

When starting a price test, configuring your Statistical Settings is crucial, as these parameters will directly affect the sample size required for accurate results. You can either manually configure these settings or use the default settings provided by WorthTestify. This guide will walk you through the meaning of each setting and how to configure them for optimal performance.

Baseline Conversion Rate​

The Baseline Conversion Rate refers to the current conversion rate for your product or control group before any changes are made. This metric helps establish a point of comparison for your price test.

  • Default Value: 1.5%
  • Purpose: The baseline conversion rate is used to determine how likely a user is to make a purchase before you introduce price variations.
  • How to Set: If you know your existing conversion rate, you can manually input it here. Otherwise, you can use the default value based on industry averages.

Minimum Detectable Effect (MDE)​

The Minimum Detectable Effect (MDE) represents the smallest percentage change in conversion rate that you want to detect in your test. This is the level of impact that would make the price change worthwhile.

  • Default Value: 1%
  • Purpose: The smaller the MDE, the more sensitive your test will be, meaning it will require a larger sample size to detect smaller differences in performance.
  • How to Set: Consider how much of a difference in conversion rate would justify making a change in pricing. For smaller changes (e.g., a 1% increase), set a low MDE. For larger, more impactful changes, set a higher MDE.

Note: Our MDE setting is an absolute number. For example, if your baseline conversion rate is 1.5% and you set an MDE of 1%, you're aiming to detect an increase to 2.5% conversion rate.

Statistical Power​

Statistical Power refers to the probability that your test will detect a real difference between groups, assuming there is one. A higher statistical power increases the likelihood of your test identifying meaningful results.

  • Default Value: 80%
  • Purpose: Power is usually set to 80%, which means there's an 80% chance of detecting a real difference if one exists. Increasing this percentage will require a larger sample size but reduce the risk of false negatives.
  • How to Set: We recommend keeping the default setting unless you have specific reasons to increase or decrease the power based on your test's goals.

Significance Level (p-value)​

The Significance Level, also known as the p-value, determines how likely it is that the observed differences in conversion rates happened by chance. A lower significance level reduces the probability of false positives (finding a difference when there isn't one).

  • Default Value: 5% (0.05)
  • Purpose: The significance level indicates the risk you're willing to take in concluding that a price difference is significant when it may have occurred by chance. A 5% significance level is standard for most tests.
  • How to Set: If your test is highly sensitive and the impact of a false positive would be high, you can lower this value. Otherwise, the default 5% is widely accepted as a good balance.

Conclusion​

Configuring your Statistical Settings properly is essential for running an effective price test. WorthTestify provides default values based on general best practices, but feel free to adjust these settings to meet your business’s specific needs. Once your settings are configured, your price test will calculate the required sample size and proceed accordingly.