Orita 4.0 Model Update: FAQs

Ahead of Labor Day, we are launching an updated version of Orita's core models. Below are the questions we are hearing most from brands.

What's changing?

We're rolling out an updated Orita model to improve your campaign performance.

The update uses a more modern approach to measurement that allows Orita to score and tag more of your list. As a result, you'll see larger Orita-tagged audiences in Klaviyo, meaning more profiles are available for campaigns and flows.

No action is required from you.

What is changing with the control group?

We are replacing the permanent control group with just-in-time holdouts.

Previously, a fixed group of profiles was held out on an ongoing basis. Going forward, Orita will create a temporary randomized holdout when measurement is needed for a specific product or campaign.

This preserves statistically valid measurement without permanently sidelining part of your list.

Why did my Orita audience get bigger? Did something break?

No. The increase is expected.

Previously, a permanent portion of profiles was held back from scoring for measurement purposes. With the model update, more profiles can be scored and included in your Orita-tagged audiences.

You may see segment growth of roughly 5 to 10%, depending on your list and audience mix. The scoring quality is unchanged or improved.

Does this change how Orita measures incrementality?

Orita will continue to measure incrementality and reporting remains statistically rigorous.

The change is in how holdouts are created, not whether performance is measured. Instead of relying on a permanent control group, Orita uses product- and context-specific holdouts when needed.

This approach is already used for newer Orita products, including Bandit flows and Direct Mail.

What will happen to profiles that were previously in the permanent control group?

Those profiles will be available to be scored immediately.

If they meet the relevant criteria, they may enter active audiences such as Highly Engaged, Moderately Engaged, Rescue, or reactivation audiences. This is the primary reason your Orita segments may grow.

Orita can provide before-and-after counts by score or engagement tier, as well as a download of formerly held control profiles if needed.

Can I identify profiles that were formerly in the control group?

In Klaviyo, these will generally appear as profiles that were previously unscored.

If you need a specific list of formerly held profiles, your Orita team can help provide one.

Does this affect Orita Grow?

Grow will work the same way from your perspective.

There will still be a weekly holdout for measurement. The difference is that the holdout will be generated for that week rather than drawn from a permanent pool. Reporting and the measurement structure should look the same.

Is the Not Engaged A/B test affected?

No. The Not Engaged A/B test is separate from this model update.

It has its own purpose-built treatment and holdout groups because those profiles need to be labeled for the test. The existing testing views should continue to be interpreted as before.

Do I need to do anything differently for my Labor Day sale?

No action is required.

The update will take effect automatically. You may notice that Orita-tagged audiences are larger than usual. That is intentional and means more of your list is scored and available.

If you want help choosing a score threshold for a specific campaign, contact your CX team. They can use the campaign performance dashboard to help balance audience reach and efficiency.