When executing concept or pack testing, the competitive set you choose can make or break your research. A competitive set is the group of products or concepts that your test item is evaluated against. This context is what shapes consumer decision-making — and ultimately drives the accuracy and actionability of your results.

Think of it this way: consumers rarely make buying decisions in a vacuum. Whether online or on a store shelf, they’re comparing your product to a lineup of alternatives. Your research should reflect this reality. Including a balanced, relevant competitive set simulates real-world choices and generates more meaningful insights.

Getting your comparison wrong can introduce significant risk

On a DIY platform like AYTM, it's easy to fall into the trap of choosing competitor examples based on gut instinct, popularity, or convenience. This can lead to:

• Stimulus bias – where the design, color, or tone of competitor materials unfairly influences respondents
• Skewed benchmarks – when competitor items are perceived as much better or worse than typical in the category
• Misguided conclusions – leading business decisions down the wrong path because you're not testing in the right context

Ultimately, the competitive set gives your insights meaning. Say you're introducing a premium water bottle with sustainable features. Testing it only against budget plastic bottles won’t tell you how well it might perform against other eco-friendly brands consumers would actually consider. A lopsided or misaligned comparison could overinflate your concept’s appeal — or, worse, cause it to underperform in testing even though it might succeed in market.

Better competitive sets lead to more actionable results

By thoughtfully choosing the right competitors, you’re not just identifying which concept “wins.” You’re answering deeper, strategic questions:

• What makes your product stand out?
• How does your proposition perform in a realistic scenario?
• Are you competing on price, packaging, benefits — or all of the above?

These questions are especially valuable when your team is working with limited time, budgets, or internal resources. Fortunately, platforms like AYTM are designed for speed — and when paired with experienced professionals from SIVO’s On Demand Talent network, you can unlock both agility and accuracy. With their real-world category knowledge, these professionals ensure your competitive sets align with how your core consumers actually shop — which leads to more trustworthy, decision-ready insights.

Now that we understand why competitive sets matter, the next step is picking the right ones. But what makes one competitor right and another wrong for a concept or pack test? The goal is balance — a mix of relevant competitors that reflect what consumers truly consider when evaluating options in your category.

Step 1: Start with the consumer perspective

Think about how your customers shop. Are they comparing by price? Ingredients? Brand prestige? Understanding these decision drivers will help you identify which products are actually in your competitive set — not just who your marketing team sees as competition.

For example, a mid-priced pet food brand might compete not just with other national brands, but with premium boutique options and emerging DTC products. The right mix should reflect a variety of price points, claims, and pack types shoppers weigh before purchasing.

Step 2: Include direct and indirect competitors

Direct competitors are those with similar features, products, or formats. Indirect competitors might offer a different product type but fulfill the same need. For example, a sparkling water brand might test against other flavored waters (direct) and low-calorie sodas (indirect), since both appeal to health-conscious drinkers looking for a fizzy option.

Step 3: Aim for diversity, not clutter

A balanced competitive set isn’t necessarily large. In fact, too many items can overwhelm respondents and complicate analysis. A common best practice is including

• 2–3 direct competitors that are meaningful in sales or mindshare
• 1–2 indirect alternatives that surface in purchase consideration

This strikes a good balance between realism and simplicity — especially when fielding on agile research platforms like AYTM.

Step 4: Standardize for fairness

To reduce stimulus bias in pack testing or concept reads, make sure all items are formatted consistently. That means using similar:

• Mockup quality (resolution, lighting, angles)
• Backgrounds and white space
• Labeling and descriptors (avoid using overly technical or promotional language for your brand unless competitors have the same)

Well-controlled presentation ensures the consumer is reacting to the product — not to visual or textual differences that could unintentionally sway their opinion.

Step 5: Leverage expert insights

Even with best practices in mind, choosing competitors can still be tricky — especially in fast-changing or niche categories. This is where experience makes a difference. By engaging consumer insights professionals from SIVO’s On Demand Talent bench, brands can tap into deep category understanding without the time commitment of hiring full-time.

These experts can help:

• Map out category decision frameworks
• Identify relevant competitors based on behavior, not assumptions
• Review stimulus for balance and fairness

They also help teach your team how to get more from platforms like AYTM — turning DIY research tools into high-confidence decision support solutions. Especially in lean environments, this kind of flexible partnership can amplify the impact of your market research tools without burning out internal resources.

Stimulus bias is one of the most common and avoidable threats to validity in concept and pack testing. It happens when the way a product or idea is presented influences the consumer response – even if the core idea itself may be strong. In order to get reliable, real-world consumer insights, minimizing this bias is key.

In agile research platforms like AYTM, where speed and automation are part of the appeal, it's important not to sacrifice objectivity. A few small design choices can have a big impact on research outcomes.

Common Sources of Stimulus Bias in Concept and Pack Testing

• Imbalanced imagery: One concept is shown in high-fidelity 3D rendering while others are in sketches or low-resolution photos.
• Descriptive language: Concepts use varying tones or buzzwords that may sway consumers unfairly.
• Brand recognition: Including well-known brands alongside unknown ones without proper contextual framing.
• Order effects: Displaying concepts in a fixed order, leading to preference or fatigue bias.

Best Practices to Maintain Objectivity

When designing concept tests using platforms like AYTM, consistency is your best defense against bias. Every product concept – whether your own or a competitor's – should be presented with equal visual, tonal, and textual clarity.

Here are a few ways to keep things fair:

• Standardize format: Use identical layouts, image types, and content structures for all concepts.
• Neutral messaging: Avoid superlatives or claims unless they're included consistently across all options.
• Rotate display order: Use randomized rotation to eliminate ordering effects.
• Align visual quality: Ensure all packaging visuals, whether real competitor items or new ideas, are equivalent in resolution, sizing, lighting, and styling.

Fictional example: If you're testing a new energy drink concept, and you use a bright, photorealistic image for your concept but blurry cell phone pictures of competitor products – the consumer might choose yours without ever consciously doing a comparison.

As research becomes more democratized through DIY tools, it's tempting to move quickly. But maintaining objectivity requires deliberate controls. The quality of insights you get is only as strong as the neutrality of your test design.

DIY and agile research platforms like AYTM are revolutionizing how businesses approach concept testing and pack testing. They're fast, affordable, and flexible – but without the right strategy, it's easy to miss out on the full value they offer. Effective use of agile market research tools requires both technical know-how and research fundamentals.

What Makes AYTM Powerful for Concept and Pack Testing?

AYTM stands out for its user-friendly interface and advanced automation capabilities. It excels in tasks such as designing surveys, managing panel recruitment, and quickly analyzing consumer preferences. For pack testing or comparing product concepts, AYTM offers features like monadic or sequential monadic testing, max-diff evaluations, and robust real-time reporting dashboards.

To maximize the platform’s effectiveness:

• Start with a clear objective: Define what decision the research will inform.
• Use the right test structure: Choose between monadic or comparative design based on your hypothesis and product pipeline stage.
• Ensure balance in visual and verbal inputs: As discussed earlier, consistent stimulus matters even more in automated platforms.
• Leverage existing libraries: Platforms like AYTM often include standardized attributes, benchmarks, and templates for efficiency without reinventing the wheel.

Where Expertise Makes a Difference

While the promise of self-service tools is appealing, experience still matters. Skilled research professionals – especially those experienced with agile research platforms – can help configure the test correctly, interpret the data intelligently, and avoid common pitfalls. For example, understanding hierarchical Bayes modeling or TURF analysis results often requires more than just a software manual.

Fictional case: A fast-moving consumer goods startup used AYTM to evaluate new skincare packaging. Early internal testing showed strong appeal. However, when an experienced researcher from their insights team reviewed the test, they found the competition was poorly chosen and pack visuals were not balanced. The revised test led to clearer consumer preferences and more impactful decisions.

In short, agile market research tools are powerful – but only when paired with the knowledge to use them to their full potential. Think of them as a high-performance vehicle: easy to drive, but truly efficient when managed by someone who understands the roadmaps of research.

As businesses increasingly adopt DIY research and agile tools like AYTM, many insights teams find themselves navigating new platforms with fewer resources, tighter timelines, and growing pressure to demonstrate ROI. This is where On Demand Talent can be a game-changer – offering expert support when it’s needed most without overextending your core team.

So When Should You Bring in On Demand Talent?

Most organizations don’t need extra hands all the time – they need the right support at the right moments. Here are a few scenarios where tapping into SIVO’s On Demand Talent network can make all the difference:

• Launching a high-stakes product: When accuracy matters most, like for a flagship product or a major brand shift, expertise ensures your concept and pack testing is airtight.
• Navigating new tools: Whether you’ve just started using AYTM or are expanding your agile platform suite, experienced researchers can train your team while executing real projects.
• Filling skill gaps: Short a quant lead? Need a pack testing specialist for 3 months? On Demand Talent gives you targeted support without complex hiring processes.
• Upskilling internal teams: Our professionals not only execute with excellence but help build long-term capability by coaching teams on research best practices within these tools.

Beyond Freelancers: Flexibility Without the Risk

Unlike ad-hoc freelancers or long-term consultants, SIVO’s On Demand Talent gives you instant access to seasoned professionals who know how to move fast, without cutting corners. These are not junior hires or generalists – they’re specialized talent who can manage complex concept testing from start to finish, including selecting the right competitive set, minimizing stimulus bias, and interpreting results through the lens of business strategy.

And because our network spans hundreds of roles and industries, we can match the expertise to your team’s exact needs – whether you’re a CPG brand testing regional pack refreshes or a tech company piloting product messaging in new markets.

With On Demand Talent, you don’t just get support – you add business impact without having to grow headcount or wait months for hiring cycles.