Our Methodology — How We Test, Score & Rank Brands

Editorial Standard · Public Since 2024

"If you can't see how we got the score, you can't trust it."

The Idea

Why Our Method Is Public.

Most comparison sites won't tell you how they rank brands. They list winners. They show stars. They don't show the work. We think that's the wrong way around.

WhichRanks publishes the methodology because a verdict you can't audit isn't really a verdict — it's a recommendation dressed up as research. Every ranking on this site goes through the same four-stage process. Every score is weighted across the same eight axes. Every price is refreshed on the same schedule.

"The cheapest way to lose a reader's trust is to win their click and then admit, three pages in, that the winner paid us to call them the winner."

We built this page so a journalist could fact-check us, a brand could understand why they ranked where they did, and a reader could decide whether the methodology matches what they actually care about. For the company story behind it, see our About page. For the policy that backs it, read our editorial policy.

Four Principles

The Rules Behind The Rules.

Before we test anything, before we score anything, every member of the editorial team agrees to these four operating principles. Break one, and the comparison doesn't ship.

Real-World Testing.

We buy products, sign up for SaaS, install apps, sleep on mattresses, eat at hotels' free breakfast. Vendor demos and press kits never make it into a final score.

Verify Every Price.

Every price on every page is cross-checked against the brand's official site within 30 days. Promotional, annual, monthly, and renewal prices all listed where they differ.

Show The Math.

The 8-axis scoring rubric is public. Weights are public. When a brand scores 9.5, you can see exactly which axes lifted it — and which ones held it back.

Commissions Don't Decide.

Affiliate partnerships fund the site, but they never determine rankings. Our scoring is locked before negotiations begin. Our editorial policy spells it out.

The Process

The Four Stages Of Every Comparison.

Every brand on WhichRanks — from Shopify to Nike to NordVPN — passes through this exact four-stage process. No shortcuts. No exceptions. Here's how it works, stage by stage.

~50

Brands Per Category

Stage 01 · Research & Shortlist

We Start By Reading Everything.

Before any product gets purchased, the category lead spends 1–2 weeks mapping the landscape. We pull every relevant brand into a working spreadsheet — typically 40–60 candidates for a major category — then narrow to a testable shortlist of 8–15.

Inclusion criteria are public on each category page: minimum customer base, minimum time in market, regulatory compliance where relevant, and availability in our coverage regions. Brands that don't make the cut are listed too — with the reason.

Brand pool: 40–60 candidates identified across SEO, social, industry reports
Shortlist: 8–15 brands pass inclusion criteria and reach testing stage
Public criteria: Every category page lists why brands were included or excluded
Reader signals: We weight what real users search for — not what brands push

30 days

Average Test Window

Stage 02 · Hands-On Testing

We Use Each Brand Like You Would.

This is the stage that takes the most time — and where most comparison sites cut corners. Every shortlisted brand gets a real test window of 14–60 days, depending on category. We don't accept vendor demos, curated press kits, or "trial accounts with elevated privileges."

For SaaS, we sign up like a regular customer using the public sign-up flow. For physical products like mattresses or sneakers, we buy them at retail price. For VPN services, we run real speed benchmarks across 25+ server locations. For hotel platforms, we book real stays.

SaaS: 30-day real-account trial · self-serve sign-up · same plan tier as users
Physical goods: Bought at retail · returned within window where applicable
Travel: Real bookings · same anonymous-browser flow most readers experience
Security: Independent speed tests, leak tests, and policy audits

Weighted Scoring Axes

Stage 03 · Scoring & Math

We Score Across Eight Axes.

Every brand gets a score from 1–10 on eight standardized axes — value, performance, ease of use, support, features, design & UX, reliability, and trust. The weighting changes by category (performance matters more for VPNs; design matters more for fashion), but the axes don't.

The final score is the weighted average, rounded to one decimal place. Two different editors independently score each brand on each axis, and we resolve any 2+ point gaps through a third reviewer. The full rubric — including current 2026 weights by category — is in the table below.

1–10 scale: Five tiers from "Poor" through "Excellent" — see scoring scale section below
Two independent scorers: Discrepancies of 2+ points trigger a third review
Public rubric: See the full 8-axis weighting table in the next section
Category-specific weights: Performance ≠ value ≠ design; weights match what readers want

Editorial Review Layers

Stage 04 · Verdict & Publish

Three Editors. One Honest Verdict.

Once scoring is locked, the category lead drafts the page. Three independent reviewers sign off before anything ships: the category lead who ran the tests, the Head of Methodology who verifies the math, and the Editor-in-Chief who reviews tone and accuracy.

Every review page and comparison is dated. When pricing or features change, we update the page and re-date it — the version history is visible at the bottom. Mistakes happen; when they do, we correct publicly and disclose what changed.

Three-reviewer sign-off: Category lead → Head of Methodology → Editor-in-Chief
Public dating: Every page shows last-updated date in footer or header
Public corrections: Errors get fixed openly, with a dated correction log
Re-test cadence: Major categories re-scored at least annually; pricing every 30 days

The Rubric

Our 8-Axis Scoring Framework.

Every brand on WhichRanks is scored across these eight axes. Weights vary by category — a VPN's "performance" matters more than its "design", while a fashion brand's design matters far more than its uptime. Below is the default weighting that applies to most categories.

Scoring Axis	Default Weight	Higher For	Lower For
ValueTotal cost of ownership vs. what you get	20%	Mattresses · Broadband	Fashion · Luxury
PerformanceSpeed, reliability, raw output, measurable benchmarks	15%	VPNs · Hosting · Analytics	Fashion · Booking sites
Ease of UseSetup time, learning curve, day-to-day friction	15%	Productivity · eCommerce	Enterprise software
SupportResponse times, channels, expertise of agents	12%	Hosting · SaaS · Travel	Open-source · Self-serve
FeaturesBreadth and depth of what you can actually do	12%	Enterprise SaaS · eCommerce	Minimalist tools · DTC
Design & UXVisual quality, app polish, brand expression	10%	Fashion · Sneakers · Furniture	Backend tools
ReliabilityUptime, build quality, durability, consistency	8%	Hosting · Mattresses · Travel	Fast fashion · Trendy items
TrustPrivacy, security, audits, transparency, ethics	8%	VPNs · Analytics · Finance	Furniture · Fashion

Default weights add up to 100%. Category-specific weights shift up to ±10 points on any single axis, with the changes documented on each category page.

Pricing Standards

How We Keep Prices Current.

Stale pricing is one of the most common ways comparison sites mislead readers. We've built our entire pricing workflow around never letting a price get older than 30 days. Every comparison page goes through an automated check that flags any price quoted from a source older than its 30-day window.

When prices change, the page gets updated and re-dated. When promotional offers expire, we say so. When introductory pricing differs from renewal pricing, both are listed. There's no editorial value in hiding the renewal rate from a reader.

✓

Verified within 30 days. Every price on every page is cross-checked against the brand's official site monthly. Anything older than 30 days gets flagged for re-verification.
✓

Renewal pricing disclosed. Where introductory pricing differs from renewal pricing — common for VPNs, web hosting, and SaaS — we publish both clearly.
✓

Annual vs monthly breakdown. Most SaaS tools price annually but charge monthly elsewhere. We list both rates side-by-side so the math is clear.
✓

Hidden costs surfaced. Add-ons, overage fees, mandatory minimums, and seat-bundle quirks are listed under each comparison's "Total Cost of Ownership" table.

Independence Safeguards

The Three Firewalls Between Money And Rankings.

We earn revenue through affiliate partnerships and clearly-labeled sponsored content. That's how the site stays free for readers. But three hard firewalls separate the commercial side from the editorial side — and they aren't negotiable.

Scoring Is Locked First.

Every brand's score is finalized — and recorded — before any commercial conversation with that brand. Affiliate teams only see rankings after publication. The math can't bend to a deal that hasn't happened yet.

The Wall Is Staffed.

Editorial and commercial teams report to different leadership. Affiliate negotiators cannot direct editors. If a brand pulls a partnership over a low score, the editor doesn't hear about it until after publication — full stop.

Update Cadence

How Often We Refresh.

Comparison content gets stale fast. Our refresh schedule is built around how quickly each type of information actually changes — so a SaaS pricing tier and a hotel star rating don't get treated the same way.

Every 30 Days

Pricing Verification

All pricing on every page cross-checked against official brand sites. Promo offers, renewal rates, and tier changes captured immediately.

Every 90 Days

Feature Audits

Major features rechecked across each brand's tier structure. New product launches, sunsetted features, and capability changes captured.

Every 6 Months

Re-Scoring Reviews

Category leads re-score brands that have shipped substantial product or pricing changes. Brand rankings shift based on the new math.

Annually

Full Category Rebuild

Each category goes through a full top-to-bottom retest annually — new brands added, old brands re-evaluated, the entire ranking re-issued.

Common Questions

Methodology, Demystified.

Eight questions readers, brands, and journalists ask most about how WhichRanks works. For anything not covered here, our contact page is the right next step.

No. This is the single most-asked question, and the answer is unambiguous: brands cannot pay WhichRanks for better placement. Not the #1 spot, not the top three, not "featured" status, not preferential coverage. Affiliate partnerships exist, and we mark them clearly — but commissions never decide rankings. Our scoring is finalized before any commercial discussion happens, and the editorial team and the commercial team report to different leadership. Read the full editorial policy for the framework.

Sometimes — but with strict rules. For high-cost physical goods like mattresses or broadband equipment, accepting a sample is sometimes the only practical way to test multiple brands. When we do, it's disclosed at the top of the review, and the brand never gets editorial preview. For SaaS, we use real customer accounts on public sign-up flows. For most categories — fashion, food, travel — we buy at retail and don't accept comped products, period.

Several possible reasons. Most commonly: the brand didn't meet our minimum inclusion criteria (typically a customer base threshold, time-in-market threshold, and availability in our coverage regions). Sometimes a brand is too new — we generally wait until a product has been live and reviewed for at least 6 months before scoring it. Occasionally, we just haven't gotten to it yet — our category coverage grows monthly. If you think we're missing a major player, our contact page takes brand suggestions and we read every one.

Two editors independently score each brand from 1–10 on each of the 8 axes (value, performance, ease of use, support, features, design, reliability, trust). Scores are then weighted using the category-specific weighting table — performance counts more for a VPN than for a fashion brand, design counts more for sneakers than for analytics. The weighted average is the final score, rounded to one decimal place. When the two editors disagree by more than 2 points on any axis, a third reviewer adjudicates. See the full rubric in the scoring table above.

Every 30 days, every page. We run automated checks that flag any price quoted from a source older than its 30-day window, and our editorial team manually verifies against the brand's official site monthly. Promotional offers, renewal rates, and tier changes are captured as they happen. Our most-trafficked comparisons — like Shopify vs WooCommerce and NordVPN vs ExpressVPN — get checked more frequently because volatility is higher in those categories.

Yes. When we get a fact wrong — pricing, feature, comparison detail — we correct it openly. The corrected page is re-dated, and a dated correction note appears at the bottom of the page describing what changed. We don't silently update and pretend nothing happened. Readers, journalists, and brands can submit corrections via our contact page, and we typically resolve verified issues within 48 hours.

Comparison content has two failure modes: too shallow to be useful, and too long to be readable. We've settled on a deliberate middle — every review and comparison covers the same eight axes, current pricing, real-world testing notes, and an honest verdict. Roughly 2,000–4,000 words per page, structured so you can skim or read deeply. If you need more depth on a specific brand, our blog publishes longer-form takes weekly.

Three revenue streams, in roughly equal weight: affiliate commissions (we earn a small fee when readers click certain "Get Deal" or "Try Free" buttons — at no cost to you), clearly-labeled sponsored content (sections marked "Ad" or "Sponsored" that look visibly different from editorial), and display advertising in marked locations. Brands can pay for visibility through these mechanisms — but they cannot pay for better rankings. The wall between commercial and editorial is described in the independence safeguards section above.

How We Test.
How We Score.
How We Stay Honest.

Why Our Method Is Public.