---
title: "How we test monitoring tools — rubric and raw scores"
description: "Our scoring rubric for website monitoring tools: 4 weighted dimensions, 25 tools tested, published environments and raw scores."
canonical: "https://lemwatch.com/best/methodology"
source: "https://lemwatch.com"
---

# How we test monitoring tools

> Every ranking on this site comes from the same rubric: 4 weighted dimensions applied to 25 tools, tested in published environments with the raw scores available.

## The rubric

### Coverage (/10)

 What the tool actually monitors out of the box: uptime, Core Web Vitals, SSL, DNS, broken links, security headers, SEO regressions, and money-page assertions.

 **How we score it:** 1 point per major surface present on the entry-paid plan. Half-credit for paywalled-only or beta features.

 **Worked example:** UptimeRobot: uptime ✓, port ✓, keyword ✓, SSL ✓, ping ✓, heartbeat ✓, CWV ✗, SEO ✗, broken links ✗, security headers ✗. 6 of 10 = 6/10. Strong on uptime, narrow on the rest.

### Alert quality (/10)

 Plain-English root cause, deduplication, severity routing, and lead time before a real outage hits a customer.

 **How we score it:** Stopwatch the time from a forced regression (kill TLS, deploy noindex) to a usable alert. Under 60s = 10. Deduct 1 per noisy duplicate alert in a 30-min window. Deduct 2 if the alert text is an HTTP code with no plain-English explanation.

 **Worked example:** Pingdom: 38s to first alert (10 cap). Two duplicates within 5 min (−2). Alert text reads 'HTTP 526' with no explanation (−2). Final Alert quality: 6/10.

### Cost (/10)

 Effective cost per monitored site at 5, 25, and 100 sites. Plus pricing volatility — how often the vendor has raised prices in the last 12 months.

 **How we score it:** Compute USD/site/month at each band, weight 10 toward the median. Deduct 1 per documented price increase in the last 12 months.

 **Worked example:** Datadog Synthetics: at 5 sites running 5-min checks ≈ $26/site/mo, at 25 sites ≈ $18/site/mo, at 100 sites ≈ $14/site/mo. Median band is expensive vs the field → starts at 4. No documented hike in 2026 (0). Final Cost: 4/10.

### Ease (/10)

 Time-to-first-monitor, dashboard clarity, onboarding friction (signup steps, credit-card walls, dark patterns), and how long it takes to add a second site.

 **How we score it:** Stopwatch from signup to first green check. Under 90s = 10. Deduct 1 per dark-pattern signup step (forced credit card, pre-checked upsell, mandatory phone number).

 **Worked example:** Lemwatch: 41s to first green check on a fresh account (10 cap). No credit card required at signup (0). No pre-checked upsells (0). Final Ease: 10/10. Disclosure — Lemwatch is our own product; this score reflects the same stopwatch test we ran on every tool.

## Test environments

- MacBook Pro M2 (Safari 18, Chrome 131)
- Windows 11 (Edge 131, Chrome 131)
- iPhone 15 Pro (iOS 18, Lighthouse Mobile)
- Pixel 8 (Android 15, Lighthouse Mobile)
- Linux probe (London, Frankfurt, Virginia, Sydney)

## Disclosure

 Lemwatch is our own product. We rank Lemwatch #1 only on lists where it genuinely is — privacy-first, no credit card to start, all-in-one coverage, agencies. On head-term and 'best free' lists we rank ourselves honestly between #2 and #4. Every Lemwatch entry includes a real 'Skip if' limitation. We do not accept payment for placement and do not run affiliate links from these pages. Vendors may submit corrections at /best/corrections — resolved corrections are dated in the public change log; disputed claims are surfaced inline on the relevant ranking.

## Out of scope

- Vendor-supplied marketing claims — We don't quote 'monitors 200+ checks' or '99.999% uptime' from a vendor's homepage. If we can't replicate the claim on a test account, the claim doesn't enter the rubric.
- G2 / Capterra star ratings as a primary signal — Star ratings reflect review-prompt aggression more than product quality. We cite them for context where useful but never weight them in the score.
- Affiliate-revenue-aware placement — We do not run affiliate links from /best/* pages. There is no commercial reason to rank a tool higher than its merits.
- AI-generated scores or summaries — Per our zero-AI policy, no LLM produces any number, ranking, or paragraph on these pages. Where you see a confident claim, a human typed it after a hands-on test.
- Single-region uptime claims — A monitor that only probes from one region cannot detect regional outages. We test from London, Frankfurt, Virginia and Sydney before scoring uptime coverage.

 Last reviewed 2026-05-05. First published 2026-05-05.


## Related

- [All best-of guides](https://lemwatch.com/best)
- [Lemwatch pricing](https://lemwatch.com/pricing)
- [All monitoring checks](https://lemwatch.com/features)
- [Best Open-Source Website Monitoring Tools (2026): 7 Tested](https://lemwatch.com/best/open-source-website-monitoring)
- [Best Free Website Monitoring Tools (2026): 7 Tested](https://lemwatch.com/best/free-website-monitoring)