Methodology
How scores are derived. The full mathematical model is published here so that any stakeholder — rater, organization, or researcher — can verify the process independently.
Charter
- No ads, no data sales, and no paid product ever moves a public score.
- Public score pages do not display raw response rows, written notes, comments, or rater identities.
- No leaderboards — comparison is against an expected range, not a rank.
- Built to help systems improve, not to farm traffic.
- Rate My Systems is an instrument of The Solar Guild. Ratings are the beginning, not the end.
Instrument
Each rating session presents 8 core dimensions (always asked) and 5 depth dimensions (opt-in).
Core dimensions are asked every rating. Depth dimensions are opt-in — a rater can go deeper or skip them.
Responses use a 5-point frequency scale: Never → Rarely → Sometimes → Usually → Always.
Positive Polarity
All items are worded so that “Always” = healthy system. Scores add coherently into the composite index.
8 Core Dimensions
5 Depth Dimensions
Opt-in after the core. Each is N/A-friendly.
Alignment Firewall
The Alignment dimension grades whether a system's own incentives match its own stated purpose — not whether the rater agrees with that purpose.
Complexity & Friction Correlation
Complexityis tracked but may be dropped from the composite index if it correlates strongly (>0.8) with Friction in live data.
N/A Handling
“Doesn't apply” answers are excluded from scoring — they reduce the number of data points for that dimension but never count as a low or zero score.
A minimum of 5 answered core dimensions (non-N/A) is required for a rating to count.
Scoring
Identity Weighting
The current model has two active weights. Anonymous ratings count; signed-in ratings count twice as much because account identity gives the system a stronger authenticity signal. Worker, guild, and ID-verification multipliers are not active until those systems exist.
| Identity | Multiplier |
|---|---|
| Anonymous | 0.5× |
| Signed in | 1× |
Recency Decay
Half-life: 270 days. A rating from 270 days ago contributes half the weight of one submitted today.
Share Cap
No single rater may account for more than 35% of a scope's total weighted mass.
Bayesian Shrinkage
Per-subject scores are pulled toward the global mean using k = 8 pseudo-ratings. This prevents small-sample scores from swinging wildly.
Index Scale
Raw category means (1–5) are mapped to a 0–100 index.
Expected Range
The “expected range” is mean ± 2σ of all subject index scores. A subject is flagged as signal only if outside these limits. This replaces rankings.
Privacy & Publication
Publication begins with the first accepted rating. A small-sample result is labeled as an early signal and should be treated as directional until more independent raters contribute.
With one contributor, a subject-level aggregate is mathematically equivalent to that contribution even though the contributor is not named. Raw answers, per-question notes, comments, account details, and device identifiers are not displayed on public score pages.
This page publishes the math and scoring dimensions. Full question text is issued only inside a rating session.
Use of Aggregate Signal
Aggregate and de-identified results may be used to identify patterns, guide research or outreach, support future stewardship work, and inform public analysis about how systems behave.
Raw response rows, written comments, per-question notes, and rater identities are not displayed on public score pages.