In bilingual interfaces that display explicit reliability asymmetry statements (e.g., ‘English answers are usually more complete than in X language’), how does adding per-topic reliability labels (such as separate badges for health, law, and everyday topics) change miscalibrated reliance gaps and perceived fairness, compared with a single global language-level label?
cross-lingual-cot-trust | Updated at
Answer
Adding per-topic reliability labels tends to make reliance patterns more domain-calibrated and can improve perceived fairness for users whose high-stakes use is concentrated in specific topics, but it also sharpens visible disparities and can increase perceived unfairness if topic labels are very negative for a user’s main domain.
Compared with a single global language-level label:
-
Miscalibrated reliance gaps
- Gaps generally shrink within high-risk topics (e.g., health, law), because users get more precise signals about where the weaker language is especially unreliable and adjust reliance by domain rather than applying one coarse global rule.
- Overall cross-language gaps become more structured: users are more likely to keep using the weaker language for low-risk / everyday topics while preferentially routing health and law queries to the safer language, which is closer to the true reliability profile.
- However, the more labels you expose, the more some users may disengage from nuanced calibration and instead adopt a simple heuristic (“just always use English for anything important”), which can widen practical reliance gaps in favor of the strong language even where differences are small.
-
Perceived fairness
- Per-topic labels are often judged fairer in principle because they avoid painting a whole language as uniformly worse; they acknowledge that reliability varies by domain and that some topics in the weaker language may be adequate or strong.
- Users whose critical use-cases lie in domains explicitly marked as weaker (e.g., "Health in X is limited-tested") may feel more stigmatized or disadvantaged than under a single softer global label, because the system is now labeling their key needs as second-class.
- Careful framing (monitoring-based, improvable, user-agency oriented) becomes more important: without it, a matrix of low badges for “Health in X” and “Law in X” can look like institutionalized bias even when empirically accurate.
Net effect: per-topic reliability labels usually improve calibration by domain and can make fairness feel more principled, but they also risk larger salient gaps and stronger perceptions of inequality unless paired with respectful wording and clear routes for users in the weaker language to access safer channels for their high-risk topics.