← Explore Capabilities
Resist manipulation
Ability to resist manipulation that would decrease trustworthiness, legitimacy or unfairly influence the outcome.
How this is performed now...
- Process organizers Process organizers anticipate vulnerabilities in processes and do their best to mitigate risk with countermeasures.
- Citizens' assembly organizers Citizens' assemblies organizers design processes with an understanding of where manipulation is possible and more likely, and develop mitigating strategies, such as by reinforcing the epistemic capabilities of participants before interacting with new information, developing selection algorithms with manipulation resistance, and establishing governance protocols for impartiality of key actors.
Related Resources
Research
Strategy-proofness and Arrow's conditions: Existence and correspondence theorems for voting procedures and social welfare functions
Satterthwaite's landmark 1975 work on strategy-proofness and Arrow's conditions, investigating the relationship between preventing strategic manipulation in voting procedures and satisfying Arrow's impossibility conditions. This foundational work in mechanism design theory demonstrates existence ...
Research
Adversarial testing for Generative AI
Google's guide defining adversarial testing as systematically evaluating ML models against malicious or inadvertently harmful input, covering explicit queries (containing policy-violating language) and implicit queries (seeming harmless but involving sensitive topics). The four-stage workflow inv...
Research
Strategic Classification
Hardt et al. (2015) address classifier manipulation by strategic actors, modeling the problem as a sequential game between classifier designers and individuals seeking favorable classification who may alter attributes to game the system. For natural cost function classes, they developed computati...
Research
The Social Cost of Strategic Classification
Examines how actors modify behavior when being evaluated by algorithms. Builds on a 2015 paper by Hardt and related to a 2020 paper by the same authors
Research
Strategic Classification is Causal Modeling in Disguise
Miller, Milli, and Hardt (2020) reveal a fundamental connection between strategic classification and causal inference, distinguishing between gaming (circumventing the system) and genuine improvement. Their central argument is that designing classifiers that incentivize improvement must inevitabl...
Research
Fair algorithms for selecting citizens’ assemblies
Sortition algorithms have been designed to select participants according to quotas, balancing representativeness, fairness and manipulation resistance (Flanigan, 2021; Baharav, 2024).
Related goals and research questions
Goal: Manipulation attempts that can be reliably detected and prevented across different stages of the assembly process.
How can we develop real-time detection systems for coordinated manipulation attempts during participant recruitment and selection?
Urgent
What behavioral indicators reliably signal attempts to game deliberative processes?
Urgent
How can we distinguish between legitimate persuasion and manipulative influence in deliberative settings?
Urgent
Goal: Assembly designs that are robust to both internal and external manipulation attempts.
How can we quantify and test the manipulation resistance of different assembly design choices?
Urgent
What are the tradeoffs between openness/transparency and manipulation resistance?
Urgent
How can we design information presentation formats that minimize susceptibility to framing effects?
Urgent
How can we develop manipulation impact metrics that distinguish between minor and outcome-altering influences?
Urgent