AI Deliberator
Guides
How to use AI DeliberatorView all 58 guides

Getting started

Evaluate and export a Council runHow to ask a better decision questionHow to choose models for an AI councilRun your first AI councilUse the Consideration MapContinue a decision with follow-up questionsEstimate and control LLM costs

Everyday decisions

Use an AI council for a career changeEvaluate a leadership role or promotionThink through a relationship decisionDecide whether to move to a new cityCompare degrees, certificates, and education optionsAnalyze a major purchaseUse live evidence mode

Work and teams

Use deliberation for a business strategy decisionSupport a hiring decision responsiblyPrioritize a product roadmapRun a decision pre-mortemMap a stakeholder disagreementAnalyze a policy proposal

Research design

Duplicate and pilot an experimentDesign an LLM deliberation experimentDefine outcomes and dependent variablesConfigure experimental conditions and independent variablesCompare deliberation with homogeneous reconsiderationBuild an independent model ensembleUse randomization, replications, and compute matching

Participant studies

Build a participant interaction studyDesign pre- and post-interaction questionnairesChoose Likert, slider, and questionnaire formatsAdd attention, manipulation, and exposure checksAnalyze and export participant-study resultsBuild a composable experiment timelineMeasure human–AI advice relianceAdd process tracing and cognitive tasksRun a human deliberation conditionBuild a human–AI hybrid deliberation chainRun a text, image, or video stimulus experimentRun a learning and tutoring experimentPlan a powered participant studyMonitor recruitment, cost, and attrition

Behavioral games

Design a behavioral economics gameRun a Dictator GameRun an Ultimatum GameRun a Trust or Investment GameRun a Prisoner’s DilemmaRun a Public Goods GameRun a Stag Hunt GameConfigure game payoffs, incentives, and feedbackManage private game links, consent, and completion

Benchmarks and results

Select benchmark questions and subsetsRun MMLU-Pro in AI DeliberatorRun PolicyBench for public-policy reasoningRun CogToM for social cognitionCreate and read 95% confidence-interval plotsCollaborate and preserve a reproducible experiment recordAnalyze sentiment and quantitative text metricsCompare two Council runs
Workspace

Participant studies

Add attention, manipulation, and exposure checks

Distinguish whether participants received, noticed, and understood the assigned interaction without hiding exclusions.

The basic idea

Attention checks assess task engagement, manipulation checks assess whether the intended treatment was perceived, and exposure diagnostics record whether it was actually delivered. These are different signals and should not be collapsed into one pass/fail label.

Example

Record which council messages loaded, ask which interaction format the participant recalls, and include a simple instructed-response item that does not require specialist knowledge.

How to do it

  1. 1

    Define delivery diagnostics from system events before adding self-report questions.

  2. 2

    Add a neutral manipulation check that does not reveal the hypothesis prematurely.

  3. 3

    Use attention checks sparingly and predefine how failures affect primary, sensitivity, and intention-to-treat analyses.

  4. 4

    Export raw check responses and derived flags rather than deleting failed cases.

What to watch for

  • Condition-dependent difficulty can make a check part of the treatment.
  • Post-treatment exclusions can bias an estimated effect.
  • A participant can receive the treatment without correctly naming it, and vice versa.

What a useful result looks like

The study can report assignment, actual exposure, perceived treatment, attention, and analysis inclusion as separate auditable fields.