AI Deliberator
Guides
How to use AI DeliberatorView all 58 guides

Getting started

Evaluate and export a Council runHow to ask a better decision questionHow to choose models for an AI councilRun your first AI councilUse the Consideration MapContinue a decision with follow-up questionsEstimate and control LLM costs

Everyday decisions

Use an AI council for a career changeEvaluate a leadership role or promotionThink through a relationship decisionDecide whether to move to a new cityCompare degrees, certificates, and education optionsAnalyze a major purchaseUse live evidence mode

Work and teams

Use deliberation for a business strategy decisionSupport a hiring decision responsiblyPrioritize a product roadmapRun a decision pre-mortemMap a stakeholder disagreementAnalyze a policy proposal

Research design

Duplicate and pilot an experimentDesign an LLM deliberation experimentDefine outcomes and dependent variablesConfigure experimental conditions and independent variablesCompare deliberation with homogeneous reconsiderationBuild an independent model ensembleUse randomization, replications, and compute matching

Participant studies

Build a participant interaction studyDesign pre- and post-interaction questionnairesChoose Likert, slider, and questionnaire formatsAdd attention, manipulation, and exposure checksAnalyze and export participant-study resultsBuild a composable experiment timelineMeasure human–AI advice relianceAdd process tracing and cognitive tasksRun a human deliberation conditionBuild a human–AI hybrid deliberation chainRun a text, image, or video stimulus experimentRun a learning and tutoring experimentPlan a powered participant studyMonitor recruitment, cost, and attrition

Behavioral games

Design a behavioral economics gameRun a Dictator GameRun an Ultimatum GameRun a Trust or Investment GameRun a Prisoner’s DilemmaRun a Public Goods GameRun a Stag Hunt GameConfigure game payoffs, incentives, and feedbackManage private game links, consent, and completion

Benchmarks and results

Select benchmark questions and subsetsRun MMLU-Pro in AI DeliberatorRun PolicyBench for public-policy reasoningRun CogToM for social cognitionCreate and read 95% confidence-interval plotsCollaborate and preserve a reproducible experiment recordAnalyze sentiment and quantitative text metricsCompare two Council runs
Workspace

Participant studies

Design pre- and post-interaction questionnaires

Measure baseline beliefs and later outcomes without losing the transcript or confusing participants at the transition.

The basic idea

Pre/post measurement estimates change more directly than a post-only score, but asking a question before treatment can itself influence later responses. Choose timing deliberately, keep wording stable when measuring change, and preserve the interaction before moving to the questionnaire.

Example

Ask policy support and confidence before the AI interaction, preserve the full discussion, display an explicit transition, then repeat the outcomes and add treatment-experience questions.

How to do it

  1. 1

    Define which outcomes are measured before, after, or at both times.

  2. 2

    Use identical wording and scales for paired pre/post outcomes unless a planned reason requires a change.

  3. 3

    Mark sensitive or nonessential items optional and order questions from essential outcomes to secondary measures.

  4. 4

    Preview the transition and both questionnaire stages as a participant before launch.

What to watch for

  • Pretesting can prime the topic or treatment mechanism.
  • Changing labels or scale direction between waves can create artificial change.
  • Do not require sensitive questions unless the study genuinely needs them and consent covers them.

What a useful result looks like

The dataset contains clearly timed, analyzable responses tied to the same participant and interaction while the user sees an understandable progression through the study.