AI Deliberator
Guides
How to use AI DeliberatorView all 58 guides

Getting started

Evaluate and export a Council runHow to ask a better decision questionHow to choose models for an AI councilRun your first AI councilUse the Consideration MapContinue a decision with follow-up questionsEstimate and control LLM costs

Everyday decisions

Use an AI council for a career changeEvaluate a leadership role or promotionThink through a relationship decisionDecide whether to move to a new cityCompare degrees, certificates, and education optionsAnalyze a major purchaseUse live evidence mode

Work and teams

Use deliberation for a business strategy decisionSupport a hiring decision responsiblyPrioritize a product roadmapRun a decision pre-mortemMap a stakeholder disagreementAnalyze a policy proposal

Research design

Duplicate and pilot an experimentDesign an LLM deliberation experimentDefine outcomes and dependent variablesConfigure experimental conditions and independent variablesCompare deliberation with homogeneous reconsiderationBuild an independent model ensembleUse randomization, replications, and compute matching

Participant studies

Build a participant interaction studyDesign pre- and post-interaction questionnairesChoose Likert, slider, and questionnaire formatsAdd attention, manipulation, and exposure checksAnalyze and export participant-study resultsBuild a composable experiment timelineMeasure human–AI advice relianceAdd process tracing and cognitive tasksRun a human deliberation conditionBuild a human–AI hybrid deliberation chainRun a text, image, or video stimulus experimentRun a learning and tutoring experimentPlan a powered participant studyMonitor recruitment, cost, and attrition

Behavioral games

Design a behavioral economics gameRun a Dictator GameRun an Ultimatum GameRun a Trust or Investment GameRun a Prisoner’s DilemmaRun a Public Goods GameRun a Stag Hunt GameConfigure game payoffs, incentives, and feedbackManage private game links, consent, and completion

Benchmarks and results

Select benchmark questions and subsetsRun MMLU-Pro in AI DeliberatorRun PolicyBench for public-policy reasoningRun CogToM for social cognitionCreate and read 95% confidence-interval plotsCollaborate and preserve a reproducible experiment recordAnalyze sentiment and quantitative text metricsCompare two Council runs
Workspace

Participant studies

Run a learning and tutoring experiment

Compare solo, Council, human, human–AI, scripted-feedback, and no-tutor conditions using immediate, transfer, and delayed learning outcomes.

The basic idea

A tutoring study needs more than a satisfaction rating. Measure the target knowledge before and immediately after tutoring, use a genuinely new transfer problem, preserve learner explanations, and schedule delayed retention as a later return visit. Keep answer keys out of tutor context and treat personalized versus standardized feedback as an explicit condition difference.

Example

Randomly assign first-year chemistry students to a solo AI tutor, a three-model Council tutor, a human tutor, a human–AI chain, a fixed feedback script, or no tutor; then compare accuracy change, transfer, confidence, mental effort, observed help-seeking, and one-week retention.

How to do it

  1. 1

    Choose Set up learning study in the participant-study builder and replace every placeholder item and answer key.

  2. 2

    State the learning objective and learner level, then add the desired tutoring presets in the Guided Condition Builder.

  3. 3

    Make the immediate and delayed items equivalent without exposing their scoring keys to tutors; add a distinct transfer problem and retain open explanations for blinded coding.

  4. 4

    Set the retention delay and response window, pilot every condition and return link, lock the protocol, and inspect completion, interaction, accuracy, confidence, cost, and delayed-response diagnostics by condition.

What to watch for

  • A no-tutor condition must make no model call; a standardized-feedback condition should display the exact locked script rather than an LLM paraphrase.
  • Self-reported misconception correction and explanation usefulness are not substitutes for objective pre/post accuracy or blinded explanation coding.
  • Delayed-retention attrition can differ by condition, so report invitation and completion rates and prespecify missing-data handling.

What a useful result looks like

The export links condition assignment, tutor transcript, immediate outcomes, process measures, open explanations, and delayed-retention responses without placing answer keys in model prompts.