AI Deliberator
Guides
How to use AI DeliberatorView all 58 guides

Getting started

Evaluate and export a Council runHow to ask a better decision questionHow to choose models for an AI councilRun your first AI councilUse the Consideration MapContinue a decision with follow-up questionsEstimate and control LLM costs

Everyday decisions

Use an AI council for a career changeEvaluate a leadership role or promotionThink through a relationship decisionDecide whether to move to a new cityCompare degrees, certificates, and education optionsAnalyze a major purchaseUse live evidence mode

Work and teams

Use deliberation for a business strategy decisionSupport a hiring decision responsiblyPrioritize a product roadmapRun a decision pre-mortemMap a stakeholder disagreementAnalyze a policy proposal

Research design

Duplicate and pilot an experimentDesign an LLM deliberation experimentDefine outcomes and dependent variablesConfigure experimental conditions and independent variablesCompare deliberation with homogeneous reconsiderationBuild an independent model ensembleUse randomization, replications, and compute matching

Participant studies

Build a participant interaction studyDesign pre- and post-interaction questionnairesChoose Likert, slider, and questionnaire formatsAdd attention, manipulation, and exposure checksAnalyze and export participant-study resultsBuild a composable experiment timelineMeasure human–AI advice relianceAdd process tracing and cognitive tasksRun a human deliberation conditionBuild a human–AI hybrid deliberation chainRun a text, image, or video stimulus experimentRun a learning and tutoring experimentPlan a powered participant studyMonitor recruitment, cost, and attrition

Behavioral games

Design a behavioral economics gameRun a Dictator GameRun an Ultimatum GameRun a Trust or Investment GameRun a Prisoner’s DilemmaRun a Public Goods GameRun a Stag Hunt GameConfigure game payoffs, incentives, and feedbackManage private game links, consent, and completion

Benchmarks and results

Select benchmark questions and subsetsRun MMLU-Pro in AI DeliberatorRun PolicyBench for public-policy reasoningRun CogToM for social cognitionCreate and read 95% confidence-interval plotsCollaborate and preserve a reproducible experiment recordAnalyze sentiment and quantitative text metricsCompare two Council runs
Workspace

Benchmarks and results

Analyze sentiment and quantitative text metrics

Compare human and model language using automatic VADER sentiment and transparent descriptive counts.

The basic idea

Automatic text features are reproducible descriptive outcomes. VADER compound and positive, neutral, and negative proportions characterize English lexical tone; word, sentence, and lexical-diversity measures characterize form. They do not by themselves measure factual accuracy, argument quality, intent, or a person's emotional state.

Example

Compare the mean VADER compound score, word count, sentence length, and lexical diversity of human and model contributions across solo-AI, council, hybrid, and human-only conditions.

How to do it

  1. 1

    Open a completed run or participant-study analysis to view automatic text measures by source and condition.

  2. 2

    Use the version field and the same unit of analysis in every arm.

  3. 3

    Export the metrics alongside transcripts or game moves, then prespecify whether inference is at the message, participant, item, session, or condition level.

  4. 4

    Inspect distributions and clustered observations before testing condition differences.

What to watch for

  • VADER is designed for English and may misread sarcasm, negation in specialist language, slang, and translated text.
  • Longer responses can mechanically change counts and lexical diversity.
  • Repeated turns from one participant or benchmark item are not independent observations.

What a useful result looks like

Every supported experiment format exposes versioned, zero-API-cost sentiment and text-form measures while retaining the underlying text for audit and alternative coding.