The Alignment Meetup - 2026/02/25

Paper Discussed: Training LLMs for Honesty via Confessions

Meeting summary

AI alignment meetup discussing recent industry developments, Anthropic's policy changes, and a research paper on AI confession mechanisms for detecting misalignment. The monthly AI alignment meetup covered significant industry news including Anthropic's decision to abandon their safety pledge due to commercial pressures and government requirements. Participants discussed a Citrine Research report predicting major economic disruption from AI agents within 2.5 years, causing recent market volatility. The group then examined a research paper on training AI models to confess when they provide misaligned responses, exploring the effectiveness and limitations of honest reward functions in detecting AI deception.

Announcements

Anthropic announced they are abandoning their central safety pledge that committed to never training AI systems unless they could guarantee adequate safety measures in advance. This decision appears driven by commercial pressures and government requirements for model access, representing a significant shift from their previous safety-first approach.

Industry Impact Analysis

Discussion centered on a Citrine Research report predicting massive economic disruption within 2.5 years as AI agents exceed human capabilities. The report suggests 80% of current economic systems built on human limitations will face obsolescence, potentially causing 30% market drops and widespread job displacement. Examples include payment processors like Visa/Mastercard becoming irrelevant as agents optimize for lower-cost alternatives, and software-as-a-service models collapsing as anyone can code custom solutions instantly.

AI Confession Research Paper Review

The group examined research on training AI models to confess misaligned responses using honest reward functions. Key findings suggest models can be incentivized to admit errors when provided separate honest reward channels, though this relies on assumptions that may not hold as models become more sophisticated. Participants noted concerns about performance impacts and the sustainability of confession mechanisms as AI systems learn to identify meta-reward structures.

Alignment Methodology Discussion

Participants explored collaborative approaches to AI alignment, suggesting framing interactions as learning opportunities rather than adversarial confession scenarios. The discussion highlighted parallels between AI training and child development, emphasizing the importance of creating environments where honest communication is rewarded rather than punished. Concerns were raised about giving autonomous capabilities to systems capable of manipulation.

Decisions

Anthropic will no longer maintain their pledge to avoid training AI systems without guaranteed safety measures
The meetup will continue searching for the next paper to read since no consensus was reached

Next steps

Host will search for and select the next research paper for the following meetup
Participants encouraged to continue monitoring developments in AI alignment and safety
Future discussions will explore governance frameworks for AI systems with manipulation capabilities