Devlog Feb 19: MoLOS-LLM-Council Development

This week we focused on the development of the MoLOS-LLM-Council module, implementing the core 3-stage AI consultation process, persona-based architecture, multi-provider support, and comprehensive UI components.

3-Stage AI Consultation Process

The heart of MoLOS-LLM-Council is its 3-stage consultation process, which ensures thorough exploration of any topic from multiple perspectives.

CouncilOrchestrator

We implemented the CouncilOrchestrator class to manage the consultation stages:

export class CouncilOrchestrator {
  constructor(
    private client: OpenRouterClient,
    private personas: Persona[],
    private prompts: CouncilPrompts
  ) {}

  async consult(question: string): Promise<CouncilResult> {
    // Stage 1: Individual Perspectives
    const perspectives = await this.stage1IndividualPerspectives(question);

    // Stage 2: Deliberation
    const deliberation = await this.stage2Deliberation(question, perspectives);

    // Stage 3: Synthesis
    const synthesis = await this.stage3Synthesis(question, perspectives, deliberation);

    return {
      question,
      perspectives,
      deliberation,
      synthesis,
      timestamp: Date.now()
    };
  }

  private async stage1IndividualPerspectives(question: string): Promise<Perspective[]> {
    const perspectives: Perspective[] = [];

    for (const persona of this.personas) {
      const prompt = this.prompts.stage1(question, persona);
      const response = await this.client.chat({
        model: persona.modelId,
        messages: [{ role: 'user', content: prompt }]
      });

      perspectives.push({
        personaId: persona.id,
        personaName: persona.name,
        content: response.content,
        timestamp: Date.now()
      });
    }

    return perspectives;
  }

  private async stage2Deliberation(
    question: string,
    perspectives: Perspective[]
  ): Promise<Deliberation> {
    const prompt = this.prompts.stage2(question, perspectives);
    const response = await this.client.chat({
      model: 'gpt-4',
      messages: [{ role: 'user', content: prompt }]
    });

    return {
      content: response.content,
      timestamp: Date.now()
    };
  }

  private async stage3Synthesis(
    question: string,
    perspectives: Perspective[],
    deliberation: Deliberation
  ): Promise<Synthesis> {
    const prompt = this.prompts.stage3(question, perspectives, deliberation);
    const response = await this.client.chat({
      model: 'gpt-4',
      messages: [{ role: 'user', content: prompt }]
    });

    return {
      content: response.content,
      timestamp: Date.now()
    };
  }
}

Stage Prompts

We created specialized prompts for each stage of the consultation:

export class CouncilPrompts {
  stage1(question: string, persona: Persona): string {
    return `You are ${persona.name}, ${persona.description}.

You are part of a council of experts discussing the following question:

"${question}"

Please provide your individual perspective on this question, focusing on your area of expertise. Be thorough and specific. Consider:
- Key factors relevant to your expertise
- Potential approaches or solutions
- Risks or concerns
- Your unique viewpoint

Provide your response in a clear, well-structured manner.`;
  }

  stage2(question: string, perspectives: Perspective[]): string {
    const perspectivesText = perspectives.map(p =>
      `### ${p.personaName}\n\n${p.content}`
    ).join('\n\n---\n\n');

    return `You are facilitating a deliberation among experts discussing:

"${question}"

Here are the individual perspectives:

${perspectivesText}

Your task is to facilitate a deliberation by:
1. Identifying common ground and agreements
2. Highlighting key disagreements or conflicts
3. Exploring where perspectives complement each other
4. Suggesting areas where more discussion is needed
5. Proposing ways to reconcile different viewpoints

Provide a structured deliberation that brings together these diverse perspectives.`;
  }

  stage3(
    question: string,
    perspectives: Perspective[],
    deliberation: Deliberation
  ): string {
    return `You are synthesizing a comprehensive answer to:

"${question}"

You have access to:
1. Individual perspectives from multiple experts
2. A structured deliberation among them

Your task is to create a synthesis that:
1. Captures the key insights from all perspectives
2. Addresses the deliberation points
3. Provides a clear, actionable answer
4. Identifies areas of consensus and disagreement
5. Recommends next steps or actions

Organize your synthesis with clear headings and actionable recommendations.`;
  }
}

Persona-Based Architecture

We migrated from a model-based to a persona-based architecture, significantly improving flexibility and context-aware consultations.

Database Schema

The new schema supports persona-provider relationships:

export const personas = sqliteTable('council_personas', {
  id: text('id').primaryKey(),
  name: text('name').notNull(),
  description: text('description').notNull(),
  expertise: text('expertise', { mode: 'json' }).$type<string[]>(),
  personality: text('personality', { mode: 'json' }).$type<Record<string, string>>(),
  providerId: text('provider_id').notNull().references(() => providers.id),
  modelId: text('model_id').notNull(),
  isSystem: integer('is_system', { mode: 'boolean' }).notNull().default(false),
  userId: text('user_id'),
  createdAt: integer('created_at').notNull(),
  updatedAt: integer('updated_at').notNull()
});

export const providers = sqliteTable('council_providers', {
  id: text('id').primaryKey(),
  name: text('name').notNull(),
  type: text('type').notNull(), // 'openai', 'anthropic', 'openrouter', 'zai', 'custom'
  apiKey: text('api_key').notNull(),
  baseUrl: text('base_url'),
  models: text('models', { mode: 'json' }).$type<Model[]>(),
  createdAt: integer('created_at').notNull(),
  updatedAt: integer('updated_at').notNull()
});

export const conversations = sqliteTable('council_conversations', {
  id: text('id').primaryKey(),
  title: text('title').notNull(),
  question: text('question').notNull(),
  personaIds: text('persona_ids', { mode: 'json' }).$type<string[]>(),
  userId: text('user_id').notNull(),
  createdAt: integer('created_at').notNull(),
  updatedAt: integer('updated_at').notNull()
});

export const messages = sqliteTable('council_messages', {
  id: text('id').primaryKey(),
  conversationId: text('conversation_id').notNull().references(() => conversations.id),
  personaId: text('persona_id'),
  stage: text('stage').notNull(), // 'perspective', 'deliberation', 'synthesis'
  content: text('content').notNull(),
  userId: text('user_id').notNull(),
  createdAt: integer('created_at').notNull()
});

Persona Repository

export class PersonaRepository {
  async create(data: CreatePersonaDTO): Promise<Persona> {
    const [persona] = await this.db
      .insert(personas)
      .values({
        id: generateId(),
        name: data.name,
        description: data.description,
        expertise: data.expertise || [],
        personality: data.personality || {},
        providerId: data.providerId,
        modelId: data.modelId,
        isSystem: false,
        userId: data.userId,
        createdAt: Date.now(),
        updatedAt: Date.now()
      })
      .returning();

    return persona;
  }

  async findById(id: string): Promise<Persona | null> {
    const result = await this.db
      .select()
      .from(personas)
      .where(eq(personas.id, id))
      .limit(1);

    return result[0] || null;
  }

  async findByUser(userId: string): Promise<Persona[]> {
    return await this.db
      .select()
      .from(personas)
      .where(eq(personas.userId, userId))
      .orderBy(asc(personas.name));
  }

  async findSystemPersonas(): Promise<Persona[]> {
    return await this.db
      .select()
      .from(personas)
      .where(eq(personas.isSystem, true))
      .orderBy(asc(personas.name));
  }

  async update(id: string, data: UpdatePersonaDTO): Promise<Persona> {
    const [persona] = await this.db
      .update(personas)
      .set({
        ...data,
        updatedAt: Date.now()
      })
      .where(eq(personas.id, id))
      .returning();

    return persona;
  }

  async delete(id: string): Promise<void> {
    await this.db.delete(personas).where(eq(personas.id, id));
  }
}

Multi-Provider Support

We implemented support for multiple AI providers through a unified client interface:

Base Provider Client

export abstract class BaseProviderClient {
  protected apiKey: string;
  protected baseUrl: string;

  constructor(config: ProviderConfig) {
    this.apiKey = config.apiKey;
    this.baseUrl = config.baseUrl || this.getDefaultBaseUrl();
  }

  abstract getDefaultBaseUrl(): string;
  abstract chat(request: ChatRequest): Promise<ChatResponse>;

  protected async makeRequest(endpoint: string, body: any): Promise<any> {
    const response = await fetch(`${this.baseUrl}${endpoint}`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${this.apiKey}`
      },
      body: JSON.stringify(body)
    });

    if (!response.ok) {
      throw new Error(`Provider request failed: ${response.statusText}`);
    }

    return response.json();
  }
}

OpenAI Client

export class OpenAIClient extends BaseProviderClient {
  getDefaultBaseUrl(): string {
    return 'https://api.openai.com/v1';
  }

  async chat(request: ChatRequest): Promise<ChatResponse> {
    const body = {
      model: request.model,
      messages: request.messages,
      stream: request.stream || false
    };

    if (request.stream) {
      return this.streamChat(body);
    } else {
      return this.nonStreamChat(body);
    }
  }

  private async nonStreamChat(body: any): Promise<ChatResponse> {
    const response = await this.makeRequest('/chat/completions', body);
    return {
      content: response.choices[0].message.content,
      usage: response.usage
    };
  }

  private async *streamChat(body: any): AsyncGenerator<string, void, unknown> {
    const response = await fetch(`${this.baseUrl}/chat/completions`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${this.apiKey}`
      },
      body: JSON.stringify({ ...body, stream: true })
    });

    const reader = response.body?.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader!.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const lines = chunk.split('\n').filter(line => line.trim().startsWith('data:'));

      for (const line of lines) {
        const data = JSON.parse(line.replace('data:', '').trim());
        if (data.choices && data.choices[0]?.delta?.content) {
          yield data.choices[0].delta.content;
        }
      }
    }
  }
}

Anthropic Client

export class AnthropicClient extends BaseProviderClient {
  getDefaultBaseUrl(): string {
    return 'https://api.anthropic.com/v1';
  }

  async chat(request: ChatRequest): Promise<ChatResponse> {
    const response = await this.makeRequest('/messages', {
      model: request.model,
      max_tokens: 4096,
      messages: request.messages.map(msg => ({
        role: msg.role,
        content: msg.content
      }))
    });

    return {
      content: response.content[0].text,
      usage: response.usage
    };
  }
}

Provider Factory

export class ProviderFactory {
  static createClient(provider: Provider): BaseProviderClient {
    switch (provider.type) {
      case 'openai':
        return new OpenAIClient(provider);
      case 'anthropic':
        return new AnthropicClient(provider);
      case 'openrouter':
        return new OpenRouterClient(provider);
      case 'zai':
        return new ZAIClient(provider);
      case 'custom':
        return new CustomClient(provider);
      default:
        throw new Error(`Unknown provider type: ${provider.type}`);
    }
  }
}

Conversation and Message Repositories

Conversation Repository

export class ConversationRepository {
  async create(data: CreateConversationDTO): Promise<Conversation> {
    const [conversation] = await this.db
      .insert(conversations)
      .values({
        id: generateId(),
        title: data.title,
        question: data.question,
        personaIds: data.personaIds,
        userId: data.userId,
        createdAt: Date.now(),
        updatedAt: Date.now()
      })
      .returning();

    return conversation;
  }

  async findById(id: string): Promise<Conversation | null> {
    const result = await this.db
      .select()
      .from(conversations)
      .where(eq(conversations.id, id))
      .limit(1);

    return result[0] || null;
  }

  async findByUser(userId: string, limit = 20): Promise<Conversation[]> {
    return await this.db
      .select()
      .from(conversations)
      .where(eq(conversations.userId, userId))
      .orderBy(desc(conversations.createdAt))
      .limit(limit);
  }

  async search(query: string, userId: string): Promise<Conversation[]> {
    const results = await this.db
      .select()
      .from(conversations)
      .where(
        and(
          eq(conversations.userId, userId),
          or(
            like(conversations.title, `%${query}%`),
            like(conversations.question, `%${query}%`)
          )
        )
      )
      .orderBy(desc(conversations.createdAt));

    return results;
  }
}

Message Repository

export class MessageRepository {
  async create(data: CreateMessageDTO): Promise<Message> {
    const [message] = await this.db
      .insert(messages)
      .values({
        id: generateId(),
        conversationId: data.conversationId,
        personaId: data.personaId,
        stage: data.stage,
        content: data.content,
        userId: data.userId,
        createdAt: Date.now()
      })
      .returning();

    return message;
  }

  async findByConversation(conversationId: string): Promise<Message[]> {
    return await this.db
      .select()
      .from(messages)
      .where(eq(messages.conversationId, conversationId))
      .orderBy(asc(messages.createdAt));
  }

  async findByStage(conversationId: string, stage: string): Promise<Message[]> {
    return await this.db
      .select()
      .from(messages)
      .where(
        and(
          eq(messages.conversationId, conversationId),
          eq(messages.stage, stage)
        )
      )
      .orderBy(asc(messages.createdAt));
  }
}

API Endpoints

Start Consultation

// POST /api/start
export async function POST({ request, locals }) {
  const { question, personaIds, providerId } = await request.json();

  // Get personas
  const personas = await Promise.all(
    personaIds.map(id => personaRepository.findById(id))
  );

  // Get provider
  const provider = await providerRepository.findById(providerId);
  const client = ProviderFactory.createClient(provider);

  // Run consultation
  const orchestrator = new CouncilOrchestrator(client, personas, new CouncilPrompts());
  const result = await orchestrator.consult(question);

  // Create conversation
  const conversation = await conversationRepository.create({
    title: question.substring(0, 100),
    question,
    personaIds,
    userId: locals.user.id
  });

  // Save messages
  for (const perspective of result.perspectives) {
    await messageRepository.create({
      conversationId: conversation.id,
      personaId: perspective.personaId,
      stage: 'perspective',
      content: perspective.content,
      userId: locals.user.id
    });
  }

  await messageRepository.create({
    conversationId: conversation.id,
    stage: 'deliberation',
    content: result.deliberation.content,
    userId: locals.user.id
  });

  await messageRepository.create({
    conversationId: conversation.id,
    stage: 'synthesis',
    content: result.synthesis.content,
    userId: locals.user.id
  });

  return json({
    conversation,
    result
  });
}

Stream Responses

// GET /api/stream/[conversationId]
export async function GET({ params }) {
  const conversation = await conversationRepository.findById(params.conversationId);
  if (!conversation) {
    throw error(404, 'Conversation not found');
  }

  const messages = await messageRepository.findByConversation(params.conversationId);
  const personas = await Promise.all(
    conversation.personaIds.map(id => personaRepository.findById(id))
  );

  const stream = new ReadableStream({
    async start(controller) {
      for (const message of messages) {
        const persona = personas.find(p => p?.id === message.personaId);

        controller.enqueue({
          type: 'message',
          stage: message.stage,
          personaName: persona?.name,
          content: message.content,
          timestamp: message.createdAt
        });

        // Simulate streaming by sending in chunks
        await new Promise(resolve => setTimeout(resolve, 100));
      }

      controller.close();
    }
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive'
    }
  });
}

UI Implementation

Council Page

<!-- src/routes/ui/+page.svelte -->
<script>
  import { onMount } from 'svelte';
  import { councilStore } from '$stores/council.store';

  let question = '';
  let selectedPersonas = [];
  let selectedProvider = '';
  let loading = false;
  let currentStage = null;
  let consultation = null;

  async function startConsultation() {
    loading = true;
    try {
      const response = await fetch('/api/start', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          question,
          personaIds: selectedPersonas.map(p => p.id),
          providerId: selectedProvider
        })
      });

      consultation = await response.json();
      currentStage = 'perspective';
    } finally {
      loading = false;
    }
  }
</script>

<div class="council-container">
  <input
    type="text"
    bind:value={question}
    placeholder="What would you like to consult the council about?"
  />

  <PersonaSelector
    bind:selected={selectedPersonas}
  />

  <ProviderSelector
    bind:selected={selectedProvider}
  />

  <button on:click={startConsultation} disabled={loading || !question}>
    {loading ? 'Consulting...' : 'Start Consultation'}
  </button>

  {#if consultation}
    <ConsultationResult result={consultation} />
  {/if}
</div>

Stage Panels

<!-- src/components/stage1-panel.svelte -->
<script>
  export let perspectives;

  let currentPersona = 0;

  function nextPersona() {
    if (currentPersona < perspectives.length - 1) {
      currentPersona++;
    }
  }

  function previousPersona() {
    if (currentPersona > 0) {
      currentPersona--;
    }
  }
</script>

<div class="stage1-panel">
  <h2>Stage 1: Individual Perspectives</h2>

  <div class="persona-perspective">
    <h3>{perspectives[currentPersona].personaName}</h3>
    <div class="content">
      {perspectives[currentPersona].content}
    </div>

    <div class="navigation">
      <button on:click={previousPersona} disabled={currentPersona === 0}>
        Previous
      </button>
      <span>
        {currentPersona + 1} / {perspectives.length}
      </span>
      <button on:click={nextPersona} disabled={currentPersona === perspectives.length - 1}>
        Next
      </button>
    </div>
  </div>
</div>

Error Handling

We implemented comprehensive error handling across all repositories and API routes:

export class RepositoryErrorHandler {
  static handle(error: unknown, context: string): never {
    if (error instanceof Prisma.PrismaClientKnownRequestError) {
      switch (error.code) {
        case 'P2002':
          throw new Error(`${context}: Record already exists`);
        case 'P2025':
          throw new Error(`${context}: Record not found`);
        default:
          throw new Error(`${context}: Database error - ${error.message}`);
      }
    }

    if (error instanceof Error) {
      throw new Error(`${context}: ${error.message}`);
    }

    throw new Error(`${context}: Unknown error`);
  }
}

export class ProviderErrorHandler {
  static handle(error: unknown, providerType: string): never {
    if (error instanceof Response) {
      throw new Error(`${providerType} provider error: ${error.statusText}`);
    }

    if (error instanceof Error) {
      throw new Error(`${providerType} error: ${error.message}`);
    }

    throw new Error(`${providerType}: Unknown error`);
  }
}

Key Commits

• Initial commit: Implement 3-stage AI consultation process with orchestrator and prompts
• Update database schema and naming conventions for council tables
• feat(council): implement persona-based architecture with multi-provider support
• feat: Refactor council database schema and introduce provider clients
• Implement search functionality
• Add ranking parser utility and improve stage management

What's Next

With the core functionality in place, we're focusing on:

• UI Polish: Improving the user interface and user experience
• Performance: Optimizing for large consultations
• Caching: Implementing response caching for common questions
• Export: Adding export functionality for consultations
• Analytics: Implementing usage analytics and insights
• Testing: Expanding test coverage for all components
• Documentation: Creating comprehensive user guides