Devlog Jan 22: Database & Infrastructure

January 25, 2026 · 8 min read

MoLOS Lead Developer

This week we focused heavily on database improvements, infrastructure enhancements, and laid the groundwork for Telegram integration. We also added search functionality and improved the overall deployment experience.

Major Accomplishments

Module Synchronization & Database Columns

We added new columns for module integration (7fb0ed1):

// Enhanced Module Schema
model Module {
  id              String    @id @default(uuid())
  name            String
  description     String?
  version         String
  author          String?
  installed       Boolean   @default(false)
  enabled         Boolean   @default(true)
  path            String?
  gitUrl          String?
  lastUpdate      DateTime?
  lastSync        DateTime?
  syncStatus      SyncStatus @default(IDLE)
  delete          Boolean   @default(false)
  config          Json?
  dependencies    String[]
  permissions     ModulePermissions @default({})
  
  aiTools         AITool[]
  errors          ModuleError[]
  migrations      _Migration[]

  @@index([installed])
  @@index([enabled])
  @@index([syncStatus])
}

model ModulePermissions {
  read            Boolean   @default(true)
  write           Boolean   @default(false)
  execute         Boolean   @default(false)
  network         Boolean   @default(false)
  filesystem      Boolean   @default(false)
}

enum SyncStatus {
  IDLE
  SYNCING
  ERROR
  SUCCESS
}

Search Functionality

Implemented comprehensive search functionality with model support and UI filters (9d78a3f):

Search API:

// Search Service
export class SearchService {
  private index: Map<string, SearchResult[]> = new Map();

  async indexModule(module: Module) {
    const results: SearchResult[] = [];

    // Index module metadata
    results.push({
      type: 'module',
      id: module.id,
      title: module.name,
      description: module.description,
      url: `/modules/${module.id}`,
      score: 1.0
    });

    // Index AI tools
    for (const tool of module.aiTools) {
      results.push({
        type: 'ai-tool',
        id: tool.id,
        title: tool.name,
        description: tool.description,
        url: `/modules/${module.id}/tools/${tool.id}`,
        score: 0.8
      });
    }

    this.index.set(module.id, results);
  }

  async search(query: string, filters?: SearchFilters): Promise<SearchResult[]> {
    const normalizedQuery = query.toLowerCase();
    const results: SearchResult[] = [];

    for (const [, moduleResults] of this.index) {
      for (const result of moduleResults) {
        // Apply filters
        if (filters?.type && result.type !== filters.type) {
          continue;
        }

        if (filters?.moduleId && result.id !== filters.moduleId) {
          continue;
        }

        // Search in title and description
        const titleMatch = result.title.toLowerCase().includes(normalizedQuery);
        const descMatch = result.description?.toLowerCase().includes(normalizedQuery);

        if (titleMatch || descMatch) {
          const score = titleMatch ? 1.0 : 0.5;
          results.push({ ...result, score });
        }
      }
    }

    // Sort by score
    return results.sort((a, b) => b.score - a.score);
  }
}

// Search API Route
app.get('/api/search', async (req, res) => {
  const { q, type, moduleId } = req.query;

  const results = await searchService.search(q as string, {
    type: type as 'module' | 'ai-tool' | 'all',
    moduleId: moduleId as string
  });

  res.json(results);
});

Search UI:

// Search Component
import { useState, useEffect } from 'react';

export function SearchBar() {
  const [query, setQuery] = useState('');
  const [results, setResults] = useState<SearchResult[]>([]);
  const [isSearching, setIsSearching] = useState(false);
  const [filters, setFilters] = useState({
    type: 'all' as 'module' | 'ai-tool' | 'all'
  });

  useEffect(() => {
    const timeoutId = setTimeout(async () => {
      if (query.trim().length === 0) {
        setResults([]);
        return;
      }

      setIsSearching(true);

      try {
        const response = await fetch(
          `/api/search?q=${encodeURIComponent(query)}&type=${filters.type}`
        );
        const results = await response.json();
        setResults(results);
      } catch (error) {
        console.error('Search failed:', error);
      } finally {
        setIsSearching(false);
      }
    }, 300);

    return () => clearTimeout(timeoutId);
  }, [query, filters]);

  return (
    <div className="search-container">
      <input
        type="text"
        placeholder="Search modules and AI tools..."
        value={query}
        onChange={(e) => setQuery(e.target.value)}
        className="search-input"
      />

      <div className="search-filters">
        <select
          value={filters.type}
          onChange={(e) => setFilters({ type: e.target.value as any })}
        >
          <option value="all">All</option>
          <option value="module">Modules</option>
          <option value="ai-tool">AI Tools</option>
        </select>
      </div>

      {isSearching && <div className="search-loading">Searching...</div>}

      {results.length > 0 && (
        <div className="search-results">
          {results.map(result => (
            <div key={`${result.type}-${result.id}`} className="search-result">
              <Link href={result.url}>
                <span className="result-type">{result.type}</span>
                <h3>{result.title}</h3>
                {result.description && (
                  <p>{result.description}</p>
                )}
              </Link>
            </div>
          ))}
        </div>
      )}
    </div>
  );
}

Docker & Infrastructure Improvements

Enhanced Docker setup with security hardening and resource limits (5054c74, fdbdafc):

Dockerfile Improvements:

# Multi-stage Dockerfile
FROM node:20-alpine AS builder

WORKDIR /app

# Install dependencies
COPY package*.json ./
RUN npm ci --only=production

# Copy application code
COPY . .

# Build application
RUN npm run build

# Production image
FROM node:20-alpine

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

WORKDIR /app

# Copy from builder
COPY --from=builder --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
COPY --from=builder --chown=nodejs:nodejs /app/package*.json ./

# Create directories
RUN mkdir -p external_modules && \
    chown -R nodejs:nodejs external_modules

USER nodejs

EXPOSE 8080

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD node -e "require('http').get('http://localhost:8080/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"

CMD ["node", "dist/server.js"]

Docker Compose:

version: '3.8'

services:
  molos:
    build: .
    ports:
      - "8080:8080"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://molos:${DB_PASSWORD}@postgres:5432/molos
      - EXTERNAL_MODULES_PATH=/app/external_modules
    volumes:
      - ./external_modules:/app/external_modules
      - ./database:/app/database
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "wget", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 512M
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      - POSTGRES_USER=molos
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=molos
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U molos"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 1G
        reservations:
          cpus: '0.25'
          memory: 256M

volumes:
  postgres_data:

Module Installation Management

Improved module installation management for better migration error handling (5054c74):

// Enhanced Module Installation
export async function installModule(gitUrl: string): Promise<Module> {
  // Clone repository
  const clonePath = await cloneRepository(gitUrl);

  try {
    // Read module manifest
    const manifest = await readModuleManifest(clonePath);

    // Validate manifest
    validateModuleManifest(manifest);

    // Install dependencies
    await installModuleDependencies(clonePath, manifest);

    // Run migrations
    await runModuleMigrations(manifest.id, clonePath);

    // Register module
    const module = await prisma.module.create({
      data: {
        id: manifest.id,
        name: manifest.name,
        description: manifest.description,
        version: manifest.version,
        author: manifest.author,
        installed: true,
        enabled: true,
        path: clonePath,
        gitUrl,
        lastUpdate: new Date(),
        lastSync: new Date(),
        syncStatus: 'SUCCESS',
        config: manifest.config || {},
        dependencies: manifest.dependencies || [],
        permissions: manifest.permissions || {}
      }
    });

    // Index module for search
    await searchService.indexModule(module);

    return module;
  } catch (error) {
    // Cleanup on failure
    await cleanupFailedInstallation(clonePath);
    throw error;
  }
}

async function runModuleMigrations(moduleId: string, modulePath: string) {
  const migrationsPath = path.join(modulePath, 'prisma', 'migrations');

  if (!fs.existsSync(migrationsPath)) {
    return; // No migrations to run
  }

  const migrationFiles = await fs.readdir(migrationsPath);
  migrationFiles.sort(); // Run in order

  for (const file of migrationFiles) {
    const filePath = path.join(migrationsPath, file);
    const sql = await fs.readFile(filePath, 'utf-8');

    try {
      // Validate SQL
      validateModuleSQL(moduleId, sql);

      // Execute migration
      await prisma.$executeRawUnsafe(sql);

      // Record migration
      await prisma._migration.create({
        data: {
          name: file,
          module_id: moduleId,
          applied_at: new Date()
        }
      });

      console.log(`✓ Applied migration: ${file}`);
    } catch (error) {
      console.error(`✗ Migration failed: ${file}`, error);
      throw new ModuleMigrationError(moduleId, file, error.message);
    }
  }
}

async function cleanupFailedInstallation(modulePath: string) {
  try {
    await fs.rm(modulePath, { recursive: true, force: true });
  } catch (error) {
    console.error('Failed to cleanup installation:', error);
  }
}

External Modules Support

Added comprehensive support for external modules with Utils folder support (fdbdafc):

// External Modules Configuration
export interface ExternalModulesConfig {
  path: string;
  autoLoad: boolean;
  gitUrls: string[];
}

export function loadExternalModules(config: ExternalModulesConfig): Module[] {
  const modules: Module[] = [];

  // Load from local directory
  if (fs.existsSync(config.path)) {
    const moduleDirs = fs.readdirSync(config.path);

    for (const dir of moduleDirs) {
      const modulePath = path.join(config.path, dir);
      const manifestPath = path.join(modulePath, 'module.json');

      if (fs.existsSync(manifestPath)) {
        const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
        
        // Load Utils folder if exists
        const utilsPath = path.join(modulePath, 'utils');
        if (fs.existsSync(utilsPath)) {
          const utilsFiles = fs.readdirSync(utilsPath);
          for (const utilFile of utilsFiles) {
            if (utilFile.endsWith('.ts') || utilFile.endsWith('.js')) {
              const utilPath = path.join(utilsPath, utilFile);
              require(utilPath);
            }
          }
        }

        modules.push({
          ...manifest,
          path: modulePath
        });
      }
    }
  }

  return modules;
}

Theme Improvements

Improved theming support (5384127):

// Theme Configuration
export interface Theme {
  name: string;
  colors: {
    primary: string;
    secondary: string;
    background: string;
    surface: string;
    text: string;
    error: string;
    success: string;
  };
  fonts: {
    primary: string;
    monospace: string;
  };
  spacing: {
    xs: string;
    sm: string;
    md: string;
    lg: string;
    xl: string;
  };
}

export const defaultTheme: Theme = {
  name: 'default',
  colors: {
    primary: '#6366f1',
    secondary: '#8b5cf6',
    background: '#f8fafc',
    surface: '#ffffff',
    text: '#1e293b',
    error: '#ef4444',
    success: '#10b981'
  },
  fonts: {
    primary: 'Inter, system-ui, sans-serif',
    monospace: 'JetBrains Mono, monospace'
  },
  spacing: {
    xs: '0.5rem',
    sm: '1rem',
    md: '1.5rem',
    lg: '2rem',
    xl: '3rem'
  }
};

// Theme Provider
export function ThemeProvider({ children, theme }: { children: React.ReactNode, theme?: Theme }) {
  const currentTheme = theme || defaultTheme;

  return (
    <div className="theme-provider" data-theme={currentTheme.name}>
      <style>{`
        :root {
          --color-primary: ${currentTheme.colors.primary};
          --color-secondary: ${currentTheme.colors.secondary};
          --color-background: ${currentTheme.colors.background};
          --color-surface: ${currentTheme.colors.surface};
          --color-text: ${currentTheme.colors.text};
          --color-error: ${currentTheme.colors.error};
          --color-success: ${currentTheme.colors.success};
          
          --font-primary: ${currentTheme.fonts.primary};
          --font-monospace: ${currentTheme.fonts.monospace};
          
          --spacing-xs: ${currentTheme.spacing.xs};
          --spacing-sm: ${currentTheme.spacing.sm};
          --spacing-md: ${currentTheme.spacing.md};
          --spacing-lg: ${currentTheme.spacing.lg};
          --spacing-xl: ${currentTheme.spacing.xl};
        }
      `}</style>
      {children}
    </div>
  );
}

Bug Fixes

Fixed SQL false positives in security validation (599a241)
Fixed welcome redirection issues (5054c74)
Removed emojis for better cross-platform compatibility (6b1dc9d)

Cleanup & Maintenance

Added cleanup symlink script (73d796e)
Added external modules to gitignore (e3d7067)
Improved dockerignore for better builds (5054c74)

What's Next

Next week we'll focus on:

Telegram Integration: Continue work on Telegram bot integration
Module Marketplace: Enhanced module discovery and installation
Performance Optimizations: Database and application performance improvements
Documentation Updates: Updated guides and API documentation

The foundation is now solid. With comprehensive search, improved infrastructure, and better module management, MoLOS is ready for the next phase of development. Stay tuned for more updates!

Related Commits:

7fb0ed1 - Added new columns for module integration
9d78a3f - Added search functionality
5054c74 - Improved module installation management
fdbdafc - Added Utils folder support
5384127 - Improved themes
599a241 - Fixed SQL false positives

Major Accomplishments​

Module Synchronization & Database Columns​

Search Functionality​

Docker & Infrastructure Improvements​

Module Installation Management​

External Modules Support​

Theme Improvements​

Bug Fixes​

Cleanup & Maintenance​

What's Next​