Maintenance & Retention Guide

Last Updated: 2026-05-26 • For Database Administrators & DevOps Engineers

Overview

This guide documents the database maintenance routines, data retention policies, logging architecture, and health monitoring systems used to keep QuranBot running smoothly at scale.

Automated database cleaners
15-day retention policies
Multi-level logging architecture
Health check & monitoring

Database Cleaners

Automated cleaners remove stale data to prevent database bloat and maintain performance:

Cleaner	Target Data	Criteria
`controlIds.cleaner.js`	Control message IDs	Removes entries for guilds bot no longer serves or invalid messages
`guildStates.cleaner.js`	Guild state entries	Removes states for left guilds, clears deleted voice channel IDs
`setupGuilds.cleaner.js`	Setup configurations	Validates channel existence, fixes broken IDs, removes invalid categories

// src/database/firebase/maintenance/cleaners/setupGuilds.cleaner.js
async function cleanSetupGuilds(client) {
    try {
        const data = await loadSetupGuildsFromFirebase();
        if (!data || !Object.keys(data).length) return { cleaned: 0, reason: 'No data' };
        
        const botGuilds = new Set(client.guilds.cache.keys());
        const valid = {};
        let removed = 0, updated = 0;
        
        for (const [gid, sData] of Object.entries(data)) {
            if (!botGuilds.has(gid)) { removed++; continue; }
            
            const guild = client.guilds.cache.get(gid);
            if (!guild) { removed++; continue; }
            
            const res = await validateSetupData(gid, sData, guild);
            if (!res.valid) { removed++; continue; }
            
            if (res.hasChanges) { 
                valid[gid] = res.data; 
                updated++; 
            } else { 
                valid[gid] = sData; 
            }
        }
        
        if (removed > 0 || updated > 0) {
            await saveSetupGuildsToFirebase(valid);
        }
        
        return { cleaned: removed, updated, remaining: Object.keys(valid).length };
    } catch (err) {
        logger.error('Error Cleaning Setup Guilds', err);
        return { cleaned: 0, error: err.message };
    }
}

Retention Policies

Guild data is retained for 15 days after bot leaves to allow quick restoration if re-added:

// src/database/firebase/retention/retention.js
const retention_days = 15;
const retention_ms = retention_days * 24 * 60 * 60 * 1000;

async function markGuildAsLeft(guildId) {
    const now = Date.now();
    const updates = {};
    updates[`retention_index/${guildId}/isLeft`] = true;
    updates[`retention_index/${guildId}/leftAt`] = now;
    
    await db.ref().update(updates);
    logger.db(`Guild ${guildId} marked as left in retention index at ${now}`);
}

async function cleanExpiredLeftData(client) {
    const now = Date.now();
    const expiredBefore = now - retention_ms;
    const snapshot = await db.ref('retention_index').once('value');
    const retentionIndex = snapshot.val() || {};
    const expiredGuilds = [];
    
    for (const [guildId, data] of Object.entries(retentionIndex)) {
        if (data?.isLeft && data?.leftAt && data.leftAt < expiredBefore) {
            expiredGuilds.push(guildId);
        }
    }
    // Bulk delete expired data...
}

Logging Architecture

Multi-level logging system with automatic rotation and archiving:

Log Level	File Pattern	Usage
`info`	`logs-general-YYYY-MM-DD.log`	Standard operational messages
`warn`	`logs-warnings-YYYY-MM-DD.log`	Non-critical issues, fallback activations
`error`	`logs-errors-YYYY-MM-DD.log`	Exceptions, failed API calls, voice disconnects
`debug`	`logs-general-YYYY-MM-DD.log`	Verbose diagnostic information (dev only)
`voice`	`voice-*/logs-general-YYYY-MM-DD.log`	Per-guild voice connection events

Log Rotation & Archiving

Logs are automatically archived daily at Cairo midnight:

Location: storage/logs/archive/
Format: logs-general-YYYY-MM-DD.zip
Retention: 60 days (configurable in logging_config)

Voice Logging System

Dedicated per-guild voice logger in src/logging/voiceLogger.js for granular audio debugging:

async function connection(gid, msg, meta = {}, guildCache = null) {
    return this.debug(`[CONN] ${msg}`, meta, gid, guildCache);
}

async function player(gid, msg, meta = {}, guildCache = null) {
    return this.debug(`[PLAYER] ${msg}`, meta, gid, guildCache);
}

async function resource(gid, msg, meta = {}, guildCache = null) {
    return this.debug(`[RESOURCE] ${msg}`, meta, gid, guildCache);
}

Health Check Endpoint

Lightweight HTTP endpoint on port HH_CH_PORT (default: 3000) for monitoring systems:

const healthServer = http.createServer((request, response) => {
    if (request.url === '/health') {
        try {
            const activeVoiceConnections = getConnectedVoiceCount();
            const botuptime = process.uptime();
            const currentMemory = process.memoryUsage();
            
            response.writeHead(200, { 'Content-Type': 'application/json' });
            response.end(JSON.stringify({
                status: 'ok',
                voiceConnections: activeVoiceConnections,
                uptime: botuptime,
                memory: { 
                    rss: currentMemory.rss, 
                    heapTotal: currentMemory.heapTotal, 
                    heapUsed: currentMemory.heapUsed 
                },
                timestamp: new Date(),
            }));
        } catch (err) {
            response.writeHead(500, { 'Content-Type': 'application/json' });
            response.end(JSON.stringify({ status: 'error', message: err.message }));
        }
    } else { 
        response.writeHead(404); 
        response.end(); 
    }
});

Backup System

Automated compressed backups sent to Discord channels at configurable intervals:

// src/database/local/localBackup.js
const BACKUP_INTERVAL_MS = parseInt(process.env.BACKUP_INTERVAL_MS);

async function performBackup() {
    const snap = await get(ref(db, '/'));
    const data = snap.val();
    const filename = generateBackupFilename();
    const backupPath = pathlra.join(BACKUP_DIR, filename);
    
    await fsPromises.writeFile(backupPath.replace('.gz', '.tmp.json'), JSON.stringify(data, null, 2));
    await compressFile(backupPath.replace('.gz', '.tmp.json'), backupPath);
    await sendBackupToDiscord(backupPath, filename);
}

function startBackupService() {
    setTimeout(() => { 
        setInterval(performBackup, BACKUP_INTERVAL_MS); 
    }, 5000);
}

Troubleshooting Maintenance

Logs growing too large

Check:

Verify logging_config.max_files_to_keep (default: 10)
Check storage/logs/archive/ for old zip files
Run manual cleanup: cleanupOldLogs('storage/logs', 60)

Health check returns 500

Check:

Port HH_CH_PORT is not in use by another service
Bot client is initialized and connected
getConnectedVoiceCount() isn't throwing on invalid guild state

Firebase data growing unbounded

Cleaners may not be running:

Verify databaseCleaner.performMaintenance(client) is scheduled
Check retention interval: cleanupInterval = 24 * 60 * 60 * 1000 (24h)
Run manual cleanup via Discord command or script