vip-coordinator/DOCKER_TROUBLESHOOTING.md

Docker Container Stopping Issues - Troubleshooting Guide

🚨 Issue Observed

During development, we encountered issues where Docker containers would hang during the stopping process, requiring forceful termination. This is concerning for production stability.

🔍 Current System Status

✅ All containers are currently running properly:

• Backend: http://localhost:3000 (responding correctly)
• Frontend: http://localhost:5173
• Database: PostgreSQL on port 5432
• Redis: Running on port 6379

Docker Configuration:

• Storage Driver: overlay2
• Logging Driver: json-file
• Cgroup Driver: systemd
• Cgroup Version: 2

🛠️ Potential Causes & Solutions

1. Graceful Shutdown Issues

Problem: Applications not handling SIGTERM signals properly Solution: Ensure applications handle shutdown gracefully

For Node.js apps (backend/frontend):

// Add to your main application file
process.on('SIGTERM', () => {
  console.log('SIGTERM received, shutting down gracefully');
  server.close(() => {
    console.log('Process terminated');
    process.exit(0);
  });
});

process.on('SIGINT', () => {
  console.log('SIGINT received, shutting down gracefully');
  server.close(() => {
    console.log('Process terminated');
    process.exit(0);
  });
});

2. Docker Compose Configuration

Current issue: Using obsolete version attribute Solution: Update docker-compose.dev.yml

# Remove this line:
# version: '3.8'

# And ensure proper stop configuration:
services:
  backend:
    stop_grace_period: 30s
    stop_signal: SIGTERM
  
  frontend:
    stop_grace_period: 30s
    stop_signal: SIGTERM

3. Resource Constraints

Problem: Insufficient memory/CPU causing hanging Solution: Add resource limits

services:
  backend:
    deploy:
      resources:
        limits:
          memory: 512M
        reservations:
          memory: 256M

4. Database Connection Handling

Problem: Open database connections preventing shutdown Solution: Ensure proper connection cleanup

// In your backend application
process.on('SIGTERM', async () => {
  console.log('Closing database connections...');
  await database.close();
  await redis.quit();
  process.exit(0);
});

🔧 Immediate Fixes to Implement

1. Update Docker Compose File

cd /home/kyle/Desktop/vip-coordinator
# Remove the version line and add stop configurations

2. Add Graceful Shutdown to Backend

# Update backend/src/index.ts with proper signal handling

3. Monitor Container Behavior

# Use these commands to monitor:
docker-compose -f docker-compose.dev.yml logs --follow
docker stats

🚨 Emergency Commands

If containers hang during stopping:

# Force stop all containers
docker-compose -f docker-compose.dev.yml kill

# Remove stopped containers
docker-compose -f docker-compose.dev.yml rm -f

# Clean up system
docker system prune -f

# Restart fresh
docker-compose -f docker-compose.dev.yml up -d

📊 Monitoring Commands

# Check container status
docker-compose -f docker-compose.dev.yml ps

# Monitor logs in real-time
docker-compose -f docker-compose.dev.yml logs -f backend

# Check resource usage
docker stats

# Check for hanging processes
docker-compose -f docker-compose.dev.yml top

🎯 Prevention Strategies

1. Regular Health Checks

   • Implement health check endpoints
   • Monitor container resource usage
   • Set up automated restarts for failed containers

2. Proper Signal Handling

   • Ensure all applications handle SIGTERM/SIGINT
   • Implement graceful shutdown procedures
   • Close database connections properly

3. Resource Management

   • Set appropriate memory/CPU limits
   • Monitor disk space usage
   • Regular cleanup of unused images/containers

🔄 Current OAuth2 Status

✅ OAuth2 is now working correctly:

• Simplified implementation without Passport.js
• Proper domain configuration for bsa.madeamess.online
• Environment variables correctly set
• Backend responding to auth endpoints

Next steps for OAuth2:

1. Update Google Cloud Console with redirect URI: https://bsa.madeamess.online:3000/auth/google/callback
2. Test the full OAuth flow
3. Integrate with frontend

The container stopping issues are separate from the OAuth2 functionality and should be addressed through the solutions above.