Lesson 3.3: Data Persistence and Backups

Data Persistence Strategies

Ensuring your data survives container restarts, updates, and system failures is crucial for a reliable personal cloud.

Docker Volume Types

Named Volumes

Docker-managed storage with automatic lifecycle management:

volumes:
  nextcloud_data:
    driver: local
  mariadb_data:
    driver: local

Bind Mounts

Direct mapping to host filesystem paths:

volumes:
  - ./nextcloud/data:/var/www/html/data
  - ./nextcloud/config:/var/www/html/config

tmpfs Mounts

In-memory storage for temporary data:

tmpfs:
  - /tmp
  - /var/cache

MCT Services Data Strategy

Our approach to data persistence:

# Critical data - bind mounts for easy backup
./nextcloud/data:/var/www/html/data
./stash/config:/root/.stash

# Database data - named volumes with backup scripts
mariadb_data:/var/lib/mysql
postgres_data:/var/lib/postgresql/data

# Cache data - tmpfs or local volumes
./redis/data:/data
./stash/cache:/cache

Backup Architecture

3-2-1 Backup Rule

• 3 copies: Original + 2 backups
• 2 different media: Local + cloud/external
• 1 offsite: Geographic separation

Backup Types

• Full Backup: Complete system snapshot
• Incremental: Changes since last backup
• Differential: Changes since last full backup
• Continuous: Real-time replication

Automated Backup Scripts

Comprehensive backup automation:

#!/bin/bash
# MCT Services Backup Script

BACKUP_ROOT="/backups/mct-services"
DATE=20251106_085258
RETENTION_DAYS=30

# Create backup directory
mkdir -p "/"

# Database backups
echo "Backing up databases..."
docker exec mariadb mysqldump --all-databases -u root -p > "//all_databases.sql"

# Application data backups
echo "Backing up application data..."
tar -czf "//nextcloud_data.tar.gz" ./nextcloud/data
tar -czf "//stash_config.tar.gz" ./stash/config

# Configuration backups
echo "Backing up configurations..."
tar -czf "//docker_configs.tar.gz" ./*.yml ./*/docker-compose.yml

# Docker volumes backup
echo "Backing up Docker volumes..."
docker run --rm -v nextcloud_data:/data -v /:/backup alpine tar czf /backup/nextcloud_volume.tar.gz -C /data .

# Cleanup old backups
echo "Cleaning up old backups..."
find "" -type d -mtime + -exec rm -rf {} +

echo "Backup completed: /"

Restore Procedures

Document and test restore procedures:

#!/bin/bash
# MCT Services Restore Script

BACKUP_DATE=
BACKUP_ROOT="/backups/mct-services"

if [ -z "" ]; then
    echo "Usage: /bin/sh "
    echo "Available backups:"
    ls -1 ""
    exit 1
fi

# Stop services
echo "Stopping services..."
docker-compose down

# Restore database
echo "Restoring database..."
docker-compose up -d mariadb
sleep 10
docker exec mariadb mysql -u root -p < "//all_databases.sql"

# Restore application data
echo "Restoring application data..."
tar -xzf "//nextcloud_data.tar.gz" -C ./
tar -xzf "//stash_config.tar.gz" -C ./

# Restart services
echo "Starting services..."
docker-compose up -d

echo "Restore completed from backup: "

Monitoring and Alerting

Set up monitoring for backup health:

• Backup Success: Monitor backup script exit codes
• Storage Space: Alert on low disk space
• Data Integrity: Regular restore testing
• Offsite Sync: Verify cloud backup synchronization

Practical Exercise

Task: Implement backup and restore system

1. Create automated backup scripts
2. Set up scheduled backups with cron
3. Test restore procedures
4. Configure offsite backup storage
5. Document recovery procedures