Complete Phase 7: Polish & Cross-Cutting Concerns
This commit implements all remaining polish tasks (T193-T210) to make the application production-ready. ## Logging & Monitoring (T193, T194, T208, T209) - Add structured JSON logging for production environments - Add human-readable logging for development - Implement comprehensive error logging across all routes: * submission.py: product access, validation, success/failure * auth.py: login attempts, successes, failures, logouts * dashboard.py: access and errors - Add /health endpoint for monitoring (checks data dir, API key) - Add environment variable validation on startup ## Security Hardening (T196-T199, T207) - Add HSTS headers in production (1 year, includeSubDomains) - Add security headers: X-Content-Type-Options, X-Frame-Options, X-XSS-Protection - Verify CSRF protection on all POST routes (Flask-WTF) - Verify session cookie security flags (HttpOnly, Secure, SameSite) - Verify XSS prevention (Jinja2 auto-escaping) - Verify no hardcoded secrets (only in test files) ## Documentation (T195, T203, T210) - Add comprehensive README.md with: * Features, quick start, project structure * Usage guides (end users, product owners, admins) * Configuration, testing, deployment instructions - Add detailed docs/deployment.md with: * Production deployment steps * ClamAV, Nginx, SSL/TLS setup * Security hardening, monitoring, backup strategies - Add requirements-dev.txt for development dependencies ## Performance Testing (T200, T201) - Add test_performance.py with 4 comprehensive tests: * 100 concurrent submissions (SC-012) * Dashboard load <3s for 1000 items (SC-008) * Large file upload handling * Rate limiting verification - Add performance marker to pytest.ini ## Testing - All 49 tests passing, 1 skipped - Fixed error handling to preserve HTTP status codes Phase 7 complete. Application is production-ready with comprehensive logging, security, monitoring, and documentation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
@@ -0,0 +1,618 @@
|
||||
# Reklamator Deployment Guide
|
||||
|
||||
This guide covers deploying Reklamator to a production Linux server with security best practices.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Linux server (Ubuntu 22.04 LTS recommended)
|
||||
- Domain name with DNS configured
|
||||
- Root or sudo access
|
||||
- SSL/TLS certificate (Let's Encrypt recommended)
|
||||
|
||||
## System Requirements
|
||||
|
||||
- Python 3.11 or higher
|
||||
- 2GB RAM minimum (4GB recommended)
|
||||
- 20GB disk space (depends on feedback volume)
|
||||
- ClamAV for virus scanning
|
||||
- Nginx as reverse proxy
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. System Setup
|
||||
|
||||
```bash
|
||||
# Update system packages
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Install required packages
|
||||
sudo apt install -y python3.11 python3.11-venv python3-pip nginx clamav clamav-daemon git
|
||||
|
||||
# Install certbot for Let's Encrypt SSL
|
||||
sudo apt install -y certbot python3-certbot-nginx
|
||||
```
|
||||
|
||||
### 2. ClamAV Configuration
|
||||
|
||||
```bash
|
||||
# Stop ClamAV daemon
|
||||
sudo systemctl stop clamav-daemon
|
||||
|
||||
# Update virus definitions
|
||||
sudo freshclam
|
||||
|
||||
# Start and enable ClamAV daemon
|
||||
sudo systemctl start clamav-daemon
|
||||
sudo systemctl enable clamav-daemon
|
||||
|
||||
# Verify ClamAV is running
|
||||
sudo systemctl status clamav-daemon
|
||||
|
||||
# Test ClamAV socket
|
||||
clamdscan --version
|
||||
```
|
||||
|
||||
**ClamAV Configuration File** (`/etc/clamav/clamd.conf`):
|
||||
```conf
|
||||
# Uncomment this line if present
|
||||
LocalSocket /var/run/clamav/clamd.ctl
|
||||
|
||||
# Set appropriate permissions
|
||||
User clamav
|
||||
SocketGroup clamav
|
||||
SocketMode 666
|
||||
|
||||
# Increase timeouts for large files
|
||||
ReadTimeout 300
|
||||
CommandReadTimeout 30
|
||||
|
||||
# Memory limits
|
||||
MaxFileSize 25M
|
||||
MaxScanSize 100M
|
||||
```
|
||||
|
||||
### 3. Application Deployment
|
||||
|
||||
```bash
|
||||
# Create application user
|
||||
sudo useradd -r -s /bin/bash -m -d /opt/reklamator reklamator
|
||||
|
||||
# Switch to application user
|
||||
sudo su - reklamator
|
||||
|
||||
# Clone repository
|
||||
git clone <repository-url> /opt/reklamator/app
|
||||
cd /opt/reklamator/app
|
||||
|
||||
# Create virtual environment
|
||||
python3.11 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create data directory
|
||||
mkdir -p /opt/reklamator/data
|
||||
|
||||
# Exit back to root
|
||||
exit
|
||||
```
|
||||
|
||||
### 4. Environment Configuration
|
||||
|
||||
Create `/opt/reklamator/app/.env`:
|
||||
|
||||
```bash
|
||||
# Security (REQUIRED)
|
||||
SECRET_KEY=<generate-secure-key>
|
||||
ANTHROPIC_API_KEY=<your-api-key>
|
||||
|
||||
# Paths
|
||||
DATA_DIR=/opt/reklamator/data
|
||||
|
||||
# ClamAV
|
||||
CLAMD_SOCKET=/var/run/clamav/clamd.ctl
|
||||
|
||||
# Rate Limiting
|
||||
RATE_LIMIT_ENABLED=true
|
||||
RATE_LIMIT_PER_HOUR=10
|
||||
|
||||
# File Upload
|
||||
MAX_CONTENT_LENGTH=10485760
|
||||
```
|
||||
|
||||
**Generate SECRET_KEY:**
|
||||
```bash
|
||||
python3 -c "import secrets; print(secrets.token_hex(32))"
|
||||
```
|
||||
|
||||
Set proper permissions:
|
||||
```bash
|
||||
sudo chown reklamator:reklamator /opt/reklamator/app/.env
|
||||
sudo chmod 600 /opt/reklamator/app/.env
|
||||
```
|
||||
|
||||
### 5. Initialize Application
|
||||
|
||||
```bash
|
||||
# Switch to application user
|
||||
sudo su - reklamator
|
||||
cd /opt/reklamator/app
|
||||
source venv/bin/activate
|
||||
|
||||
# Initialize admin user
|
||||
python init_admin.py
|
||||
# Follow prompts to create admin user
|
||||
|
||||
# Create test product (optional)
|
||||
mkdir -p /opt/reklamator/data/products/test-product
|
||||
cat > /opt/reklamator/data/products/test-product/config.yaml <<EOF
|
||||
product_id: test-product
|
||||
name: Test Product
|
||||
submission_url_slug: test-feedback
|
||||
owner_language: en
|
||||
assigned_owner_ids:
|
||||
- admin
|
||||
status: active
|
||||
EOF
|
||||
|
||||
exit
|
||||
```
|
||||
|
||||
### 6. Systemd Service Configuration
|
||||
|
||||
Create `/etc/systemd/system/reklamator.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Reklamator Feedback Platform
|
||||
After=network.target clamav-daemon.service
|
||||
Requires=clamav-daemon.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=reklamator
|
||||
Group=reklamator
|
||||
WorkingDirectory=/opt/reklamator/app
|
||||
Environment="PATH=/opt/reklamator/app/venv/bin"
|
||||
Environment="FLASK_ENV=production"
|
||||
ExecStart=/opt/reklamator/app/venv/bin/python run.py
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/opt/reklamator/data
|
||||
RestartSec=10
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Enable and start the service:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable reklamator
|
||||
sudo systemctl start reklamator
|
||||
sudo systemctl status reklamator
|
||||
```
|
||||
|
||||
### 7. Nginx Reverse Proxy Configuration
|
||||
|
||||
Create `/etc/nginx/sites-available/reklamator`:
|
||||
|
||||
```nginx
|
||||
# Redirect HTTP to HTTPS
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name feedback.yourdomain.com;
|
||||
|
||||
location /.well-known/acme-challenge/ {
|
||||
root /var/www/html;
|
||||
}
|
||||
|
||||
location / {
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
# HTTPS server
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
listen [::]:443 ssl http2;
|
||||
server_name feedback.yourdomain.com;
|
||||
|
||||
# SSL Configuration
|
||||
ssl_certificate /etc/letsencrypt/live/feedback.yourdomain.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/feedback.yourdomain.com/privkey.pem;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
ssl_ciphers HIGH:!aNULL:!MD5;
|
||||
ssl_prefer_server_ciphers on;
|
||||
ssl_session_cache shared:SSL:10m;
|
||||
ssl_session_timeout 10m;
|
||||
|
||||
# Security Headers (additional to Flask's HSTS)
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-XSS-Protection "1; mode=block" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
|
||||
# Max upload size (must match Flask's MAX_CONTENT_LENGTH)
|
||||
client_max_body_size 35M; # 3 files × 10MB + overhead
|
||||
|
||||
# Timeouts
|
||||
proxy_connect_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
proxy_read_timeout 60s;
|
||||
|
||||
# Logging
|
||||
access_log /var/log/nginx/reklamator_access.log;
|
||||
error_log /var/log/nginx/reklamator_error.log;
|
||||
|
||||
# Proxy to Flask app
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:5000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Don't buffer large uploads
|
||||
proxy_request_buffering off;
|
||||
}
|
||||
|
||||
# Health check endpoint (no authentication)
|
||||
location /health {
|
||||
proxy_pass http://127.0.0.1:5000/health;
|
||||
access_log off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Enable the site:
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/reklamator /etc/nginx/sites-enabled/
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### 8. SSL/TLS Certificate (Let's Encrypt)
|
||||
|
||||
```bash
|
||||
# Obtain certificate
|
||||
sudo certbot --nginx -d feedback.yourdomain.com
|
||||
|
||||
# Test auto-renewal
|
||||
sudo certbot renew --dry-run
|
||||
```
|
||||
|
||||
Certbot will automatically update the Nginx configuration with SSL settings.
|
||||
|
||||
### 9. Firewall Configuration
|
||||
|
||||
```bash
|
||||
# Allow SSH, HTTP, and HTTPS
|
||||
sudo ufw allow OpenSSH
|
||||
sudo ufw allow 'Nginx Full'
|
||||
sudo ufw enable
|
||||
sudo ufw status
|
||||
```
|
||||
|
||||
## Post-Deployment Verification
|
||||
|
||||
### Health Check
|
||||
|
||||
```bash
|
||||
curl https://feedback.yourdomain.com/health
|
||||
```
|
||||
|
||||
Expected response:
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"timestamp": "2024-01-15T12:00:00Z",
|
||||
"environment": "production"
|
||||
}
|
||||
```
|
||||
|
||||
### Test Submission
|
||||
|
||||
1. Visit `https://feedback.yourdomain.com/submit/test-feedback`
|
||||
2. Submit test feedback
|
||||
3. Check logs: `sudo journalctl -u reklamator -f`
|
||||
4. Verify file creation: `ls -la /opt/reklamator/data/products/test-product/feedback/`
|
||||
|
||||
### Test Dashboard
|
||||
|
||||
1. Visit `https://feedback.yourdomain.com/login`
|
||||
2. Log in with admin credentials
|
||||
3. Verify dashboard loads: `https://feedback.yourdomain.com/dashboard`
|
||||
|
||||
## Monitoring and Logging
|
||||
|
||||
### Application Logs
|
||||
|
||||
```bash
|
||||
# Real-time logs
|
||||
sudo journalctl -u reklamator -f
|
||||
|
||||
# Last 100 lines
|
||||
sudo journalctl -u reklamator -n 100
|
||||
|
||||
# Logs with timestamps
|
||||
sudo journalctl -u reklamator --since "1 hour ago"
|
||||
```
|
||||
|
||||
### Nginx Logs
|
||||
|
||||
```bash
|
||||
# Access logs
|
||||
sudo tail -f /var/log/nginx/reklamator_access.log
|
||||
|
||||
# Error logs
|
||||
sudo tail -f /var/log/nginx/reklamator_error.log
|
||||
```
|
||||
|
||||
### Health Monitoring
|
||||
|
||||
Set up automated health checks with your monitoring service:
|
||||
|
||||
```bash
|
||||
# Example with curl in cron
|
||||
*/5 * * * * curl -f https://feedback.yourdomain.com/health || echo "Reklamator health check failed" | mail -s "Alert: Reklamator Down" admin@yourdomain.com
|
||||
```
|
||||
|
||||
### Log Rotation
|
||||
|
||||
Create `/etc/logrotate.d/reklamator`:
|
||||
|
||||
```
|
||||
/opt/reklamator/app/*.log {
|
||||
daily
|
||||
missingok
|
||||
rotate 14
|
||||
compress
|
||||
delaycompress
|
||||
notifempty
|
||||
create 0640 reklamator reklamator
|
||||
sharedscripts
|
||||
}
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Database Backup (YAML Files)
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# /opt/reklamator/backup.sh
|
||||
|
||||
BACKUP_DIR="/opt/reklamator/backups"
|
||||
DATA_DIR="/opt/reklamator/data"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
|
||||
mkdir -p $BACKUP_DIR
|
||||
|
||||
# Backup data directory
|
||||
tar -czf $BACKUP_DIR/reklamator_data_$DATE.tar.gz -C $DATA_DIR .
|
||||
|
||||
# Keep only last 7 days
|
||||
find $BACKUP_DIR -name "reklamator_data_*.tar.gz" -mtime +7 -delete
|
||||
|
||||
echo "Backup completed: $BACKUP_DIR/reklamator_data_$DATE.tar.gz"
|
||||
```
|
||||
|
||||
Make executable and add to cron:
|
||||
```bash
|
||||
chmod +x /opt/reklamator/backup.sh
|
||||
sudo crontab -e -u reklamator
|
||||
# Add: 0 2 * * * /opt/reklamator/backup.sh
|
||||
```
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Update Application
|
||||
|
||||
```bash
|
||||
sudo su - reklamator
|
||||
cd /opt/reklamator/app
|
||||
|
||||
# Pull latest code
|
||||
git pull
|
||||
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate
|
||||
|
||||
# Update dependencies
|
||||
pip install -r requirements.txt --upgrade
|
||||
|
||||
# Exit back to root
|
||||
exit
|
||||
|
||||
# Restart service
|
||||
sudo systemctl restart reklamator
|
||||
sudo systemctl status reklamator
|
||||
```
|
||||
|
||||
### Update ClamAV Virus Definitions
|
||||
|
||||
```bash
|
||||
# Manual update
|
||||
sudo freshclam
|
||||
|
||||
# Automatic updates are configured by default in /etc/clamav/freshclam.conf
|
||||
```
|
||||
|
||||
### Disk Space Management
|
||||
|
||||
Monitor feedback storage:
|
||||
```bash
|
||||
du -sh /opt/reklamator/data/products/*/feedback
|
||||
```
|
||||
|
||||
Archive old feedback:
|
||||
```bash
|
||||
# Example: Move feedback older than 1 year to archive
|
||||
find /opt/reklamator/data/products/*/feedback/ -type d -mtime +365 \
|
||||
-exec mv {} /opt/reklamator/archive/ \;
|
||||
```
|
||||
|
||||
## Security Hardening
|
||||
|
||||
### File Permissions
|
||||
|
||||
```bash
|
||||
# Application files
|
||||
sudo chown -R reklamator:reklamator /opt/reklamator/app
|
||||
sudo chmod -R 755 /opt/reklamator/app
|
||||
sudo chmod 600 /opt/reklamator/app/.env
|
||||
|
||||
# Data directory
|
||||
sudo chown -R reklamator:reklamator /opt/reklamator/data
|
||||
sudo chmod -R 750 /opt/reklamator/data
|
||||
```
|
||||
|
||||
### ClamAV Permissions
|
||||
|
||||
Add reklamator user to clamav group:
|
||||
```bash
|
||||
sudo usermod -a -G clamav reklamator
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
Nginx can provide additional rate limiting:
|
||||
|
||||
```nginx
|
||||
# Add to http block in /etc/nginx/nginx.conf
|
||||
limit_req_zone $binary_remote_addr zone=submission:10m rate=10r/h;
|
||||
|
||||
# Add to location block for /submit/*
|
||||
location ~ ^/submit/ {
|
||||
limit_req zone=submission burst=2 nodelay;
|
||||
proxy_pass http://127.0.0.1:5000;
|
||||
# ... other proxy settings
|
||||
}
|
||||
```
|
||||
|
||||
### Intrusion Detection
|
||||
|
||||
Install and configure fail2ban:
|
||||
```bash
|
||||
sudo apt install fail2ban
|
||||
|
||||
# Create /etc/fail2ban/jail.local
|
||||
[nginx-limit-req]
|
||||
enabled = true
|
||||
filter = nginx-limit-req
|
||||
logpath = /var/log/nginx/reklamator_error.log
|
||||
maxretry = 5
|
||||
bantime = 3600
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
sudo systemctl status reklamator
|
||||
|
||||
# Check logs
|
||||
sudo journalctl -u reklamator -n 50 --no-pager
|
||||
|
||||
# Common issues:
|
||||
# - Missing environment variables
|
||||
# - ClamAV not running
|
||||
# - Permissions on data directory
|
||||
# - Port 5000 already in use
|
||||
```
|
||||
|
||||
### ClamAV Connection Errors
|
||||
|
||||
```bash
|
||||
# Check ClamAV daemon status
|
||||
sudo systemctl status clamav-daemon
|
||||
|
||||
# Test socket connection
|
||||
clamdscan --version
|
||||
|
||||
# Check permissions
|
||||
ls -la /var/run/clamav/clamd.ctl
|
||||
|
||||
# Restart ClamAV
|
||||
sudo systemctl restart clamav-daemon
|
||||
```
|
||||
|
||||
### High Memory Usage
|
||||
|
||||
```bash
|
||||
# Check memory usage
|
||||
free -h
|
||||
|
||||
# Restart application to clear memory leaks
|
||||
sudo systemctl restart reklamator
|
||||
|
||||
# Consider increasing server resources if processing large volumes
|
||||
```
|
||||
|
||||
### Slow AI Analysis
|
||||
|
||||
```bash
|
||||
# Check Anthropic API rate limits in logs
|
||||
sudo journalctl -u reklamator | grep "analysis"
|
||||
|
||||
# Consider increasing timeout in config/production.py
|
||||
# ANTHROPIC_API_TIMEOUT = 60 # seconds
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### Gunicorn Configuration (Optional)
|
||||
|
||||
For production deployments with high traffic, use Gunicorn instead of Flask's development server.
|
||||
|
||||
Install Gunicorn:
|
||||
```bash
|
||||
sudo su - reklamator
|
||||
source /opt/reklamator/app/venv/bin/activate
|
||||
pip install gunicorn
|
||||
```
|
||||
|
||||
Update systemd service (`/etc/systemd/system/reklamator.service`):
|
||||
```ini
|
||||
[Service]
|
||||
ExecStart=/opt/reklamator/app/venv/bin/gunicorn -w 4 -b 127.0.0.1:5000 --timeout 60 'run:app'
|
||||
```
|
||||
|
||||
Restart:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart reklamator
|
||||
```
|
||||
|
||||
### Nginx Caching (Optional)
|
||||
|
||||
For static assets:
|
||||
```nginx
|
||||
location /static/ {
|
||||
alias /opt/reklamator/app/static/;
|
||||
expires 1y;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
```
|
||||
|
||||
## Support
|
||||
|
||||
For issues during deployment:
|
||||
1. Check application logs: `sudo journalctl -u reklamator -f`
|
||||
2. Check Nginx logs: `sudo tail -f /var/log/nginx/reklamator_error.log`
|
||||
3. Verify health endpoint: `curl https://feedback.yourdomain.com/health`
|
||||
4. Review configuration files for typos
|
||||
5. Ensure all environment variables are set in `.env`
|
||||
|
||||
For additional help, consult the main README.md or open an issue on GitHub.
|
||||
Reference in New Issue
Block a user