diff --git a/.gitignore b/.gitignore index d85bcc1..8a6471d 100644 --- a/.gitignore +++ b/.gitignore @@ -36,3 +36,4 @@ package-lock.json .env .env.* !.env.example +.envrc diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md index b0b4ee6..33c160f 100644 --- a/DEPLOYMENT.md +++ b/DEPLOYMENT.md @@ -1,28 +1,28 @@ # Deployment Guide -This guide explains how to deploy the Hugo-built static site to the production server at www.markusgraf.ch. +This guide explains how to deploy the Hugo-built static site to the production server via rsync over SSH, properly mirroring the content. ## Prerequisites Before deploying, ensure you have: 1. **Built the site**: Run `hugo build` to generate the `public/` directory -2. **lftp installed**: The deployment script requires the `lftp` command-line tool -3. **FTPS credentials**: You need the FTPS username and password for the server +2. **SSH access configured**: You need SSH access to your server with key-based authentication +3. **Rsync available**: The deployment script requires the `rsync` command-line tool -### Installing lftp +### Installing Rsync -If `lftp` is not installed on your system: +Rsync is typically pre-installed on most systems, but if needed: ```bash # Ubuntu/Debian -sudo apt-get install lftp +sudo apt-get install rsync -# macOS -brew install lftp +# macOS (usually pre-installed) +# No action needed, or: brew install rsync # Fedora -sudo dnf install lftp +sudo dnf install rsync ``` ## Deployment Process @@ -39,14 +39,16 @@ This generates the static files in the `public/` directory. ### 2. Set Environment Variables -Set the FTPS credentials as environment variables: +Set the SSH connection details as environment variables: ```bash -export FTPS_USER='gurix' -export FTPS_PASSWORD='your-password-here' +export SSH_USER='your-username' +export SSH_HOST='your-server.com' +export SSH_PORT='22' # Optional, defaults to 22 +export REMOTE_ROOT='/httpsdocs' # Optional, defaults to /httpsdocs ``` -**Important**: Never commit passwords to version control. Credentials should only be stored as environment variables. +**Important**: Ensure SSH key-based authentication is set up for password-less access. ### 3. Run the Deployment Script @@ -58,8 +60,9 @@ Execute the deployment script: The script will: - Check that all prerequisites are met -- Connect to www.markusgraf.ch via FTPS -- Upload all files from `public/` to `httpsdocs/` on the server +- Connect to your server via rsync over SSH +- Sync all files from `public/` to the specified remote directory +- Delete remote files that don't exist locally (true mirror behavior) - Display progress and completion status ### Complete Example @@ -70,9 +73,11 @@ Here's the full workflow: # Build the site hugo build -# Set credentials (only needed once per session) -export FTPS_USER='gurix' -export FTPS_PASSWORD='your-password' +# Set connection details (only needed once per session) +export SSH_USER='your-username' +export SSH_HOST='your-server.com' +export SSH_PORT='22' # Optional +export REMOTE_ROOT='/httpsdocs' # Optional # Deploy ./scripts/deploy.sh @@ -82,20 +87,22 @@ export FTPS_PASSWORD='your-password' The deployment script uses the following configuration: -- **Server**: www.markusgraf.ch -- **Protocol**: FTPS (FTP over SSL/TLS) -- **Username**: gurix -- **Target Directory**: httpsdocs/ +- **Protocol**: rsync over SSH +- **Default Port**: 22 (configurable via SSH_PORT) +- **Default Target Directory**: /httpsdocs (configurable via REMOTE_ROOT) - **Source Directory**: public/ +- **Sync Behavior**: Mirror mode with `--delete` flag (removes remote files not present locally) ## Security Notes -### Credential Management +### SSH Key-Based Authentication -- **Never** hardcode passwords in scripts or configuration files -- **Never** commit passwords to version control -- Use environment variables to pass credentials securely -- The deployment script never displays passwords in its output +The deployment script uses SSH/SCP, which supports key-based authentication: + +- **Set up SSH keys**: Generate an SSH key pair and add your public key to the server's `~/.ssh/authorized_keys` +- **Test SSH access**: Verify you can connect without a password: `ssh -p 22 user@server` +- **No password required**: SCP will use your SSH keys automatically +- **Never** commit private keys to version control ### Optional: Using .env Files @@ -103,7 +110,10 @@ For convenience, you can create a `.env` file (which is ignored by git): ```bash # .env -FTPS_PASSWORD=your-password +export SSH_USER='your-username' +export SSH_HOST='your-server.com' +export SSH_PORT='22' +export REMOTE_ROOT='/httpsdocs' ``` Then source it before deployment: @@ -121,58 +131,81 @@ source .env **Solution**: Run `hugo build` before deploying. -### Error: "lftp is not installed" +### Error: "rsync is not installed" -**Cause**: The `lftp` tool is not available on your system. +**Cause**: The `rsync` tool is not available on your system. -**Solution**: Install lftp using the instructions above. +**Solution**: Install rsync using the instructions above. -### Error: "FTPS_PASSWORD environment variable is not set" +### Error: "SSH_USER environment variable is not set" -**Cause**: The password environment variable hasn't been set. +**Cause**: Required environment variables haven't been set. -**Solution**: Run `export FTPS_PASSWORD='your-password'` before deploying. +**Solution**: Set the required variables: +```bash +export SSH_USER='your-username' +export SSH_HOST='your-server.com' +``` -### Connection Failures +### SSH Connection Failures -**Cause**: Network issues or incorrect credentials. +**Cause**: SSH authentication issues or network problems. **Solutions**: -- Verify your internet connection -- Check that the password is correct -- Ensure the server (www.markusgraf.ch) is accessible -- Verify firewall settings aren't blocking FTPS (port 21) +- Verify you can connect manually: `ssh -p 22 user@server` +- Check that your SSH key is added to the server's `~/.ssh/authorized_keys` +- Ensure your internet connection is working +- Verify the server is accessible +- Check firewall settings aren't blocking SSH (default port 22) -### Partial Upload Failures +### Permission Denied on Remote Directory -**Cause**: Network interruption during upload. +**Cause**: User doesn't have write permissions to the target directory. -**Solution**: Simply run the deployment script again. The `mirror` command will resume and complete the upload. +**Solution**: +- Verify the REMOTE_ROOT directory exists on the server +- Ensure your SSH user has write permissions to that directory +- Contact your server administrator if needed ## Advanced Usage -### Testing Without Deploying +### Testing SSH Connection -To test the script without actually uploading files, you can modify the `deploy()` function temporarily to use the `--dry-run` flag: +To test your SSH connection before deploying: ```bash -mirror --reverse --delete --verbose --parallel=3 --dry-run $SOURCE_DIR $TARGET_DIR; +# Test basic SSH connection +ssh -p "${SSH_PORT:-22}" "$SSH_USER@$SSH_HOST" + +# Test that you can write to the target directory +ssh -p "${SSH_PORT:-22}" "$SSH_USER@$SSH_HOST" "touch ${REMOTE_ROOT:-/httpsdocs}/test.txt && rm ${REMOTE_ROOT:-/httpsdocs}/test.txt" ``` ### Deployment from CI/CD For automated deployments from CI/CD pipelines: -1. Store `FTPS_PASSWORD` as a secret in your CI/CD system -2. Ensure the CI/CD environment has `lftp` installed -3. Run the deployment script after successful builds +1. Set up SSH key-based authentication for your CI/CD runner +2. Store `SSH_USER` and `SSH_HOST` as environment variables or secrets +3. Ensure the CI/CD environment has `rsync` installed +4. Run the deployment script after successful builds Example GitHub Actions workflow snippet: ```yaml +- name: Setup SSH key + run: | + mkdir -p ~/.ssh + echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa + chmod 600 ~/.ssh/id_rsa + ssh-keyscan -H ${{ secrets.SSH_HOST }} >> ~/.ssh/known_hosts + - name: Deploy to server env: - FTPS_PASSWORD: ${{ secrets.FTPS_PASSWORD }} + SSH_USER: ${{ secrets.SSH_USER }} + SSH_HOST: ${{ secrets.SSH_HOST }} + SSH_PORT: ${{ secrets.SSH_PORT }} + REMOTE_ROOT: ${{ secrets.REMOTE_ROOT }} run: ./scripts/deploy.sh ``` @@ -180,7 +213,7 @@ Example GitHub Actions workflow snippet: If you encounter issues not covered in this guide: -1. Check the server logs -2. Verify network connectivity to www.markusgraf.ch -3. Ensure the `httpsdocs/` directory exists on the server +1. Test SSH connectivity manually: `ssh user@server` +2. Check server logs for authentication or permission issues +3. Verify the target directory exists and has correct permissions 4. Contact your hosting provider for server-side issues diff --git a/openspec/changes/add-ftps-deployment/proposal.md b/openspec/changes/add-ftps-deployment/proposal.md index 1e00a15..1bad004 100644 --- a/openspec/changes/add-ftps-deployment/proposal.md +++ b/openspec/changes/add-ftps-deployment/proposal.md @@ -1,18 +1,21 @@ # Change Proposal: add-ftps-deployment ## Why -Currently there is no automated way to deploy the built Hugo site to the production server. Manual file transfers are error-prone and time-consuming. This change adds a deployment script to automate FTPS uploads to www.markusgraf.ch. +Currently there is no automated way to deploy the built Hugo site to the production server. Manual file transfers are error-prone and time-consuming. This change adds a deployment script to automate uploads via rsync over SSH, properly mirroring content (including deletions). ## What Changes -- Add deployment script `scripts/deploy.sh` that uploads `public/` directory via FTPS -- Use environment variable `FTPS_PASSWORD` for secure credential management -- Implement prerequisite checks (public/ exists, lftp installed, credentials set) +- Add deployment script `scripts/deploy.sh` that syncs `public/` directory via rsync +- Use environment variables `SSH_USER`, `SSH_HOST`, `SSH_PORT`, and `REMOTE_ROOT` for configuration +- Implement mirror behavior with `--delete` flag to remove remote files not present locally +- Implement prerequisite checks (public/ exists, rsync installed, credentials set) - Add error handling and progress reporting - Update documentation with deployment instructions ## Impact - Affected specs: New `deployment` capability - Affected code: New file `scripts/deploy.sh` -- Dependencies: Requires `lftp` tool to be installed -- Configuration: User must set `FTPS_PASSWORD` environment variable +- Dependencies: Requires `rsync` to be installed +- Configuration: User must set `SSH_USER` and `SSH_HOST` environment variables + - `SSH_PORT` defaults to 22 + - `REMOTE_ROOT` defaults to /httpsdocs - No changes to existing Hugo templates, content, or build process diff --git a/openspec/changes/add-ftps-deployment/specs/deployment/spec.md b/openspec/changes/add-ftps-deployment/specs/deployment/spec.md index 3bda08e..00d446c 100644 --- a/openspec/changes/add-ftps-deployment/specs/deployment/spec.md +++ b/openspec/changes/add-ftps-deployment/specs/deployment/spec.md @@ -2,29 +2,30 @@ ## ADDED Requirements -### Requirement: FTPS Deployment Script SHALL be provided -The system SHALL provide an automated deployment script that uploads the built Hugo site to the production server via FTPS. +### Requirement: Rsync Deployment Script SHALL be provided +The system SHALL provide an automated deployment script that syncs the built Hugo site to the production server via rsync over SSH, properly mirroring content. #### Scenario: User deploys site after building **Given** the Hugo site has been built successfully (public/ directory exists) -**And** the user has set the FTPS_PASSWORD environment variable +**And** the user has set the SSH_USER and SSH_HOST environment variables **When** the user runs the deployment script -**Then** the script connects to www.markusgraf.ch via FTPS using the username "gurix" -**And** uploads all files from the public/ directory to the httpsdocs/ directory on the server -**And** displays upload progress and completion status +**Then** the script connects to the specified SSH_HOST via rsync over SSH +**And** syncs all files from the public/ directory to the REMOTE_ROOT directory on the server +**And** deletes remote files that don't exist locally (mirror behavior) +**And** displays sync progress and completion status **And** exits with status code 0 on success #### Scenario: Script fails when credentials are missing **Given** the Hugo site has been built -**And** the FTPS_PASSWORD environment variable is not set +**And** the SSH_USER or SSH_HOST environment variable is not set **When** the user runs the deployment script -**Then** the script displays an error message explaining the missing credential +**Then** the script displays an error message explaining the missing configuration **And** exits with a non-zero status code **And** does not attempt to connect to the server #### Scenario: Script handles connection failures gracefully -**Given** the FTPS_PASSWORD is set correctly -**And** the network connection to www.markusgraf.ch is unavailable or fails +**Given** the SSH_USER and SSH_HOST are set correctly +**And** the network connection to the server is unavailable or SSH authentication fails **When** the user runs the deployment script **Then** the script displays a clear error message about the connection failure **And** exits with a non-zero status code @@ -33,18 +34,19 @@ The system SHALL provide an automated deployment script that uploads the built H ### Requirement: Secure Credential Management SHALL be enforced The deployment process SHALL handle credentials securely without exposing them in version control or script output. -#### Scenario: Credentials stored as environment variables +#### Scenario: Configuration stored as environment variables **Given** the user needs to deploy the site **When** the user reviews the deployment documentation -**Then** the documentation instructs them to set FTPS_PASSWORD as an environment variable -**And** the deployment script reads credentials only from environment variables +**Then** the documentation instructs them to set SSH_USER, SSH_HOST, SSH_PORT, and REMOTE_ROOT as environment variables +**And** the deployment script reads configuration only from environment variables **And** credentials are never hardcoded in scripts or configuration files +**And** SSH key-based authentication is used for secure, password-less access #### Scenario: Script does not expose credentials in output **Given** the deployment script is running **When** the script displays status messages or logs -**Then** the password is never displayed in plain text -**And** connection strings mask the password portion +**Then** SSH keys or passwords are never displayed in plain text +**And** connection strings show only the host and user information **And** error messages do not reveal credential values ### Requirement: Deployment Status Feedback SHALL be provided @@ -84,6 +86,13 @@ The deployment script SHALL verify that prerequisites are met before attempting #### Scenario: Script checks for required tools **Given** the deployment script starts **When** it performs prerequisite checks -**Then** it verifies that lftp is installed and available -**And** displays an installation message if lftp is missing +**Then** it verifies that rsync is installed and available +**And** displays an installation message if rsync is missing **And** exits with an error if required tools are unavailable + +#### Scenario: Script supports configurable ports and remote directories +**Given** the user needs to deploy to a non-standard SSH port or directory +**When** the user sets SSH_PORT and REMOTE_ROOT environment variables +**Then** the script uses the specified port instead of the default (22) +**And** the script uploads to the specified remote directory instead of the default (/httpsdocs) +**And** if these variables are not set, the script uses sensible defaults diff --git a/openspec/changes/add-ftps-deployment/tasks.md b/openspec/changes/add-ftps-deployment/tasks.md index 3908fa6..4828d23 100644 --- a/openspec/changes/add-ftps-deployment/tasks.md +++ b/openspec/changes/add-ftps-deployment/tasks.md @@ -10,30 +10,31 @@ ### 2. Implement prerequisite checks - [x] Check for existence and non-empty state of `public/` directory -- [x] Verify `lftp` is installed and available in PATH -- [x] Check that `FTPS_PASSWORD` environment variable is set +- [x] Verify `rsync` is installed and available in PATH +- [x] Check that `SSH_USER` and `SSH_HOST` environment variables are set - [x] Display clear error messages for any missing prerequisites - **Validates**: Script exits early with helpful errors when prerequisites are missing -### 3. Implement FTPS connection logic -- [x] Configure lftp connection to www.markusgraf.ch with username "gurix" -- [x] Use `FTPS_PASSWORD` environment variable for authentication -- [x] Set FTPS-specific lftp settings (SSL/TLS requirements) -- [x] Implement connection timeout and retry logic -- **Validates**: Script can establish FTPS connection with correct credentials +### 3. Implement SSH/rsync connection logic +- [x] Configure rsync to connect using SSH_USER and SSH_HOST +- [x] Support configurable SSH_PORT (defaults to 22) +- [x] Support configurable REMOTE_ROOT (defaults to /httpsdocs) +- [x] Use SSH key-based authentication for secure, password-less access +- **Validates**: Script can establish SSH connection with correct credentials -### 4. Implement file upload functionality -- [x] Use lftp mirror command to upload `public/` contents to `httpsdocs/` -- [x] Configure upload to preserve file permissions and timestamps -- [x] Enable parallel transfers for improved performance -- [x] Handle special files (symlinks, hidden files) appropriately -- **Validates**: All files from public/ are correctly uploaded to httpsdocs/ +### 4. Implement file sync functionality +- [x] Use rsync command to sync `public/` contents to remote directory +- [x] Configure sync to preserve file permissions and timestamps (-a flag) +- [x] Enable compression during transfer (-z flag) +- [x] Implement mirror behavior with --delete flag (removes remote files not present locally) +- [x] Handle all file types appropriately +- **Validates**: All files from public/ are correctly synced to the remote directory and old files are removed ### 5. Add progress and status reporting - [x] Display connection status messages -- [x] Show upload progress (file counts, current file being uploaded) -- [x] Report upload completion with summary statistics -- [x] Ensure password is never displayed in output +- [x] Show sync progress through rsync verbose output +- [x] Report sync completion with summary +- [x] Ensure SSH keys or passwords are never displayed in output - **Validates**: User receives clear feedback during deployment process ### 6. Implement error handling @@ -45,9 +46,9 @@ ### 7. Add script documentation - [x] Add header comments explaining script purpose and usage -- [x] Document required environment variables +- [x] Document required environment variables (SSH_USER, SSH_HOST, SSH_PORT, REMOTE_ROOT) - [x] Include example usage in comments -- [x] Add inline comments for complex lftp commands +- [x] Add inline comments for rsync command options - **Validates**: Script is self-documenting for future maintenance ### 8. Update project documentation @@ -58,10 +59,11 @@ - **Validates**: User documentation exists and covers deployment process ### 9. Test deployment script -- [x] Test with missing prerequisites (no public/, no lftp, no password) -- [x] Test with incorrect credentials -- [x] Test successful deployment with valid credentials -- [x] Verify uploaded files match local public/ directory +- [x] Test with missing prerequisites (no public/, no rsync, no SSH_USER/SSH_HOST) +- [x] Test with SSH authentication issues +- [x] Test successful deployment with valid credentials and SSH keys +- [x] Verify synced files match local public/ directory +- [x] Verify remote files are deleted when removed locally (mirror behavior) - **Validates**: Script behaves correctly in success and failure scenarios ### 10. Create .gitignore entry for environment files diff --git a/scripts/deploy.sh b/scripts/deploy.sh index e94aa88..28ee876 100755 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -1,24 +1,26 @@ #!/bin/bash # -# FTPS Deployment Script for markusgraf.ch +# Rsync Deployment Script for markusgraf.ch # # This script deploys the Hugo-built static site to the production server -# via FTPS (FTP over SSL/TLS). +# via rsync over SSH, mirroring the content (including deletions). # # Prerequisites: # - Hugo site must be built (public/ directory exists) -# - lftp must be installed -# - FTPS_PASSWORD environment variable must be set +# - SSH access to the server must be configured +# - SSH_USER, SSH_HOST environment variables must be set # # Usage: -# export FTPS_PASSWORD='your-password' +# export SSH_USER='your-username' +# export SSH_HOST='your-server.com' +# export SSH_PORT='22' # optional, defaults to 22 +# export REMOTE_ROOT='/httpsdocs' # optional, defaults to /httpsdocs # ./scripts/deploy.sh # # Server Details: -# - Server: www.markusgraf.ch -# - Username: gurix -# - Target Directory: httpsdocs/ +# - Default Target Directory: /httpsdocs +# - Default Port: 22 # - Source Directory: public/ # @@ -32,8 +34,8 @@ YELLOW='\033[1;33m' NC='\033[0m' # No Color # Configuration -FTPS_SERVER="www.markusgraf.ch" -TARGET_DIR="httpsdocs" +SSH_PORT="${SSH_PORT:-22}" +REMOTE_ROOT="${REMOTE_ROOT:-/httpsdocs}" SOURCE_DIR="public" # @@ -80,71 +82,64 @@ check_prerequisites() { error "Directory '$SOURCE_DIR' is empty. Please run 'hugo build' first." fi - # Check if lftp is installed - if ! command -v lftp &> /dev/null; then - error "lftp is not installed. Please install it first: - Ubuntu/Debian: sudo apt-get install lftp - macOS: brew install lftp - Fedora: sudo dnf install lftp" + # Check if rsync is installed + if ! command -v rsync &> /dev/null; then + error "rsync is not installed. Please install rsync: + Ubuntu/Debian: sudo apt-get install rsync + macOS: rsync is pre-installed + Fedora: sudo dnf install rsync" fi - # Check if FTPS_PASSWORD is set - if [ -z "${FTPS_PASSWORD:-}" ]; then - error "FTPS_PASSWORD environment variable is not set. + # Check if SSH_USER is set + if [ -z "${SSH_USER:-}" ]; then + error "SSH_USER environment variable is not set. Please set it before running this script: - export FTPS_PASSWORD='your-password'" + export SSH_USER='your-username'" fi - # Check if FTPS_USER is set - if [ -z "${FTPS_USER:-}" ]; then - error "FTPS_USER environment variable is not set. + # Check if SSH_HOST is set + if [ -z "${SSH_HOST:-}" ]; then + error "SSH_HOST environment variable is not set. Please set it before running this script: - export FTPS_USER='user'" + export SSH_HOST='your-server.com'" fi success "All prerequisites met." } # -# Deploy site to server via FTPS +# Deploy site to server via rsync # deploy() { - info "Starting deployment to $FTPS_SERVER..." - info "Connecting as user: $FTPS_USER" - info "Uploading from: $SOURCE_DIR/" - info "Target directory: $TARGET_DIR/" + info "Starting deployment to $SSH_HOST..." + info "Connecting as user: $SSH_USER" + info "Port: $SSH_PORT" + info "Syncing from: $SOURCE_DIR/" + info "Target directory: $REMOTE_ROOT/" - # Use lftp to upload files via FTPS - # Note: Password is passed via environment variable to avoid exposure in process list - lftp -e " - set ftps:initial-prot ''; - set ftp:ssl-force true; - set ftp:ssl-protect-data true; - set ssl:verify-certificate no; - set net:timeout 30; - set net:max-retries 3; - set net:reconnect-interval-base 5; - open ftps://$FTPS_USER:$FTPS_PASSWORD@$FTPS_SERVER; - cd $TARGET_DIR || mkdir -p $TARGET_DIR; - lcd $SOURCE_DIR; - mirror --reverse --delete --verbose --parallel=3; - bye - " + # Use rsync to mirror the content + # -a: archive mode (preserves permissions, timestamps, etc.) + # -v: verbose output + # -z: compress data during transfer + # --delete: delete files on remote that don't exist locally (mirror behavior) + # -e: specify SSH with custom port + rsync -avz --delete -e "ssh -p $SSH_PORT" "$SOURCE_DIR/" "$SSH_USER@$SSH_HOST:$REMOTE_ROOT/" local exit_code=$? if [ $exit_code -eq 0 ]; then success "Deployment completed successfully!" - info "Your site is now live at https://markusgraf.ch" + info "Files have been synced to $SSH_HOST:$REMOTE_ROOT" return 0 else error "Deployment failed with exit code $exit_code. Troubleshooting: - - Verify your password is correct - - Check network connectivity to $FTPS_SERVER - - Ensure the target directory '$TARGET_DIR' exists on the server - - Check server logs for additional details" + - Verify SSH credentials and key-based authentication is set up + - Check network connectivity to $SSH_HOST + - Ensure the target directory '$REMOTE_ROOT' exists on the server + - Verify SSH_USER has write permissions to $REMOTE_ROOT + - Try connecting manually: ssh -p $SSH_PORT $SSH_USER@$SSH_HOST" fi } @@ -152,7 +147,7 @@ Troubleshooting: # Main execution # main() { - info "=== FTPS Deployment Script ===" + info "=== Rsync Deployment Script ===" info "" check_prerequisites