# Kinometric Server Migration Runbook

**Purpose:** Migrate Kinometric from current Linode (unencrypted disk) to a new Linode instance with disk encryption enabled.

**Audience:** Claude Code running on the new server, plus the human operator for Linode Cloud Manager steps.

**Estimated time:** 1-2 hours
**Downtime:** ~2 minutes (IP swap only)

---

## Architecture

```
OLD SERVER (current)                    NEW SERVER (target)
  Linode, Ubuntu 24.04                    Linode, Ubuntu 24.04
  No disk encryption                      Disk encryption ON
  IP: kinometric.com                      Temp IP (swapped later)
       │                                       │
       └── Code: git@gitlab-backend ───────────┘  (same repo)
       └── Data: restic on NAS ────────────────┘  (restore from backup)
       └── Secrets: manual transfer ───────────┘  (via SSH/WG)
```

---

## What Carries Over vs What Gets Rebuilt

| Source | Method | Size |
|--------|--------|------|
| PHP/Python/HTML code | `git clone` from GitLab | ~2 MB |
| Composer packages (vendor/) | `composer install` | ~15 MB (rebuilt) |
| Python venv | `python3 -m venv` + `pip install` | ~100 MB (rebuilt) |
| PostgreSQL (kinosave) | `restic restore` or `pg_dump` + `pg_restore` | ~12 MB |
| CSV sensor data | `restic restore` or `rsync` | ~280 MB |
| Config files (not in git) | Manual copy via SSH | ~2 KB |
| Secrets | Manual paste via SSH | ~200 bytes |
| SSL certs | `certbot` (new cert, same domain) | Auto |
| tuning_cache.npz | Regenerated by rebuild script | ~10 MB |
| APKs in builds/ | Copy current APK only | ~70 MB |

### What Does NOT Carry Over (dead weight)

These exist on the old server but are not migrated:

**Root-level binaries (~1.2 GB):** 13 loose APK files, kinoVideo.mov, .snaps screenshot, generated CSVs/XLSX — all superseded by builds/ directory or not needed.

**Dead PHP files (~25 files):** dbpage.php, dbpagenew.php (old DB connectors), provider_filter.php + backAddProvider.php (provider system dropped Feb 2026), fpdf.php + getPDF.php + linegraph.php + PDF_Combined.php + PDF_Ellipse.php (legacy FPDF, replaced by jsPDF/Dart), tempBackInsertQuestion.php, getFallRisk.php, getTestResults.php, checkVersion.php, patientLogin.php, patientNew.php, patientold.php, test.php, test-back-auth.php, hash_password.php, disable_2fa.php, setup_2fa.php, verify_2fa.php (2FA not implemented), index-old.php, old.html, videoDownload.html, info.php (phpinfo security risk), analyze.py, analyze4.py, analyzetest5.py, findStable.py (replaced by test/tuning_engine.py), commit.sh, tmp_kino.sh, tmp_secure.sh, test_ai_analysis.sh, test_email.sh, PHP_INTEGRATION_README.md, oldversion.json, output-metadata.json, mail_config.json, deploy_token.json.

**Dead directories (~10 MB):** docs1/ (old athena drafts), excel_images/, processed/, new/, tutorial/ + doc/ + makefont/ + font/ (FPDF library), temp12-18/, phpgangsta/ (old 2FA), "ol 9-2-24"/, old/, oauth/, backups/, learn/ (in web root, real data at /home/kinometric/), __pycache__/, tmpsh/.

**downloads/ (452 MB):** Old APK dump. Replaced by builds/ + app_builds table.

**Legacy PostgreSQL:** PG 14 and PG 12 are installed on old server. New server only installs PG 16.

**xdebug:** Installed on old server. Do NOT install on new server (security risk in production).

---

## Human Steps (Linode Cloud Manager)

These steps require browser access to cloud.linode.com.

### Step H1: Create New Linode
1. Cloud Manager → Create Linode
2. **Image:** Ubuntu 24.04 LTS
3. **Region:** Same as current (required for IP transfer)
4. **Plan:** Same as current
5. **Encrypt Disk: ON**
6. Set root password, add SSH key
7. Note the temporary IP

### Step H2: Initial SSH Setup
```bash
ssh root@NEW_TEMP_IP
adduser efsi
usermod -aG sudo efsi
usermod -aG www-data efsi

# Add your SSH key for efsi
mkdir -p /home/efsi/.ssh
cp ~/.ssh/authorized_keys /home/efsi/.ssh/
chown -R efsi:efsi /home/efsi/.ssh
chmod 700 /home/efsi/.ssh
```

### Step H3: Install Claude Code on New Server
SSH in as efsi, install Claude Code CLI. Then point it at this migration doc.

### Step H4: IP Swap (after Claude Code finishes all phases)
1. Cloud Manager → Old Linode → Networking → IP Transfer
2. Swap IPs between old and new
3. Immediately run certbot on new server (Claude Code does this)

### Step H5: Post-Migration (24-48 hours later)
1. Confirm new server is stable
2. Disable old server's backup timer and cron
3. Delete old Linode

---

## Claude Code Phases (run in order on new server)

Claude Code should read this doc, then execute each phase. Each phase ends with a verification step — do not proceed until verification passes.

### Phase 1: System Packages

```bash
apt update && apt upgrade -y

# Core stack
apt install -y \
  apache2 libapache2-mod-php8.4 \
  php8.4 php8.4-pgsql php8.4-curl php8.4-gd php8.4-mbstring \
  php8.4-xml php8.4-zip php8.4-soap php8.4-imagick php8.4-sodium \
  php8.4-intl php8.4-readline \
  postgresql-16 postgresql-client-16 \
  python3 python3-venv python3-pip \
  git curl certbot python3-certbot-apache \
  wireguard wireguard-tools \
  restic ufw qpdf

# If PHP 8.4 not in default repos: add-apt-repository ppa:ondrej/php first

# Apache modules
a2enmod rewrite ssl headers deflate
```

**Do NOT install xdebug.**

**Verify:**
```bash
apache2 -v && php -v && psql --version && python3 --version && restic version && wg --version
```

### Phase 2: PostgreSQL

```bash
# Create user and database
sudo -u postgres psql -c "CREATE USER efsi WITH PASSWORD 'efsi1';"
sudo -u postgres psql -c "CREATE DATABASE kinosave OWNER efsi;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE kinosave TO efsi;"
```

Configure `/etc/postgresql/16/main/postgresql.conf`:
```
listen_addresses = 'localhost'
port = 5432
```

Configure `/etc/postgresql/16/main/pg_hba.conf` — add:
```
local   all   efsi   md5
host    all   efsi   127.0.0.1/32   md5
```

```bash
systemctl restart postgresql
```

Create `.pgpass`:
```bash
su - efsi
echo "localhost:5432:kinosave:efsi:efsi1" > ~/.pgpass
chmod 600 ~/.pgpass
```

**Note:** Old server used port 5433 (due to legacy PG14). New server uses standard 5432. Check `dbpage2.php` after git pull — if it hardcodes a port, update it.

**Verify:**
```bash
PGPASSWORD=efsi1 psql -U efsi -d kinosave -c "SELECT 1;"
```

### Phase 3: GitLab SSH + Clone

```bash
su - efsi
mkdir -p ~/.ssh && chmod 700 ~/.ssh
```

The GitLab SSH key (`id_ed25519_gitlab_backend`) must be transferred from the old server or regenerated and added to GitLab.

```bash
cat > ~/.ssh/config << 'EOF'
Host gitlab-backend
  HostName git.efsi.com
  Port 2222
  User git
  IdentityFile ~/.ssh/id_ed25519_gitlab_backend

Host git.efsi.com
  Port 2222
  User git
  IdentityFile ~/.ssh/id_ed25519_gitlab_backend
EOF
chmod 600 ~/.ssh/config
```

Clone:
```bash
cd /var/www
sudo git clone git@gitlab-backend:kinometric/backend.git kinometric
sudo chown -R efsi:www-data kinometric
```

**Verify:**
```bash
ls /var/www/kinometric/CLAUDE.md && echo "Git clone OK"
```

### Phase 4: Dependencies

```bash
# Composer
cd /var/www/kinometric
curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer
composer install --no-dev

# Python venv
python3 -m venv venv
source venv/bin/activate
pip install numpy pandas matplotlib openpyxl lxml psycopg2-binary python-docx pymupdf pillow
deactivate
```

**Verify:**
```bash
php -r "require 'vendor/autoload.php'; echo 'Composer OK';"
source venv/bin/activate && python3 -c "import numpy, pandas; print('Python OK')" && deactivate
```

### Phase 5: WireGuard

```bash
sudo mkdir -p /etc/wireguard && sudo chmod 700 /etc/wireguard
wg genkey | sudo tee /etc/wireguard/privatekey | wg pubkey | sudo tee /etc/wireguard/publickey
sudo chmod 600 /etc/wireguard/privatekey

PRIVKEY=$(sudo cat /etc/wireguard/privatekey)

sudo bash -c "cat > /etc/wireguard/wg0.conf << WGEOF
[Interface]
Address = 10.10.0.10/24
PrivateKey = $PRIVKEY

[Peer]
PublicKey = rxyoR6aJlUJPpNUeCrpxp7ydRk3psUxRcnMLVsWv/Sw=
Endpoint = serv.efsi.com:51820
AllowedIPs = 192.168.3.0/24, 10.10.0.0/24
PersistentKeepalive = 25
WGEOF"

sudo chmod 600 /etc/wireguard/wg0.conf
```

**Coordinate with Kinometric-home Claude:**
1. Share new public key: `sudo cat /etc/wireguard/publickey`
2. Kinometric-home Claude updates the peer in efsiServer's wg0.conf (replace old pubkey)
3. Then:
```bash
sudo wg-quick up wg0
sudo systemctl enable wg-quick@wg0
```

**Verify:**
```bash
ping -c 3 10.10.0.1 && ping -c 3 192.168.3.8
```

### Phase 6: Restore Data from Restic

Transfer secrets from old server (via WireGuard tunnel or SSH):
```bash
# Restic credentials
sudo mkdir -p /etc/restic && sudo chmod 700 /etc/restic
# Paste repo password into /etc/restic/repo.pass
# Paste NAS HTTP password into /etc/restic/nas_http.pass
sudo chmod 400 /etc/restic/repo.pass /etc/restic/nas_http.pass
```

Restore:
```bash
export RESTIC_PASSWORD_FILE=/etc/restic/repo.pass
HTTP_PW=$(sudo cat /etc/restic/nas_http.pass)
export RESTIC_REPOSITORY="rest:http://linode:${HTTP_PW}@192.168.3.8:8000/linode/kinometric/"

# Check latest snapshot
restic snapshots --latest 1

# Restore database dump
mkdir -p /tmp/restore
restic restore latest --target /tmp/restore --include "/tmp/kinosave_backup.dump"
PGPASSWORD=efsi1 pg_restore -U efsi -d kinosave /tmp/restore/tmp/kinosave_backup.dump
rm -rf /tmp/restore

# Restore CSV data
sudo mkdir -p /home/kinometric/data /home/kinometric/learn/allfiles
sudo chown -R efsi:www-data /home/kinometric
restic restore latest --target / --include "/home/kinometric/data" --include "/home/kinometric/learn/allfiles"
```

**Verify:**
```bash
PGPASSWORD=efsi1 psql -U efsi -d kinosave -c "SELECT count(*) FROM patients;"
# Should return 393+
ls /home/kinometric/learn/allfiles/ | wc -l
# Should return 833+
```

### Phase 7: Secrets and Config (not in git)

These files are excluded from git. Transfer from old server via SSH or paste manually.

```bash
# Athena config (contains encrypted secret)
# From old server: cat /var/www/kinometric/athena_config.json
# Paste into /var/www/kinometric/athena_config.json
chmod 640 /var/www/kinometric/athena_config.json
chown efsi:www-data /var/www/kinometric/athena_config.json

# Mail config (Microsoft Graph API credentials)
# From old server: cat /var/www/kinometric/mail_graph_config.json
# Paste into /var/www/kinometric/mail_graph_config.json
chmod 640 /var/www/kinometric/mail_graph_config.json
chown efsi:www-data /var/www/kinometric/mail_graph_config.json

# Athena encryption key
sudo mkdir -p /etc/kinometric
sudo chmod 750 /etc/kinometric
sudo chown root:www-data /etc/kinometric
# From old server: sudo cat /etc/kinometric/athena_key
# Paste into /etc/kinometric/athena_key
sudo chmod 640 /etc/kinometric/athena_key
sudo chown root:www-data /etc/kinometric/athena_key
```

**Secrets inventory (all must be transferred):**

| Secret | Old Location | Notes |
|--------|-------------|-------|
| Athena config | `/var/www/kinometric/athena_config.json` | Contains encrypted client_secret |
| Mail config | `/var/www/kinometric/mail_graph_config.json` | MS Graph API creds |
| Athena key | `/etc/kinometric/athena_key` | 44 bytes, sodium encryption key |
| Restic repo password | `/etc/restic/repo.pass` | Backup encryption |
| NAS HTTP password | `/etc/restic/nas_http.pass` | NAS auth, user: linode |
| GitLab SSH key | `~/.ssh/id_ed25519_gitlab_backend` | Or regenerate + add to GitLab |
| PostgreSQL password | `~/.pgpass` | Created fresh in Phase 2 |

**Never transfer secrets over the public internet.** Use WireGuard tunnel or paste via SSH session.

### Phase 8: Directory Structure

```bash
# Log directories
sudo mkdir -p /var/www/kinometric/logs/archives
sudo chown -R efsi:www-data /var/www/kinometric/logs

# Builds directory
sudo mkdir -p /var/www/kinometric/builds
sudo chown efsi:www-data /var/www/kinometric/builds

# Copy current APK from old server (only the active build)
# From old server: ls /var/www/kinometric/builds/
# rsync just the current APK
```

### Phase 9: Apache + PHP Config

```bash
sudo cat > /etc/apache2/sites-available/kinometric.conf << 'EOF'
<VirtualHost *:443>
    ServerAdmin charlie@efsi.com
    ServerName kinometric.com
    ServerAlias www.kinometric.com
    DocumentRoot /var/www/kinometric

    <Directory /var/www/kinometric>
        AllowOverride All
        Require all granted
    </Directory>

    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/kinometric.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/kinometric.com/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

<VirtualHost *:80>
    ServerName kinometric.com
    ServerAlias www.kinometric.com
    RewriteEngine On
    RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [R=301,L]
</VirtualHost>
EOF

sudo a2ensite kinometric.conf
sudo a2dissite 000-default.conf
```

PHP config — set in `/etc/php/8.4/apache2/php.ini`:
```ini
display_errors = Off
log_errors = On
error_log = /var/log/php_errors.log
upload_max_filesize = 50M
post_max_size = 50M
max_execution_time = 60
```

**Do not restart Apache yet** — SSL cert needed first (comes after IP swap).

### Phase 10: Firewall

```bash
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
```

### Phase 11: Backup System

```bash
# Local restic repo
sudo mkdir -p /var/backups/restic-local && sudo chmod 700 /var/backups/restic-local
sudo RESTIC_PASSWORD_FILE=/etc/restic/repo.pass restic -r /var/backups/restic-local init

# Backup runner infrastructure
sudo mkdir -p /var/www/_backup/runner /var/www/_backup/manifests /var/www/_backup/runbooks
sudo mkdir -p /var/log/backup/kinometric
sudo chmod 750 /var/www/_backup /var/log/backup/kinometric

# Copy runner scripts (these are in tmpsh/ in git, or /var/www/_backup/runner/ on old server)
sudo cp /var/www/kinometric/tmpsh/backup_kinometric.sh /var/www/_backup/runner/
sudo cp /var/www/kinometric/tmpsh/backup_alert.php /var/www/_backup/runner/
sudo chmod 750 /var/www/_backup/runner/backup_kinometric.sh
sudo chmod 640 /var/www/_backup/runner/backup_alert.php

# Update backup script if PG port changed (5433 → 5432)
# Check: grep -n "5433" /var/www/_backup/runner/backup_kinometric.sh

# Systemd units
sudo cp /var/www/kinometric/tmpsh/backup-kinometric.service /etc/systemd/system/
sudo cp /var/www/kinometric/tmpsh/backup-kinometric.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable backup-kinometric.timer
sudo systemctl start backup-kinometric.timer
```

**Do NOT install the old cron job** (`0 2 * * * kinometric_backup.sh`). The systemd timer replaces it.

**Verify:**
```bash
systemctl list-timers | grep backup
```

### Phase 12: Pre-Swap Verification

Run all checks before requesting the IP swap:

```bash
# Database
PGPASSWORD=efsi1 psql -U efsi -d kinosave -c "SELECT count(*) FROM patients;"

# PHP + DB connection
php -r "include '/var/www/kinometric/dbpage2.php'; echo 'DB OK';"

# Python scoring
source /var/www/kinometric/venv/bin/activate
python3 /var/www/kinometric/test/test_tuning_unit.py
deactivate

# WireGuard
ping -c 1 192.168.3.8

# Restic
sudo RESTIC_PASSWORD_FILE=/etc/restic/repo.pass restic -r /var/backups/restic-local snapshots

# Apache config test (will warn about missing SSL cert — that's OK pre-swap)
sudo apache2ctl configtest
```

**Report results to the human operator. If all pass, request IP swap (Step H4).**

### Phase 13: Post-Swap (immediately after IP transfer)

```bash
# Get SSL cert (domain now points to this server)
sudo certbot --apache -d kinometric.com -d www.kinometric.com

# Restart Apache with SSL
sudo systemctl restart apache2

# Run first backup
sudo systemctl start backup-kinometric.service
sudo journalctl -u backup-kinometric.service --no-pager | tail -20
```

### Phase 14: Full Verification

```bash
# API test suite
curl -s https://kinometric.com/test/test_api.php | tail -5

# Cross-validation (quick)
source /var/www/kinometric/venv/bin/activate
python3 /var/www/kinometric/test/tuning-test/cross_validate.py --limit 50
deactivate

# Athena connectivity
# Use SPA Athena tab or curl the proxy endpoint with valid credentials

# Verify backup alert email works
php /var/www/_backup/runner/backup_alert.php "[TEST] Migration Complete" "New server is live."
```

---

## Rollback Plan

If anything goes wrong after IP swap:
1. Swap IPs back in Linode Cloud Manager (2 minutes)
2. Old server is immediately live — it was never modified
3. No data loss

Keep old server running 48 hours after migration.

---

## Post-Migration Cleanup (on old server)

After 48 hours of stable operation on the new server:
1. Disable backup timer: `sudo systemctl stop backup-kinometric.timer && sudo systemctl disable backup-kinometric.timer`
2. Disable cron: `crontab -e` and comment out the backup line
3. Disable WireGuard: `sudo systemctl stop wg-quick@wg0 && sudo systemctl disable wg-quick@wg0`
4. **Kinometric-home Claude:** Remove old WireGuard peer key
5. Delete old Linode when confident