# Critical Gotchas — Read Before Running Commands ## Unprivileged LXC containers (Proxmox) — mount limitations - **NFS and CIFS client mounts both fail** inside an unprivileged LXC even with the `mount=nfs;cifs` features flag set — "Operation not permitted." This is a hard kernel-capability limitation, not a config mistake; don't retry it. - **Fix:** mount the share on the PVE host itself (fstab, `_netdev`), then pass it into the container as an **LXC mountpoint** (`pct set -mpN /host/path,mp=/container/path`). This persists across reboots on both the host and the container. - A network share's root export directory can have restrictive permissions (e.g. Synology's `hide` NFS option sets mode `000` on the share root) that real root on the PVE host silently bypasses via DAC-override, but an unprivileged container's "root" cannot — causes `Permission denied` even though subfolders are readable. Fix on the source NAS/host, not in the container. - Passing a TUN device (needed for WireGuard-based VPN containers like gluetun) needs explicit `pct set -devN /dev/net/tun`. ## MSP360 / CloudBerry Backup - CLI binary path has a space: `/opt/local/MSP360\ Backup/bin/cbb` (legacy CLI, `plan -l/-r/-s`) or `cbbV2` (full plan/account create/edit) — always quote it. - **`dpkg-repack`ing an already-initialized install captures a package missing its own bootstrap config templates** (`cloudBackup.conf`, `wt_config.xml` get consumed/deleted from the package tree after first run). A fresh install from that repacked .deb hangs forever waiting on a file that will never appear. Fix: copy the *generated* versions of those files from the source machine's live runtime directory into the new install before first start. - **A leftover worker process from a killed/deleted plan silently blocks a new plan's worker** — the tell is near-zero CPU usage despite the job showing "Running" for many minutes. `ps aux | grep cbbWorker`, kill anything referencing an old/deleted plan ID. - Free/Freeware license self-registers via `cbbV2 license activate -f -e -c` — no manual key needed. ## PHP / OpenLiteSpeed (DO server) - **NEVER** use `lsphp -l` for syntax check — it segfaults. Use `php8.3 -l file.php` - Run CLI scripts with: `/usr/local/lsws/lsphp85/bin/lsphp /path/script.php` - If endpoint uses `ob_start()` + header.php pattern → add `ob_end_clean()` before CSV/JSON output - MySQL charset: always `utf8mb4_unicode_ci` — mixing with `general_ci` breaks JOINs (error 1267) - Cron bug: `*/3 * * * *` restarts `lshttpd` if any `.htaccess` is newer than `/usr/local/lsws/cgid` — nothing re-touches that dir after restart, causing a restart loop until you `touch /usr/local/lsws/cgid` manually. ## SSH / Networking - DO server (165.22.1.228) **cannot reach the local network** (10.48.200.x) directly. - PVE1 SSH via FortiGate DDNS works: `root@orbisne.fortiddns.com` (survives WAN IP changes, which happen fairly often — check `curl ifconfig.me` before trusting a documented WAN IP). - PVE2 has no external port forward — only reachable locally or via cluster API through PVE1. - Proxmox API port 8006 IS forwarded: `orbisne.fortiddns.com:8006`. - FortiGate admin GUI is on **port 9443**, not 443 (443/80 both redirect there). - A host's SSH interactive access can be intermittently flaky (e.g. through nested nested plink→ssh→ssh hops) while its own automated scripts using the same key work fine every time — that's usually not a real security/access problem, just a quirk of the ad-hoc path. ## JARVIS Agents - Agent config: `/etc/jarvis-agent/config.json` | Runtime state: `/var/lib/jarvis-agent/state.json` - **401 "Invalid agent key"** → state.json has a stale key. Fix: overwrite with correct `agent_id`/`api_key` from `registered_agents` table, then `systemctl restart jarvis-agent`. - Metrics stored as JSON in `metric_data` column — use `JSON_EXTRACT(metric_data,'$.cpu_percent')`, no direct columns. - **Set the container/VM's actual OS hostname correctly before first agent registration** — a hostname typo becomes part of the generated `agent_id` and creates a stale duplicate registration that needs manual DB cleanup later. ## Groq AI - Model name: `compound-beta-mini` — NOT `groq/compound-beta-mini` (that's OpenAI router syntax, 404s). ## Proxmox - `--nameserver` must be space-separated: `"8.8.8.8 1.1.1.1"` (comma causes a netplan bug). - Run commands in a VM even when SSH/network is down: `qm guest exec -- bash -c 'cmd'` (needs guest agent; also needs `export HOME=/root` for many CLI tools since guest-exec has no HOME env by default). - Run commands in an LXC: `pct exec -- bash -c 'cmd'`. ## Deploy - Repos with auto-deploy webhooks (JARVIS, web-dashboard, StreamHoard's compose isn't webhook-deployed — it's manual `git pull` + `docker compose up -d`): always `git add + commit + push` — don't forget to also trigger the Gitea mirror-sync afterward (`curl -X POST -u myron:Joker1974!!! https://gitea.orbishosting.com/api/v1/repos/myron//mirror-sync`). - The 7 DO-hosted sites have **no auto-deploy** — manual SSH + git pull only. ## API Endpoint Auth (JARVIS) - Netscan endpoint (`/api/netscan`) bypasses main auth — uses `X-Registration-Key` header. - Admin portal uses a separate PHP session name (`jarvis_admin`) — different from the main JARVIS session. - Cloudflare real IP: use `$_SERVER['HTTP_CF_CONNECTING_IP']` (or `HTTP_X_FORWARDED_FOR`), not `REMOTE_ADDR`. ## Network Scan - The JARVIS "RUN NETWORK SCAN" button doesn't scan from wherever the browser is — it queues a shell command to the PVE1 agent, which runs nmap and pushes results to `/api/netscan`. Results appear ~40s after clicking. - Chat "scan network" intent returns real DB data — never hallucinated. ## FusionPBX / VoIP See [voip.md](voip.md) for the full reference — SignalWire has several non-obvious credential-type and API-endpoint gotchas that cost real debugging time in the past; don't re-derive them from scratch. - SIP config changes need the sofia.conf Lua cache cleared, not just a service restart: `rm /var/cache/fusionpbx/.configuration.sofia.conf` ## Git push confirmation Pushing to a repo's default branch (master/main) needs an explicit, branch-named confirmation from the user — a general "yes" to "want me to push this?" isn't enough and will get blocked. Ask "confirm push directly to master?" up front to save a round-trip. If a chained `git add && commit && push` gets denied, nothing in the chain ran — re-check `git status` before retrying individual steps. ## Live log popup convention (JARVIS admin) Any server-side action triggered from JARVIS admin (setup, deploy, install, run backup) must show a live log popup — NOT a `confirm()`/`toast()`. Pattern: `openModal()` + scrollable dark log div + backend log-polling endpoint (`{lines:[], next_line:N}`) + color-coded lines (green=ok, red=error, yellow=warn) + auto-detect completion + CLOSE button that stops polling. Poll every 2s, snapshot log position before triggering the action to avoid replaying old output. `runIntentGenerator()` in the admin JS is the canonical reference implementation.