ai-context/jarvis.md

# JARVIS System Reference Card

## Access
- **Dashboard:** https://jarvis.orbishosting.com (login: myron / Joker1974!!!)
- **Admin portal:** https://jarvis.orbishosting.com/admin/ (same login)
- **DB:** `jarvis_db` on VM 211 localhost — user: `jarvis_user` / `J4rv1s_Pr0t0c0l_2026!`
- **GitHub repo:** `myronblair/jarvis` (private; auto-deploy on push to `master`)

## Server Location
JARVIS runs on **PVE1 VM 211** at `10.48.200.211`.
- **Stack:** nginx / PHP 8.3 / MariaDB / Redis / Arc Reactor
- **Web root:** `/var/www/jarvis/` — this directory **is** the live git repo, push from it directly
- **TLS:** terminated by NPM (VM105, 10.48.200.200), proxy_host → `10.48.200.211:80`, Let's Encrypt cert. **The old `:1972` port reference is wrong/dead** — use plain `http://jarvis.orbishosting.com` (port 80) or the HTTPS URL above. GitHub webhook is `https://jarvis.orbishosting.com/webhook.php`.

## File Structure (on VM 211 at /var/www/jarvis/)
```
public_html/
  index.html          — main Iron Man HUD (all UI)
  api.php             — API router
  admin/index.php     — admin portal (single ~5000-line PHP+JS file, deliberately left unsplit — see gotchas)
  admin/downloads/INFRASTRUCTURE-REFERENCE.md — master live infra doc, gitignored, edit here first
  agent/              — agent installers
api/
  config.php          — all credentials/constants (gitignored)
  lib/db.php           — JarvisDB class (query/execute/single/insert)
  lib/kb_engine.php    — KBEngine intent matching
  endpoints/
    agent.php          — agent registration/heartbeat/metrics/commands
    chat.php           — tiered chat: KB → action intents → Ollama → Groq → Claude
    network.php        — network device list + scan endpoint
    netscan.php        — push endpoint for PVE1 nmap results (uses X-Registration-Key, bypasses main auth)
    stats_cache.php    — every 5min cron: Proxmox cluster API, HA, weather, news
    facts_collector.php — every 3min cron: system stats, site health
    kb_intent_generator.php — every 6h cron, self-expanding knowledge base (see below)
    alerts.php         — alert CRUD + auto-generate
deploy/
  reactor.py           — Arc Reactor source (copy to /opt/jarvis-arc/reactor.py + `systemctl restart jarvis-arc` to deploy)
```

## Arc Reactor (AI routing daemon)
- **Daemon:** `/opt/jarvis-arc/reactor.py` (Python venv), service: `jarvis-arc`, port 7474
- **Logs:** `/var/log/jarvis/arc.log` (setup log: `arc-setup.log`)
- **Vision cascade:** Claude API first; if Claude credits depleted, falls back to Ollama `llava:7b` (config via `/etc/systemd/system/jarvis-arc.service.d/vision.conf` → `OLLAMA_VISION_MODEL=llava:7b`)
- Secrets (Claude/Groq keys, DB pass, Gmail/iCloud app passwords) live in `/etc/jarvis-arc/reactor.env` (root:www-data 0640, loaded via systemd EnvironmentFile drop-in) — NOT in the repo.

## AI tiers (chat + reactor)
```
Tier 0.7: KB intents / planner
Tier 1:   Knowledge Base (MySQL, self-expanding — see KB Generator below)
Tier 1.5: Ollama llama3.1:8b (10.48.200.210:11434) — CPU-only, ~30s+/reply, can 504 on default 60s nginx timeout
Vision:   Ollama llava:7b (via Arc Reactor vision cascade)
Tier 2:   Groq (model name: compound-beta-mini — NOT groq/compound-beta-mini, that 404s)
Tier 3:   Claude API (fallback; credits need periodic refill at console.anthropic.com)
```
Groq is effectively the fast/primary path when Claude credits are depleted — Ollama is too slow for interactive chat (504s).

## Agent System
- Agents register via `/api/agent` using a shared **registration key** (rotated multiple times historically — always read the current value from `/var/www/jarvis/api/config.php` `AGENT_REGISTRATION_KEY`, don't trust an old hardcoded value from any doc). Per-agent heartbeats then use their own `api_key` from the `registered_agents` table, not the shared reg key.
- **Agent script:** `/opt/jarvis-agent/jarvis-agent.py` (Python3, no external deps), config at `/etc/jarvis-agent/config.json`:
  ```json
  {
    "jarvis_url": "http://10.48.200.211",
    "ssl_verify": false,
    "registration_key": "<current key>",
    "hostname": "<host>",
    "agent_type": "linux",
    "poll_interval": 30,
    "heartbeat_every": 10,
    "watch_services": ["<systemd unit names to health-check>"]
  }
  ```
  Systemd unit `jarvis-agent.service` (Restart=always). State file `/var/lib/jarvis-agent/state.json`.
- **401 "Invalid agent key"** → state.json has a stale key; delete it and restart the service to force re-registration, or manually overwrite with the correct `agent_id`/`api_key` from the DB.
- Agent auto-detects capabilities: `docker` (if docker binary present), `proxmox`, `ollama`, `homeassistant`.
- **Known registered agents (not exhaustive, check `registered_agents` table for current truth):** PVE1, PVE2, NetworkBackup, Home Assistant, Homebridge, NovaCPX, Ollama, StreamHoard/CT104 (added 2026-07-23, hostname `streamhoard104`). Jellyfin/MediaStack/WireGuard agents no longer exist (VMs destroyed).

## Network Scanning
- PVE1 cron `*/3 * * * *` runs `jarvis-netscan.sh` → nmap → POSTs JSON to `/api/netscan` with `X-Registration-Key` header.
- Chat "scan network" intent returns real DB data, never hallucinated.

## Chat Architecture / DB
```sql
registered_agents    — agent_id, hostname, agent_type, ip_address, api_key, status, last_seen, version
agent_metrics        — agent_id, metric_type, metric_data(JSON), recorded_at  -- use JSON_EXTRACT, no direct cpu/mem columns
agent_commands       — agent_id, command_type, command_data(JSON), status(pending/delivered)
network_devices      — ip, mac, hostname, alias, device_type, status, last_seen
alerts               — alert_type, title, message, severity, resolved
kb_facts             — category, fact_key, fact_value  -- fact_value is MEDIUMTEXT (was TEXT, HA entity-map JSON outgrew it and silently failed writes for days before this was caught)
kb_intents           — intent_name, pattern(regex), response_template, action_type, priority, active
kb_generator_topics   — id, topic_id, category, topic_name, description, active, run_count, last_run_at
tasks                — title, notes, category, priority, status, due_date, due_time, completed_at
appointments         — title, description, category, start_at, end_at, location, all_day, reminder_min
```

## KB Intent Generator (self-expanding knowledge base)
- Cron `0 */6 * * *`, processes 25 topics/run (BATCH_SIZE) cycling through `kb_generator_topics` table.
- Topic count grew from 140 → 4530 across 170 categories (general knowledge, Fortune-500 companies, space/exploration, etc.) — at 25/run a full cycle now takes ~45 days; raise BATCH_SIZE if freshness matters.
- Admin UI "MANAGE TOPICS" button — browse/add/edit/delete/toggle topics.
- Groq model `llama-3.3-70b-versatile`, 20 intents/call, max_tokens=5000 (raising from 40/3500 fixed an 88% truncation failure rate).

## Backups
- Script: `/usr/local/bin/jarvis-backup.sh` → `/var/backups/jarvis/jarvis_backup_TIMESTAMP.tar.gz`, daily ~7am UTC, 7-day retention, downloadable from admin panel.
- Also captures `/etc/jarvis-arc` (reactor.env), `/etc/jarvis` (db.env), service drop-ins, `/usr/local/bin/jarvis-*.sh`.
- Health monitoring (`jarvis-health.sh`, PVE1 cron */5) checks stalled crons, stuck arc_jobs, disk >85%, watchdog restarts — auto-resolving `alerts` rows + email to `myronblair@gmail.com`.

## API Auth
- Main JARVIS API: session token via `X-Session-Token` header (or PHP session)
- Agent endpoints: per-agent key via header (see Agent System above)
- Netscan endpoint: `X-Registration-Key` header (shared registration key)
- Admin portal: separate PHP session name (`jarvis_admin`)
- Cloudflare passes real client IP in `CF-Connecting-IP` header (or `HTTP_X_FORWARDED_FOR`)

## Security posture (Phase 1 hardening, 2026-07-07 — mostly done, some user follow-up still open)
- Secrets pulled out of git-tracked files into `/etc/jarvis-arc/reactor.env` and `/etc/jarvis/db.env`. **Still pending (user action): rotate Claude/Groq/Gmail-app-pw/iCloud-app-pw/DB-pass since they remain in GitHub history**, and do a `git filter-repo` + force-push history purge.
- Login: Redis per-IP rate limit (10 fails/15min), session_regenerate_id on success.
- `INFRASTRUCTURE-REFERENCE.md` is gitignored/untracked in the repo — lives only on-disk on VM211 (and its VM110 mirror copy), not in GitHub.