Architect8999 commited on
Commit
95b8512
Β·
verified Β·
1 Parent(s): 6ed9a00

feat: integrate camofox-browser anti-detection browser

Browse files

feat: integrate camofox-browser anti-detection browser

Add camofox-browser (https://github.com/jo-inc/camofox-browser) as an
embedded anti-detection browser server so the orchestrator and analysis
engines (cve_intel, repo_harvester, knowledge_rag, red_team_fuzzer,
bounty_gateway) can browse the live web without being fingerprinted or
blocked by Cloudflare/Google bot detection.

camofox wraps the Camoufox engine - a Firefox fork with fingerprint
spoofing patched at the C++ implementation level - behind a small REST
API designed for AI agents (accessibility snapshots, stable element
refs, per-userId session isolation, cookie import, residential proxy
with auto GeoIP).

Added:
* camofox_client.py - pure-requests Python client + helpers
* CAMOFOX_INTEGRATION.md - architecture, env vars, usage docs
Modified:
* Dockerfile - install @askjo/camofox-browser under
/opt/camofox + X/GTK runtime libs
required by the Camoufox Firefox build
* entrypoint.sh - launch camofox server on 127.0.0.1:9377
alongside the OpenClaude gRPC daemons

Graceful degradation: if the server is unreachable, callers receive
CamofoxUnavailable / {available: False} and can fall back to plain
HTTP fetch - no hard dependency introduced.

Files changed (4) hide show
  1. CAMOFOX_INTEGRATION.md +94 -0
  2. Dockerfile +36 -6
  3. camofox_client.py +392 -0
  4. entrypoint.sh +40 -4
CAMOFOX_INTEGRATION.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Camofox-Browser Integration
2
+
3
+ Rhodawk now ships with an embedded
4
+ [camofox-browser](https://github.com/jo-inc/camofox-browser) anti-detection
5
+ browser server so the orchestrator and analysis engines can browse the live
6
+ web without being fingerprinted, blocked by Cloudflare, or flagged by
7
+ Google's bot detection.
8
+
9
+ ## Why
10
+
11
+ Several Rhodawk subsystems already need to fetch real-world web content:
12
+
13
+ | Subsystem | Today | With camofox-browser |
14
+ |----------------------|-----------------------------|--------------------------------------------|
15
+ | `cve_intel.py` | Plain `requests` to NVD/SSEC | Real-browser fetch with stable element refs |
16
+ | `repo_harvester.py` | GitHub REST + raw HTML | Authenticated session reuse, no rate-limit captcha |
17
+ | `knowledge_rag.py` | Static doc scraping | JS-rendered docs (Notion, Confluence, etc.) |
18
+ | `red_team_fuzzer.py` | n/a | Live target reconnaissance |
19
+ | `bounty_gateway.py` | HackerOne / Bugcrowd REST | UI fallback when the REST API is down |
20
+
21
+ camofox patches Firefox at the **C++ implementation level** β€”
22
+ `navigator.hardwareConcurrency`, WebGL renderers, AudioContext, screen
23
+ geometry, WebRTC β€” all spoofed before JavaScript ever sees them. No shims,
24
+ no wrappers, no tells.
25
+
26
+ ## Architecture
27
+
28
+ ```
29
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
30
+ β”‚ app.py / orchestrator β”‚ HTTP β”‚ /opt/camofox (node) β”‚
31
+ β”‚ cve_intel.py / harvester β”‚ ──────▢ β”‚ 127.0.0.1:9377 β”‚
32
+ β”‚ knowledge_rag.py / … β”‚ β”‚ camofox-browser server β”‚
33
+ β”‚ β”‚ β”‚ β”œβ”€β”€ Camoufox (Firefox fork) β”‚
34
+ β”‚ camofox_client.py β”‚ β”‚ β”œβ”€β”€ Playwright contexts β”‚
35
+ β”‚ └─ CamofoxClient (REST) β”‚ β”‚ └── per-userId session jar β”‚
36
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
37
+ ```
38
+
39
+ * **Server** is launched by `entrypoint.sh` alongside the OpenClaude gRPC
40
+ daemons. Listens on `127.0.0.1:9377` (not exposed publicly by default).
41
+ * **Client** is `camofox_client.py` β€” a thin, dependency-free
42
+ (`requests`-only) Python wrapper that any module can import.
43
+ * **Sessions** are isolated per `userId` so concurrent research jobs
44
+ cannot leak cookies into each other.
45
+
46
+ ## Quick Start (Python)
47
+
48
+ ```python
49
+ from camofox_client import get_client, fetch_snapshot
50
+
51
+ # One-shot: fetch a snapshot of a URL with element refs.
52
+ snap = fetch_snapshot(
53
+ "https://nvd.nist.gov/vuln/detail/CVE-2024-3094",
54
+ user_id="cve_intel",
55
+ )
56
+ if snap.get("available"):
57
+ print(snap["snapshot"][:2000])
58
+
59
+ # Long-running session:
60
+ client = get_client()
61
+ if client.is_available():
62
+ tab = client.create_tab("harvester", "https://github.com/torvalds/linux")
63
+ snap = client.snapshot(tab)
64
+ client.click(tab, "e1") # click the first interactive element
65
+ client.type_text(tab, "e2", "fix") # type into the search box
66
+ png = client.screenshot(tab, full_page=True)
67
+ client.close_tab(tab)
68
+ ```
69
+
70
+ If the camofox server is not running, `is_available()` returns `False` and
71
+ `fetch_snapshot()` returns `{"available": False, "reason": "..."}` so callers
72
+ can degrade gracefully to plain `requests`.
73
+
74
+ ## Environment Variables
75
+
76
+ | Variable | Default | Purpose |
77
+ |--------------------------|--------------------------------|-------------------------------------------|
78
+ | `CAMOFOX_BASE_URL` | `http://127.0.0.1:9377` | Where the Python client looks for the API |
79
+ | `CAMOFOX_PORT` | `9377` | Port the node server binds to |
80
+ | `CAMOFOX_HOST` | `127.0.0.1` | Host the node server binds to |
81
+ | `CAMOFOX_HEADLESS` | `virtual` | `virtual` uses xvfb; `true` uses Firefox headless mode |
82
+ | `CAMOFOX_PROFILE_DIR` | `/data/camofox/profiles` | Persistent per-user storage state |
83
+ | `CAMOFOX_COOKIES_DIR` | `/data/camofox/cookies` | Drop Netscape cookie files here for import |
84
+ | `CAMOFOX_API_KEY` | *(unset β†’ cookie writes disabled)* | Bearer token required for `import_cookies` |
85
+ | `PROXY_HOST` / `PROXY_PORT` / `PROXY_USERNAME` / `PROXY_PASSWORD` | *(unset)* | Route browser traffic through a residential proxy. Camoufox's GeoIP then auto-aligns locale + timezone. |
86
+
87
+ ## Files Added
88
+
89
+ | Path | What it is |
90
+ |----------------------------|---------------------------------------------|
91
+ | `camofox_client.py` | Python REST client + module-level helpers |
92
+ | `CAMOFOX_INTEGRATION.md` | This document |
93
+ | `Dockerfile` (modified) | Installs `@askjo/camofox-browser` under `/opt/camofox` and the X/GTK runtime libs Camoufox needs |
94
+ | `entrypoint.sh` (modified) | Launches the camofox server before the OpenClaude daemons |
Dockerfile CHANGED
@@ -22,6 +22,14 @@ ENV DEBIAN_FRONTEND=noninteractive \
22
  RUN apt-get update && apt-get install -y --no-install-recommends \
23
  git curl ca-certificates build-essential unzip xz-utils \
24
  nodejs npm \
 
 
 
 
 
 
 
 
25
  && rm -rf /var/lib/apt/lists/*
26
 
27
  # uv (fast Python installer used by sandboxed test runs)
@@ -50,6 +58,18 @@ RUN npm install -g --quiet \
50
  @modelcontextprotocol/server-sequential-thinking \
51
  @modelcontextprotocol/server-brave-search
52
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  # ─── Stage 3: final runtime image ───────────────────────────────────────
54
  FROM base AS runtime
55
  LABEL org.opencontainers.image.title="Rhodawk AI DevSecOps Engine" \
@@ -65,15 +85,25 @@ ENV GRADIO_SERVER_NAME=0.0.0.0 \
65
  OPENCLAUDE_GRPC_HOST=127.0.0.1 \
66
  OPENCLAUDE_GRPC_PORT_DO=50051 \
67
  OPENCLAUDE_GRPC_PORT_OR=50052 \
68
- MCP_RUNTIME_CONFIG=/tmp/mcp_runtime.json
 
 
 
 
 
 
 
 
 
 
69
 
70
  # HuggingFace UID 1000 handling (idempotent)
71
  RUN id -u 1000 >/dev/null 2>&1 && (userdel -r "$(id -un 1000)" || true) || true && \
72
  useradd -m -u 1000 -s /bin/bash rhodawk
73
 
74
- RUN mkdir -p /data /app /opt/openclaude && \
75
- chmod 777 /data && \
76
- chown -R rhodawk:rhodawk /app /opt/openclaude
77
 
78
  # Bring the prebuilt OpenClaude bundle in as a vendored artifact.
79
  COPY --from=openclaude-builder --chown=rhodawk:rhodawk /openclaude /opt/openclaude
@@ -100,6 +130,6 @@ RUN python -m grpc_tools.protoc \
100
  sed -i 's/^import openclaude_pb2/from . import openclaude_pb2/' \
101
  openclaude_grpc/openclaude_pb2_grpc.py
102
 
103
- EXPOSE 7860 50051 50052
104
 
105
- ENTRYPOINT ["/app/entrypoint.sh"]
 
22
  RUN apt-get update && apt-get install -y --no-install-recommends \
23
  git curl ca-certificates build-essential unzip xz-utils \
24
  nodejs npm \
25
+ # ─── camofox-browser runtime deps ────────────────────────────
26
+ # Camoufox is a Firefox fork; it needs the standard X/GTK
27
+ # display libraries even when running headless, plus xvfb so
28
+ # we can attach a virtual display when --headless=virtual.
29
+ xvfb libgtk-3-0 libdbus-glib-1-2 libxt6 libasound2 \
30
+ libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxfixes3 \
31
+ libxi6 libxrandr2 libxss1 libxtst6 libnss3 libpango-1.0-0 \
32
+ libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 libgbm1 \
33
  && rm -rf /var/lib/apt/lists/*
34
 
35
  # uv (fast Python installer used by sandboxed test runs)
 
58
  @modelcontextprotocol/server-sequential-thinking \
59
  @modelcontextprotocol/server-brave-search
60
 
61
+ # ─── camofox-browser anti-detection browser server ──────────────────────
62
+ # Anti-detection browser for AI agents (https://github.com/jo-inc/camofox-browser).
63
+ # Installed under /opt/camofox so the orchestrator can launch it via
64
+ # entrypoint.sh on 127.0.0.1:9377. The Camoufox Firefox-fork binary
65
+ # (~300MB) is fetched lazily on first launch by camoufox-js β€” keeping
66
+ # the image slim while still giving the orchestrator full access to the
67
+ # REST API exposed by camofox_client.py.
68
+ RUN mkdir -p /opt/camofox && cd /opt/camofox && \
69
+ npm init -y >/dev/null 2>&1 && \
70
+ npm install --quiet --omit=dev @askjo/camofox-browser@^1.6.0 || \
71
+ npm install --quiet --omit=dev camofox-browser
72
+
73
  # ─── Stage 3: final runtime image ───────────────────────────────────────
74
  FROM base AS runtime
75
  LABEL org.opencontainers.image.title="Rhodawk AI DevSecOps Engine" \
 
85
  OPENCLAUDE_GRPC_HOST=127.0.0.1 \
86
  OPENCLAUDE_GRPC_PORT_DO=50051 \
87
  OPENCLAUDE_GRPC_PORT_OR=50052 \
88
+ MCP_RUNTIME_CONFIG=/tmp/mcp_runtime.json \
89
+ # ─── camofox-browser runtime defaults ────────────────────────────
90
+ # The orchestrator talks to the local camofox server through
91
+ # camofox_client.py. CAMOFOX_API_KEY gates cookie-import β€” leave
92
+ # it unset to keep cookie writes disabled (server returns 403).
93
+ CAMOFOX_BASE_URL=http://127.0.0.1:9377 \
94
+ CAMOFOX_PORT=9377 \
95
+ CAMOFOX_HOST=127.0.0.1 \
96
+ CAMOFOX_HEADLESS=virtual \
97
+ CAMOFOX_PROFILE_DIR=/data/camofox/profiles \
98
+ CAMOFOX_COOKIES_DIR=/data/camofox/cookies
99
 
100
  # HuggingFace UID 1000 handling (idempotent)
101
  RUN id -u 1000 >/dev/null 2>&1 && (userdel -r "$(id -un 1000)" || true) || true && \
102
  useradd -m -u 1000 -s /bin/bash rhodawk
103
 
104
+ RUN mkdir -p /data /data/camofox/profiles /data/camofox/cookies /app /opt/openclaude && \
105
+ chmod -R 777 /data && \
106
+ chown -R rhodawk:rhodawk /app /opt/openclaude /opt/camofox
107
 
108
  # Bring the prebuilt OpenClaude bundle in as a vendored artifact.
109
  COPY --from=openclaude-builder --chown=rhodawk:rhodawk /openclaude /opt/openclaude
 
130
  sed -i 's/^import openclaude_pb2/from . import openclaude_pb2/' \
131
  openclaude_grpc/openclaude_pb2_grpc.py
132
 
133
+ EXPOSE 7860 9377 50051 50052
134
 
135
+ ENTRYPOINT ["/app/entrypoint.sh"]
camofox_client.py ADDED
@@ -0,0 +1,392 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ camofox_client.py
3
+ ─────────────────
4
+ Python client for the embedded camofox-browser anti-detection browser server
5
+ (https://github.com/jo-inc/camofox-browser).
6
+
7
+ camofox-browser wraps the Camoufox engine β€” a Firefox fork with fingerprint
8
+ spoofing patched at the C++ implementation level β€” behind a small REST API
9
+ designed for AI agents:
10
+
11
+ * accessibility snapshots with stable element refs (e1, e2, e3 …)
12
+ * session isolation per userId / sessionKey
13
+ * Netscape-format cookie import for authenticated browsing
14
+ * residential-proxy + GeoIP routing
15
+ * search macros (@google_search, @youtube_search, …)
16
+ * download capture, DOM image extraction, screenshot snapshots
17
+
18
+ Inside the Rhodawk container the camofox node server is launched by
19
+ `entrypoint.sh` on 127.0.0.1:9377. This module is the thin Python
20
+ adapter the orchestrator and other engines (repo_harvester, cve_intel,
21
+ red_team_fuzzer, knowledge_rag, …) use to drive it.
22
+
23
+ Design goals
24
+ ────────────
25
+ 1. Zero hard dependency β€” if the camofox server is not running the
26
+ client raises ``CamofoxUnavailable`` and the caller can degrade
27
+ gracefully (e.g. fall back to plain ``requests.get``).
28
+ 2. No ``console.print``-style side effects β€” pure return values + the
29
+ structured ``audit_logger`` for production observability.
30
+ 3. Safe defaults β€” every browsing call carries a ``userId`` and an
31
+ optional ``sessionKey`` so different research jobs cannot leak
32
+ cookies into each other.
33
+ """
34
+
35
+ from __future__ import annotations
36
+
37
+ import json
38
+ import logging
39
+ import os
40
+ import time
41
+ from dataclasses import dataclass, field
42
+ from typing import Any, Dict, List, Optional
43
+
44
+ import requests
45
+
46
+ log = logging.getLogger("rhodawk.camofox")
47
+
48
+ # ─── Configuration ─────────────────────────────────────────────────────
49
+ CAMOFOX_BASE_URL = os.environ.get("CAMOFOX_BASE_URL", "http://127.0.0.1:9377").rstrip("/")
50
+ CAMOFOX_API_KEY = os.environ.get("CAMOFOX_API_KEY", "")
51
+ CAMOFOX_DEFAULT_TIMEOUT = float(os.environ.get("CAMOFOX_TIMEOUT", "60"))
52
+ CAMOFOX_HEALTH_TIMEOUT = float(os.environ.get("CAMOFOX_HEALTH_TIMEOUT", "3"))
53
+
54
+
55
+ # ─── Errors ────────────────────────────────────────────────────────────
56
+ class CamofoxError(RuntimeError):
57
+ """Base class for any camofox-browser interaction failure."""
58
+
59
+
60
+ class CamofoxUnavailable(CamofoxError):
61
+ """Raised when the camofox server is not reachable.
62
+
63
+ Callers should treat this as a soft failure and fall back to a
64
+ non-browser code path (e.g. plain HTTP fetch) instead of aborting.
65
+ """
66
+
67
+
68
+ class CamofoxAPIError(CamofoxError):
69
+ """Raised when the server is reachable but returns a non-2xx response."""
70
+
71
+ def __init__(self, status: int, body: str, *, endpoint: str = ""):
72
+ super().__init__(f"[{endpoint}] HTTP {status}: {body[:512]}")
73
+ self.status = status
74
+ self.body = body
75
+ self.endpoint = endpoint
76
+
77
+
78
+ # ─── Session handle ────────────────────────────────────────────────────
79
+ @dataclass
80
+ class CamofoxTab:
81
+ """Lightweight handle to a single tab inside the camofox server.
82
+
83
+ Tabs are owned by ``(userId, sessionKey)``. Two researchers using
84
+ different ``userId`` values get fully isolated cookie jars and
85
+ storage state β€” important when one job is logged-in to GitHub and
86
+ the other is performing CVE intel scraping anonymously.
87
+ """
88
+ tab_id: str
89
+ user_id: str
90
+ session_key: str = "default"
91
+ url: Optional[str] = None
92
+ extra: Dict[str, Any] = field(default_factory=dict)
93
+
94
+
95
+ # ─── Client ────────────────────────────────────────────────────────────
96
+ class CamofoxClient:
97
+ """Thin REST wrapper around the local camofox-browser server."""
98
+
99
+ def __init__(
100
+ self,
101
+ base_url: str = CAMOFOX_BASE_URL,
102
+ api_key: str = CAMOFOX_API_KEY,
103
+ timeout: float = CAMOFOX_DEFAULT_TIMEOUT,
104
+ ) -> None:
105
+ self.base_url = base_url.rstrip("/")
106
+ self.api_key = api_key
107
+ self.timeout = timeout
108
+ self._session = requests.Session()
109
+
110
+ # ── Internal ──────────────────────────────────────────────────────
111
+ def _headers(self) -> Dict[str, str]:
112
+ h = {"Content-Type": "application/json"}
113
+ if self.api_key:
114
+ h["Authorization"] = f"Bearer {self.api_key}"
115
+ return h
116
+
117
+ def _request(
118
+ self,
119
+ method: str,
120
+ path: str,
121
+ *,
122
+ json_body: Optional[Dict[str, Any]] = None,
123
+ params: Optional[Dict[str, Any]] = None,
124
+ timeout: Optional[float] = None,
125
+ ) -> Dict[str, Any]:
126
+ url = f"{self.base_url}{path}"
127
+ try:
128
+ resp = self._session.request(
129
+ method,
130
+ url,
131
+ json=json_body,
132
+ params=params,
133
+ headers=self._headers(),
134
+ timeout=timeout or self.timeout,
135
+ )
136
+ except requests.exceptions.ConnectionError as exc:
137
+ raise CamofoxUnavailable(
138
+ f"camofox server not reachable at {self.base_url}: {exc}"
139
+ ) from exc
140
+ except requests.exceptions.Timeout as exc:
141
+ raise CamofoxError(f"camofox request timed out: {exc}") from exc
142
+
143
+ if resp.status_code >= 400:
144
+ raise CamofoxAPIError(resp.status_code, resp.text, endpoint=path)
145
+
146
+ if not resp.content:
147
+ return {}
148
+ ct = resp.headers.get("content-type", "")
149
+ if "application/json" not in ct:
150
+ return {"raw": resp.text}
151
+ try:
152
+ return resp.json()
153
+ except json.JSONDecodeError as exc:
154
+ raise CamofoxError(f"invalid JSON from {path}: {exc}") from exc
155
+
156
+ # ── Health ────────────────────────────────────────────────────────
157
+ def is_available(self) -> bool:
158
+ """Return True if the camofox server is up. Never raises."""
159
+ try:
160
+ self._request("GET", "/health", timeout=CAMOFOX_HEALTH_TIMEOUT)
161
+ return True
162
+ except CamofoxError:
163
+ return False
164
+
165
+ def wait_ready(self, max_wait: float = 30.0, poll_interval: float = 0.5) -> bool:
166
+ """Block until the server responds to /health or ``max_wait`` elapses."""
167
+ deadline = time.time() + max_wait
168
+ while time.time() < deadline:
169
+ if self.is_available():
170
+ return True
171
+ time.sleep(poll_interval)
172
+ return False
173
+
174
+ # ── Tab lifecycle ────────────────────────────────────────────────
175
+ def create_tab(
176
+ self,
177
+ user_id: str,
178
+ url: str,
179
+ *,
180
+ session_key: str = "default",
181
+ wait_until: str = "domcontentloaded",
182
+ ) -> CamofoxTab:
183
+ body = {
184
+ "userId": user_id,
185
+ "sessionKey": session_key,
186
+ "url": url,
187
+ "waitUntil": wait_until,
188
+ }
189
+ out = self._request("POST", "/tabs", json_body=body)
190
+ tab_id = out.get("tabId") or out.get("id")
191
+ if not tab_id:
192
+ raise CamofoxError(f"create_tab returned no tabId: {out}")
193
+ return CamofoxTab(
194
+ tab_id=tab_id,
195
+ user_id=user_id,
196
+ session_key=session_key,
197
+ url=out.get("url", url),
198
+ extra={k: v for k, v in out.items() if k not in {"tabId", "id", "url"}},
199
+ )
200
+
201
+ def list_tabs(self, user_id: str) -> List[Dict[str, Any]]:
202
+ out = self._request("GET", "/tabs", params={"userId": user_id})
203
+ return out.get("tabs", []) if isinstance(out, dict) else []
204
+
205
+ def close_tab(self, tab: CamofoxTab) -> None:
206
+ self._request(
207
+ "DELETE",
208
+ f"/tabs/{tab.tab_id}",
209
+ params={"userId": tab.user_id},
210
+ )
211
+
212
+ # ── Snapshot / interaction ───────────────────────────────────────
213
+ def snapshot(
214
+ self,
215
+ tab: CamofoxTab,
216
+ *,
217
+ include_screenshot: bool = False,
218
+ offset: int = 0,
219
+ limit: Optional[int] = None,
220
+ ) -> Dict[str, Any]:
221
+ """Accessibility snapshot β€” ~90% smaller than raw HTML,
222
+ annotated with stable element refs (e1, e2, …)."""
223
+ params: Dict[str, Any] = {"userId": tab.user_id}
224
+ if include_screenshot:
225
+ params["screenshot"] = "true"
226
+ if offset:
227
+ params["offset"] = offset
228
+ if limit is not None:
229
+ params["limit"] = limit
230
+ return self._request("GET", f"/tabs/{tab.tab_id}/snapshot", params=params)
231
+
232
+ def click(self, tab: CamofoxTab, ref: str) -> Dict[str, Any]:
233
+ return self._request(
234
+ "POST",
235
+ f"/tabs/{tab.tab_id}/click",
236
+ json_body={"userId": tab.user_id, "ref": ref},
237
+ )
238
+
239
+ def type_text(
240
+ self,
241
+ tab: CamofoxTab,
242
+ ref: str,
243
+ text: str,
244
+ *,
245
+ press_enter: bool = False,
246
+ ) -> Dict[str, Any]:
247
+ return self._request(
248
+ "POST",
249
+ f"/tabs/{tab.tab_id}/type",
250
+ json_body={
251
+ "userId": tab.user_id,
252
+ "ref": ref,
253
+ "text": text,
254
+ "pressEnter": press_enter,
255
+ },
256
+ )
257
+
258
+ def navigate(
259
+ self,
260
+ tab: CamofoxTab,
261
+ url: str,
262
+ *,
263
+ wait_until: str = "domcontentloaded",
264
+ ) -> Dict[str, Any]:
265
+ return self._request(
266
+ "POST",
267
+ f"/tabs/{tab.tab_id}/navigate",
268
+ json_body={
269
+ "userId": tab.user_id,
270
+ "url": url,
271
+ "waitUntil": wait_until,
272
+ },
273
+ )
274
+
275
+ def scroll(
276
+ self,
277
+ tab: CamofoxTab,
278
+ *,
279
+ direction: str = "down",
280
+ amount: int = 1,
281
+ ) -> Dict[str, Any]:
282
+ return self._request(
283
+ "POST",
284
+ f"/tabs/{tab.tab_id}/scroll",
285
+ json_body={
286
+ "userId": tab.user_id,
287
+ "direction": direction,
288
+ "amount": amount,
289
+ },
290
+ )
291
+
292
+ def screenshot(
293
+ self,
294
+ tab: CamofoxTab,
295
+ *,
296
+ full_page: bool = False,
297
+ ) -> bytes:
298
+ """Return raw PNG bytes for ``tab``."""
299
+ url = f"{self.base_url}/tabs/{tab.tab_id}/screenshot"
300
+ try:
301
+ resp = self._session.get(
302
+ url,
303
+ params={"userId": tab.user_id, "fullPage": str(full_page).lower()},
304
+ headers={k: v for k, v in self._headers().items() if k != "Content-Type"},
305
+ timeout=self.timeout,
306
+ )
307
+ except requests.exceptions.ConnectionError as exc:
308
+ raise CamofoxUnavailable(str(exc)) from exc
309
+ if resp.status_code >= 400:
310
+ raise CamofoxAPIError(resp.status_code, resp.text, endpoint="/screenshot")
311
+ return resp.content
312
+
313
+ # ── Cookies / auth ───────────────────────────────────────────────
314
+ def import_cookies(
315
+ self,
316
+ user_id: str,
317
+ cookies: List[Dict[str, Any]],
318
+ ) -> Dict[str, Any]:
319
+ """Inject pre-exported cookies into a session.
320
+
321
+ Requires ``CAMOFOX_API_KEY`` to be set; without it the camofox
322
+ server rejects cookie writes with 403 by design.
323
+ """
324
+ if not self.api_key:
325
+ raise CamofoxError(
326
+ "CAMOFOX_API_KEY env var is required for cookie import"
327
+ )
328
+ return self._request(
329
+ "POST",
330
+ f"/sessions/{user_id}/cookies",
331
+ json_body={"cookies": cookies},
332
+ )
333
+
334
+ # ── YouTube transcript (optional, ships with camofox) ────────────
335
+ def youtube_transcript(self, video_url: str) -> Dict[str, Any]:
336
+ return self._request(
337
+ "POST",
338
+ "/youtube/transcript",
339
+ json_body={"url": video_url},
340
+ )
341
+
342
+
343
+ # ─── Module-level convenience singleton ────────────────────────────────
344
+ _default_client: Optional[CamofoxClient] = None
345
+
346
+
347
+ def get_client() -> CamofoxClient:
348
+ """Lazy module-wide singleton β€” most callers just want this."""
349
+ global _default_client
350
+ if _default_client is None:
351
+ _default_client = CamofoxClient()
352
+ return _default_client
353
+
354
+
355
+ def fetch_snapshot(
356
+ url: str,
357
+ *,
358
+ user_id: str = "rhodawk",
359
+ session_key: str = "default",
360
+ include_screenshot: bool = False,
361
+ close_when_done: bool = True,
362
+ ) -> Dict[str, Any]:
363
+ """One-shot helper: open ``url`` in a fresh tab, return its snapshot,
364
+ then close the tab. Returns ``{"available": False}`` if the camofox
365
+ server is not running so the caller can fall back gracefully.
366
+ """
367
+ client = get_client()
368
+ if not client.is_available():
369
+ return {"available": False, "reason": "camofox server not reachable"}
370
+ tab = client.create_tab(user_id, url, session_key=session_key)
371
+ try:
372
+ snap = client.snapshot(tab, include_screenshot=include_screenshot)
373
+ snap["available"] = True
374
+ snap["tabId"] = tab.tab_id
375
+ return snap
376
+ finally:
377
+ if close_when_done:
378
+ try:
379
+ client.close_tab(tab)
380
+ except CamofoxError as exc:
381
+ log.warning("close_tab failed for %s: %s", tab.tab_id, exc)
382
+
383
+
384
+ __all__ = [
385
+ "CamofoxClient",
386
+ "CamofoxTab",
387
+ "CamofoxError",
388
+ "CamofoxUnavailable",
389
+ "CamofoxAPIError",
390
+ "get_client",
391
+ "fetch_snapshot",
392
+ ]
entrypoint.sh CHANGED
@@ -2,19 +2,52 @@
2
  # ─────────────────────────────────────────────────────────────────────
3
  # Rhodawk runtime bootstrap.
4
  #
5
- # 1. Launch the OpenClaude headless gRPC daemon for the DigitalOcean
 
 
6
  # Inference provider on :50051 (PRIMARY).
7
- # 2. Launch the OpenClaude headless gRPC daemon for OpenRouter on
8
  # :50052 (FALLBACK) β€” only if OPENROUTER_API_KEY is present.
9
- # 3. Wait briefly for both to bind, then hand control to app.py which
10
- # talks to them over gRPC.
11
  # ─────────────────────────────────────────────────────────────────────
12
  set -eo pipefail
13
 
14
  OC_DIR=/opt/openclaude
 
15
  LOG_DIR="${LOG_DIR:-/tmp}"
16
  mkdir -p "${LOG_DIR}"
17
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  start_daemon() {
19
  local label=$1 port=$2 base_url=$3 api_key=$4 model=$5
20
  if [[ -z "${api_key}" ]]; then
@@ -38,6 +71,9 @@ start_daemon() {
38
  )
39
  }
40
 
 
 
 
41
  # DigitalOcean Inference (PRIMARY)
42
  DO_BASE="${DO_INFERENCE_BASE_URL:-https://inference.do-ai.run/v1}"
43
  DO_MODEL="${DO_INFERENCE_MODEL:-llama3.3-70b-instruct}"
 
2
  # ─────────────────────────────────────────────────────────────────────
3
  # Rhodawk runtime bootstrap.
4
  #
5
+ # 1. Launch the camofox-browser anti-detection browser server on
6
+ # 127.0.0.1:9377 (used by the orchestrator via camofox_client.py).
7
+ # 2. Launch the OpenClaude headless gRPC daemon for the DigitalOcean
8
  # Inference provider on :50051 (PRIMARY).
9
+ # 3. Launch the OpenClaude headless gRPC daemon for OpenRouter on
10
  # :50052 (FALLBACK) β€” only if OPENROUTER_API_KEY is present.
11
+ # 4. Wait briefly for everything to bind, then hand control to app.py
12
+ # which talks to them over gRPC + HTTP.
13
  # ─────────────────────────────────────────────────────────────────────
14
  set -eo pipefail
15
 
16
  OC_DIR=/opt/openclaude
17
+ CAMOFOX_DIR=/opt/camofox
18
  LOG_DIR="${LOG_DIR:-/tmp}"
19
  mkdir -p "${LOG_DIR}"
20
 
21
+ # ─── camofox-browser ─────────────────────────────────────────────────
22
+ # Anti-detection Firefox-fork browser server. Lazily downloads the
23
+ # Camoufox engine (~300MB) on first launch into the user's home dir,
24
+ # so the first start may take a minute. Subsequent starts are fast.
25
+ start_camofox() {
26
+ local entry="${CAMOFOX_DIR}/node_modules/@askjo/camofox-browser/server.js"
27
+ [[ -f "${entry}" ]] || entry="${CAMOFOX_DIR}/node_modules/camofox-browser/server.js"
28
+ if [[ ! -f "${entry}" ]]; then
29
+ echo "[entrypoint] camofox-browser not installed β€” skipping"
30
+ return 0
31
+ fi
32
+ echo "[entrypoint] starting camofox-browser on ${CAMOFOX_HOST:-127.0.0.1}:${CAMOFOX_PORT:-9377}"
33
+ (
34
+ cd "${CAMOFOX_DIR}"
35
+ PORT="${CAMOFOX_PORT:-9377}" \
36
+ HOST="${CAMOFOX_HOST:-127.0.0.1}" \
37
+ CAMOFOX_HEADLESS="${CAMOFOX_HEADLESS:-virtual}" \
38
+ CAMOFOX_PROFILE_DIR="${CAMOFOX_PROFILE_DIR:-/data/camofox/profiles}" \
39
+ CAMOFOX_COOKIES_DIR="${CAMOFOX_COOKIES_DIR:-/data/camofox/cookies}" \
40
+ CAMOFOX_API_KEY="${CAMOFOX_API_KEY:-}" \
41
+ PROXY_HOST="${PROXY_HOST:-}" \
42
+ PROXY_PORT="${PROXY_PORT:-}" \
43
+ PROXY_USERNAME="${PROXY_USERNAME:-}" \
44
+ PROXY_PASSWORD="${PROXY_PASSWORD:-}" \
45
+ node "${entry}" \
46
+ > "${LOG_DIR}/camofox.log" 2>&1 &
47
+ echo $! > "${LOG_DIR}/camofox.pid"
48
+ )
49
+ }
50
+
51
  start_daemon() {
52
  local label=$1 port=$2 base_url=$3 api_key=$4 model=$5
53
  if [[ -z "${api_key}" ]]; then
 
71
  )
72
  }
73
 
74
+ # camofox first β€” slowest to bind because of the lazy engine download.
75
+ start_camofox
76
+
77
  # DigitalOcean Inference (PRIMARY)
78
  DO_BASE="${DO_INFERENCE_BASE_URL:-https://inference.do-ai.run/v1}"
79
  DO_MODEL="${DO_INFERENCE_MODEL:-llama3.3-70b-instruct}"