28 Commits
Author SHA1 Message Date
WHC e0b0eddf3a v0.8.6: general mode, clipboard text, PDF text extraction
Working mode: --mode coding|general (CST_MODE, settings.json, /mode,
WebUI run settings). general swaps the default framing for a general
assistant one -- workflow, safety notes, tool wording (run_shell, the
subagents) -- and /status shows it; minimal (the paper profile) stays
byte-frozen. The prompt state carries the mode so a switch leaves a
system note in history.

Clipboard: clipboard_read / clipboard_write tools (16 MiB cap read,
default 64 KiB window; writing is a WRITE permission with a preview) on
pbcopy / wl-copy / xclip / pbpaste / wl-paste / xclip and the Win32
clipboard. In the TUI, /attach paste falls back to clipboard text when
there is no image on it.

PDF: a dependency-free text extractor (src/pdf_text.c): inflate,
Flate/ASCIIHex/ASCII85, PNG/TIFF predictors, layout-independent object
scan, object streams, page tree walk with inherited resources, content
lexer (Tj/TJ/'/" and the Td/TD/T*/Tm spacing heuristics), per-font
ToUnicode CMaps and inline images skipped. Encrypted files and pages
with no text layer are refused with the reason.

Attachments: /attach (a path or files dropped on the terminal) now take
images, PDFs and text. A PDF is stored as its text layer with the mime
still saying application/pdf and its page count; anything else has to be
UTF-8 text. Documents travel inline as text parts, so CST_NO_IMAGES and
text-only models no longer drop them, while images keep their old path.

Tests: test_pdf_text (plain, bare and zlib-framed Flate, inline image,
CID + ToUnicode, object stream, error paths), documents in test_attach,
inlined documents in test_llm_body, mode in test_config.
2026-10-11 11:09:39 +08:00
WHC 50abcf1a91 build: finish cross-platform release tooling 2026-10-10 16:21:00 +08:00
WHC 9bf7707912 v0.8.5: WebUI process timeline, Gemini-style images, aligned layout
Process timeline: a run of consecutive thinking and tool calls is one
block that folds to a single line ("思考 4 次 · 工具 6 次"; while running
it shows the current action) and opens into a compact timeline, one row
per thought or call. Visible text, a note or the next prompt closes the
block; history and the live stream follow the same rule. show_thinking
now also applies while streaming instead of forcing thoughts open.

Layout: the avatar gutter and the flex-wrap that dropped it under long
answers are gone, so answer, process blocks, notes and composer share one
left edge. The text column and composer are both 720px (--content); the
rail and run panel grow with clamp() on wide screens. A rebuild after
each turn no longer replays every entrance animation, and the turn time
survives it. Message actions are icon buttons and only appear under text.

Images: a prompt's images sit above the bubble, right-aligned (one keeps
its shape, several are 152px rounded tiles) and open in a lightbox (Esc,
arrow keys). Images uploaded from the page show as thumbnails over the
composer and appear on the prompt as soon as it is sent.

Also: ?theme= is no longer overridden by app.js, the user bubble is
visible in the light theme, and the narrow top bar no longer wraps.
2026-10-10 14:54:36 +08:00
WHC 7e1dbd9afa v0.8.4: WebUI sessions, diff, settings, slash commands and extensions
Sessions: rename, delete and search; /api/retry regenerates the last
answer, editing a prompt resends from there; /api/export downloads the
transcript as Markdown (cst_session_export_md). Session messages carry
note/tool_result markers, so a reload rebuilds the tool steps with
their arguments, output and exit status; subagent progress is shown.

Diff and settings: /api/diff shows the real diff; /api/settings covers
protocol, max_steps, tool switches and confirm; /api/permissions edits
the rules; sandbox status is reported. Session rename no longer reads
past a short title.

Commands: slash commands with completion in the composer (/model,
/provider, /new, /retry, /undo, /compact, /export, /diff, /init,
/skill and more), a Ctrl+P palette, Shift+Tab cycles the approval
mode, F1 or ? shows the keys. "Thinking expanded" is saved in the
settings (show_thinking), "expand all tool output" in the browser.

Extensions: GET /api/extensions lists MCP servers, plugins, skills,
agent roles and subagent runs; POST /api/extensions/reload and
/api/extensions/trust; a workspace switch reloads them. /api/init and
/api/skill run on the server (cst_init_prompt shared with the TUI).

Errors, statuses and [cst] notes are shown in Chinese with a readable
reason instead of rc=; a failed turn's error survives the reload.

Tests: scripts/e2e_webui_sessions.py, e2e_webui_workspace.py and
e2e_webui_ext.py, run from e2e_webui.sh.
2026-10-10 13:57:17 +08:00
WHC 2d1052fd5f v0.8.3: WebUI models, providers and keys; garbled thinking fixed
Streaming: Chinese reasoning showed up garbled in the WebUI. The agent
loop now feeds visible/thinking text in runs instead of byte by byte,
and the WebUI holds an incomplete UTF-8 tail until the next chunk
(cst_utf8_hold_feed), so every think_delta/assistant_delta is whole.

src/provider_switch.c: model list fetch/cache, key verify-and-save and
provider switching moved out of tui.c and shared by TUI and WebUI.

WebUI API:
- GET /api/providers, GET /api/models[?refresh=1] (401 on a rejected
  key, still listing built-ins), POST /api/provider, POST /api/key
  (key checked against /v1/models, saved to credentials 0600, never
  echoed in replies, events or logs)
- /api/config validates everything before applying, persists model,
  effort and approval (not yolo), warns when effort does not apply;
  plan_mode no longer forces read-only
- state: turn_usage, api_base, key_set/key_from, model_info,
  effort_applies, sees_images, max_steps, tool_profile, perm_timeout;
  permission events carry their timeout

Frontend: searchable model picker with refresh and manual entry,
provider picker and key dialog, effort/vision hints, toasts instead of
silent failures, Chinese error text, real protocol in the settings and
usage panels, approval countdown from the server's timeout.

Tests: utf8 hold unit test, scripts/e2e_webui_settings.py (fake
/v1/models, wrong key, switching, persistence, Chinese thinking) run
from e2e_webui.sh, scripts/tui_keys/provider.py.
2026-10-09 15:54:53 +08:00
WHC e238f5f37d v0.8.2: build fixes, WebUI security and correctness
Build: <fcntl.h> in tui.c, format-truncation in http_srv.c and /web.

WebUI:
- refuse cross-site requests (Origin must be the page itself) in the
  default no-token mode: any page could POST approval=yolo and a prompt
- session/workspace/undo/compact/attach/model changes are 409 while a
  turn runs (use-after-free, messages landing in the wrong session)
- ask_user returned "dismissed" at once instead of waiting
- cancel and shutdown wake a turn parked on a permission or question;
  SIGTERM mid-turn no longer hangs or crashes
- a page opened while a question is pending still gets it;
  Last-Event-ID resumes the stream; event seq matches the SSE id
- token mode: page assets load without the token, images via cookie
- --web-open works, --web-port/--web-token are validated
- frontend: gate keys ignored while typing, clear-images button,
  reasoning no longer wiped while streaming

HTTP server: auth and size cap before reading the body, strict
Content-Length, NUL in url_decode, 413 on keep-alive, idle timeout,
absolute-form only without a leading '/', case-insensitive Connection,
header deadline, 400 for long request lines, unlink before close,
poll() instead of select(), SO_EXCLUSIVEADDRUSE on Windows.

Markdown sanitizer escapes quotes in attribute values.

/web: passes this session's provider/model/base/effort/approval (key
via the environment), keeps output off the TUI, shows the real URL and
keeps CURRENT on the TUI's session.

Tests: scripts/e2e_webui_ask.py, scripts/e2e_webui_http.py (run from
e2e_webui.sh), sanitizer quote test.
2026-10-09 13:51:34 +08:00
WHC adf3e9f024 v0.8.0: WebUI redesign, /web, workspace switch, 75% budget, no shell timeout
The front end is rebuilt on the design tokens the desktop app ships (surfaces,
color-mix borders, alpha hovers, accent): a 30-icon inline SVG sprite, dark and
light themes, layout switches as deep links (?side=0, ?panel=0, ?theme=light)
resolved before the first paint. The empty page is one line and leaves with the
first turn; the right rail keeps three cards - todo, file changes (+N -M) and
context (tokens, prompt cache hit rate). Both rails animate in and out, and a
collapsed rail widens its grid track to 0 instead of being removed, so the
transcript never slides into a zero-width column.

- src/webui.c: POST /api/workspace - re-root the agent at another directory
  (permission rules, metrics, AGENTS.md follow) and start a session there, as
  sessions live per directory; state also reports cached/cache_miss tokens
- src/agent_loop.c: cst_agent_set_workspace(), rebuilt prompt and re-measured
  budget; the auto-compact threshold is 75% of the window (was 60%)
- src/tui.c: /web forks a WebUI next to the TUI - separate process and a new
  session, because cst_webui_run() blocks and two writers corrupt one session
- src/os_compat.c, src/os_win.c: run_shell runs without a time limit unless
  CST_SHELL_TIMEOUT asks for one, so builds and long clones finish
- src/main.c, include/cst_webui.h: one CST_WEBUI_DEFAULT_PORT for --help, the
  docs and /web; Termux opens the browser through termux-open-url
- README.md, docs/webui.md, docs/env_openai.md: /web, /api/workspace, the new
  state fields, the 75% threshold and the layout deep links

tests/e2e_webui.sh green (permission round-trip, token paths, --web-dev);
make check green.
2026-10-09 11:50:13 +08:00
WHC 6e81513e36 v0.7.8: WebUI (--web): HTTP/1.1 + SSE front end, engine API v1, tests
A second implementation of cst_agent_hooks_t next to the TUI: the turn runs on
a worker thread, the hooks publish events into a ring buffer that SSE clients
read, so the engine itself needs no changes.

- src/net.c, include/cst_net.h: POSIX/Winsock sockets, non-blocking connect
- src/http_srv.c, include/cst_http.h: HTTP/1.1, keep-alive, chunked streaming,
  body caps (413), Expect: 100-continue, connection cap, exact routing
- src/web_md.c, include/cst_web_md.h: streaming markdown -> sanitized HTML
  (tag/attribute allowlist, javascript: URLs and off-site images dropped)
- src/webui.c, include/cst_webui.h: event ring + SSE broadcast, all routes,
  token (Bearer/Cookie/query) and loopback Host checks, 3 s auto-deny when no
  client is attached
- src/webui_stub.c, CST_WEBUI=0 / CST_BUILD_WEBUI=OFF: build without the WebUI
- web/: no-build front end (transcript, approvals, todo, sessions, image
  upload, status bar)
- docs/webui.md: engine API v1 - events, requests, security model
- tests/test_webui.c, scripts/e2e_webui.sh: unit tests plus an end-to-end run
  driven by a recorded SSE stream (no API key needed)
- src/main.c: --web --web-port --web-host --web-token --web-dev --web-open
  and CST_WEB_PORT/HOST/TOKEN/ROOT

An empty POST body is now accepted without Content-Length (RFC 9112 6.3) so
curl-driven clients can reach cancel/undo/compact/quit; CMakeList's core tests
learned about sashidown and test_webui, and CST_WEBUI=0 defines CST_NO_WEBUI.
2026-10-09 10:20:09 +08:00
WHC c7b32c3fd6 v0.7.5: arrow keys, image request fix, deepseek-flash vision, musl CI
TUI keys: register both CSI and SS3 forms of the cursor/Home/End keys and
the xterm modifier sequences when terminfo lacks them, so an arrow sent in
the "other" form no longer turns into Esc + a typed letter. Up/Down move by
visual (soft-wrapped) row, and the draft survives a trip through history.
scripts/tui_keys drives the TUI in a pty to check all of this.

Images: text inside a multimodal content array is now a {"type":"text"}
part; DeepSeek rejected the bare string with HTTP 422. deepseek-flash
(and its deepseek-chat / deepseek-reasoner aliases) read images, so the
registry marks them vision; deepseek-v4-pro stays text only.

CI: models.c lost _DEFAULT_SOURCE in 0.7.0 and gmtime_r broke the musl
build; tests sleep with poll() instead of usleep for the same reason.
2026-10-08 22:35:34 +08:00
WHC 32b4afd772 v0.7.1: allow network inside the run_shell sandbox
Landlock no longer handles TCP, the Seatbelt profile drops (deny network*),
and the seccomp filter stops refusing IPv4/IPv6 sockets. Sandboxed commands
run without asking in auto mode on every backend (Low-IL no longer asks for
network commands), and connection errors are not treated as sandbox blocks.
2026-10-08 21:05:14 +08:00
WHC 895b47274c v0.7.0: image input, model registry, prefix-cache savings
Images
- vision gate: text-only models get a note instead of image parts and no
  view_image tool; unknown models retry once without images on HTTP 400
- paste: Ctrl+V and /attach paste read the clipboard (wl-paste, xclip,
  pngpaste/osascript, Win32 PNG or CF_DIB converted to PNG); pasting image
  file paths attaches them
- MCP image content is stored in the session and sent with the tool result
- attach: rel path validation, base64 decode, /retry keeps images

Model registry
- built-in src/models_builtin.json embedded at build time, merged with
  <config>/models.json and cached GET /models metadata (context, vision,
  pricing); most specific rule wins per field
- /model labels, /status source, --list-models [provider]

Cache and cost
- stable request prefix (frozen tools, notes as messages), DeepSeek prefix
  completion, OpenAI/Anthropic cache hints, /cost and hit rate in the status
2026-10-08 20:34:09 +08:00
WHC 0226f417ff providers: add teamorouter; pick the first /models id when no default model 2026-10-08 14:30:00 +08:00
WHC 1772c26d43 ci: auto-publish Windows zip to Gitea Releases
After a successful windows-cross build on main or v* tags, upload the zip
and .sha256 via scripts/release_windows.sh (creates/updates v$VERSION).
Prefer secrets.RELEASE_TOKEN; fall back to GITHUB_TOKEN. Skip on PRs.
2026-10-07 21:43:41 +08:00
WHC 630e8f4295 ci: Windows zip only builds cst-agent; link util for replay tests
os_win.c needs cst_buf_* / cst_utf8_valid from util.c after os_compat was
added to the small replay CMake targets. Release packaging only needs
cst-agent.exe, so bootstrap_windows.sh builds that target alone.
2026-10-07 21:09:44 +08:00
WHC 94e8746091 ci: multi-mirror GitHub clone for mingw third_party
windows-cross died cloning busybox-w32 through a global gitclone.com
insteadOf (HTTP 502). Try ghfast/ghproxy/gitclone/github per repo instead
of forcing one proxy for every fetch.
2026-10-07 11:43:12 +08:00
WHC 546a4c7eeb ci: link os_compat for replay CMake tests; fix llvm-mingw mirrors
gcc -O0 keeps unused cst_test_util.h helpers that need os_*; clang GC'd
them so only the gcc CMake step failed. Prefer ghfast/ghproxy.net for the
llvm-mingw tarball (gitclone 500 / github unreachable on the runner).
2026-10-07 02:57:38 +08:00
WHC 9b2c7acfae ci: fix test_subagent strcasestr crash; llvm-mingw Windows zip
clang under strict C11 left strcasestr undeclared (returns int → pointer
truncation → SIGSEGV). Parse Content-Length portably; keep -Werror on
make check. Drop Nix for windows-cross (root /nix fails) and use pinned
llvm-mingw UCRT instead; Alpine musl gets linux-headers for audit.h.
2026-10-07 02:39:36 +08:00
WHC 420f9b242c ci: drop sanitizers job; harden test_subagent fake server
test_subagent segfaulted in the Docker runner: the fake SSE handler took
user + strlen(user) - 1 with an empty user string. Guard JSON access and
format the child report with an empty-safe last character. Remove the
sanitizers job entirely (if: false still queued it and blocked the runner).
2026-10-07 02:27:00 +08:00
WHC e5ea869681 ci: install unzip for windows-cross zip listing 2026-10-07 02:21:28 +08:00
WHC d6ae1a67c4 ci: drop macOS, skip ASan, checkout without GitHub, MinGW zip
- Replace actions/checkout (pulls from github.com) with scripts/ci_checkout.sh
  fetched from this Gitea over raw/commit.
- Remove the macOS job (no runner).
- Disable sanitizers until the Docker runner has SYS_PTRACE; it was wedging
  the only slot with AddressSanitizer:DEADLYSIGNAL.
- Add windows-cross: Nix UCRT mingw via TUNA, bootstrap_windows.sh, verify
  dist/cst-agent-*-windows-x86_64.zip. Third-party git clones go through
  gitclone.com.
2026-10-07 02:21:18 +08:00
WHC 41c7e04940 ci: cancel stuck runs; cap ASan wall time; silence path snprintf warning
The sanitizers job wedged the only runner by printing AddressSanitizer
DEADLYSIGNAL forever (Docker without SYS_PTRACE). Concurrency now cancels
in-progress jobs on a new push, and make asan is wrapped in timeout -s KILL
480 with shallow ASAN_OPTIONS.

Also copy the path with memcpy after the length check so -O1 asan builds
do not warn about snprintf truncation into OS_PATH_MAX.
2026-10-07 01:55:51 +08:00
WHC 77de812207 sashidown: keep -Iinclude when parent make overrides CFLAGS
CI passes CFLAGS=-Werror on the command line; that wiped the library's
CFLAGS ?= ... -Iinclude -Isrc, so fsm.h could not find sashidown.h.
Always append the include paths via override CPPFLAGS.
2026-10-07 01:52:42 +08:00
WHC 9ec21b7fc0 ci: use Tsinghua apt/apk mirrors on the runner
Install was pulling bookworm packages from deb.debian.org; the domestic
Gitea runner crawls that for minutes. Rewrite sources to TUNA before
apt-get/apk, with CST_CI_MIRROR to override.
2026-10-07 01:42:56 +08:00
WHC daa92a6cfb seccomp: require child memory peek in the availability probe
With Yama ptrace_scope=2 and no CAP_SYS_PTRACE, process_vm_readv and
/proc/<pid>/mem both fail, so sc_decide cannot read path strings and every
workspace write gets EACCES. The old probe only checked the USER_NOTIF
round trip, so CST_SANDBOX=seccomp and the forced-seccomp test pass looked
available and then failed on the first write.

sc_can_peek() forks a child, places a marker on its stack, and retries the
same sc_peek path. sc_probe() now needs that too, so those hosts fall back
to none (or keep landlock) instead of a silent deny-all sandbox.
2026-10-07 01:39:08 +08:00
WHC 1476f1774a Temp dir falls back when TMPDIR or /tmp is unusable (Termux); v0.6.1
os_temp_dir() used $TMPDIR or "/tmp" without checking either. On Termux
/tmp usually does not exist (or SELinux denies it) unless TMPDIR points at
$PREFIX/tmp, so plugin and MCP temp files failed and shell children that
need a temp file (heredocs) broke.

  * os_temp_dir() picks, once, the first of $TMPDIR, $PREFIX/tmp, /tmp,
    /var/tmp and ~/.cache/cst-agent/tmp where a file can really be
    created; if none works it keeps the old answer
  * os_temp_dir_export() at startup writes the choice back to TMPDIR, so
    shell children and the sandbox's writable temp dir agree
  * tests/test_tmpdir.c covers the fallbacks in fresh forks
2026-10-06 23:32:21 +08:00
WHC 2a43ae6622 Windows: official port -- UCRT static build, UTF-8 throughout, Low IL sandbox, ncurses TUI, installer; v0.6.0
Build
  * cross build switches to pkgsCross.ucrt64 (shell-win.nix); curl (Schannel),
    winpthreads and ncurses 6.6 are linked statically, so the release is
    cst-agent.exe + busybox.exe (busybox-w32 Unicode) with no extra DLLs
  * windres embeds a manifest with activeCodePage=UTF-8 and a version resource
  * ncurses 6.6 Windows console driver patches (mouse button state, A_DIM,
    wide-char declarations): scripts/patches/
  * scripts/bootstrap_msys2.sh builds the same release natively in MSYS2
    UCRT64; PR #1's MINGW64 notes now point there
  * vendor/tre-regex (musl's TRE) provides <regex.h> on MinGW

UTF-8
  * paths cross the system call boundary as UTF-16 (*W APIs); console code
    pages set to 65001 and restored on exit; files are read and written in
    binary mode
  * child output that is not valid UTF-8 is decoded from the console, OEM or
    ANSI system code page
  * read_file hides CR; edit_file, apply_patch and write_file keep CRLF line
    endings and a UTF-8 BOM

OS layer and processes
  * os_dir_* and cst_fnmatch replace the POSIX-only directory walks in
    subagent, plugins, skills, undo and fs_tools; realpath / path_under and
    symlink rejection work on Windows
  * every child goes through CreateProcessW with its own environment block
    inside a Job Object, so timeouts and Esc end the whole process tree

Sandbox (auto mode, "auto · lowil")
  * run_shell runs with a restricted Low integrity token; the workspace gets
    an inheritable Low label once, the sandbox TEMP is labeled Low, and the
    credentials file gets a no-read-up label
  * drive roots, the profile and system folders are never labeled
  * network commands still ask, since Low IL does not block the network;
    Access denied and MSYS2's 0xc0000142 offer a rerun outside the sandbox
  * /sandbox lists labeled folders, /sandbox reset-labels restores them

TUI
  * raster console fonts are switched to Consolas / NSimSun while running
  * the home logo falls back to text when the console font lacks U+2580
  * mouse drag copy goes to the Windows clipboard

Distribution
  * scripts/install_windows.ps1 installs a Gitea release into
    %LOCALAPPDATA%\Programs\cst-agent and adds it to the user PATH;
    cst-agent-update.cmd reruns it; scripts/release_windows.sh uploads the
    zip with the token taken from the environment
  * scripts/run_windows.bat is gone; docs/README-windows.md is rewritten

Tests
  * new test_lowil, test_win_utf8 and test_os_dir; tests made portable
    (cst_test_util.h)
  * scripts/win_test.sh and scripts/winvm/ run the suite on English and
    Chinese Windows Server VMs as Administrator, a standard user and a
    Chinese user name: 29/29 on all six
2026-10-06 23:08:14 +08:00
XYZ 9862c29f58 Windows: point libcurl at a CA bundle
An OpenSSL-based libcurl keeps the CA path it was built with. MSYS2's is
"/mingw64/etc/ssl/certs/ca-bundle.crt", which resolves to nothing once the
binary runs outside an MSYS2 shell, and that build ignores CURL_CA_BUNDLE,
SSL_CERT_FILE and SSL_CERT_DIR. Every HTTPS request then failed with
CURLE_SSL_CACERT_BADFILE (77).

llm_apply_ca_bundle() sets CURLOPT_CAINFO from $CST_CA_BUNDLE, falling back
to ca-bundle.crt beside the executable. If neither exists nothing is set,
so a libcurl linked against Schannel (the Windows certificate store)
behaves exactly as before.
2026-10-06 17:56:56 +08:00
XYZ 929f7125b8 Windows: encode and decode text as UTF-8 instead of trusting the locale
setlocale(LC_ALL, "") maps to the ANSI code page on Windows -- 936 (GBK)
on a Chinese system -- so wcrtomb() produces GBK where the protocol wants
UTF-8, and mbrtowc() reads UTF-8 as GBK. Typed Chinese therefore reached
the model as mojibake, and once the buffer held correct UTF-8 the input
line rendered as mojibake instead. Windows' C runtime has no ".UTF-8"
locale to switch to, so the conversion has to be explicit.

os_compat.[ch]
  * add os_utf8_encode() and os_utf8_decode(), locale-independent, with
    range and overlong-form checks
src/tui.c
  * input_put_wc(), the command-palette filter and the ask_user buffer use
    os_utf8_encode() instead of wcrtomb()
src/textsel.c
  * cst_utf8_decode() uses os_utf8_decode() instead of mbrtowc()
2026-10-06 17:56:55 +08:00
149 changed files with 32445 additions and 1851 deletions
+2
View File
@@ -0,0 +1,2 @@
*.cmd text eol=crlf
*.bat text eol=crlf
+142 -42
View File
@@ -2,74 +2,174 @@ name: ci
on:
push:
branches: [main]
tags: ['v*']
pull_request:
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
linux:
# Verification: TinyCC builds and tests everything in seconds. tcc warns on
# `#pragma GCC`, so no -Werror here; the gcc release builds below carry it.
tcc-check:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
cc: [gcc, clang]
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Checkout
env:
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_SHA: ${{ github.sha }}
run: |
curl -fsSL "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/raw/commit/$GITHUB_SHA/scripts/ci_checkout.sh" \
-o /tmp/ci_checkout.sh
bash /tmp/ci_checkout.sh
- name: Install dependencies
run: |
SUDO=$(command -v sudo || true)
$SUDO bash scripts/ci_pkg_mirror.sh
$SUDO apt-get update
$SUDO apt-get install -y --no-install-recommends \
build-essential clang cmake pkg-config python3 \
build-essential pkg-config python3 curl ca-certificates \
libcurl4-openssl-dev libncurses-dev
- name: Build (warnings are errors)
run: make CC=${{ matrix.cc }} CFLAGS="-std=c11 -Wall -Wextra -O2 -g -Werror" -j"$(nproc)"
- name: Build TinyCC
run: bash scripts/ci_tcc.sh
- name: Build
run: make CC=tcc AR=ar CFLAGS="-std=c11 -Wall -O2 -g" -j"$(nproc)"
- name: Unit tests
run: make CC=${{ matrix.cc }} check
- name: CMake build + ctest
run: |
cmake -B build -DCMAKE_C_COMPILER=${{ matrix.cc }} -DCST_BUILD_TUI=ON
cmake --build build -j"$(nproc)"
ctest --test-dir build --output-on-failure
run: make CC=tcc AR=ar CFLAGS="-std=c11 -Wall -O2 -g" check
- name: Smoke test
run: |
./cst-agent --help
./cst-agent --workspace "$PWD" --list-sessions
./cst-agent --demo-ctl --workspace .
- name: WebUI end-to-end (recorded stream, no API key)
run: sh scripts/e2e_webui.sh
sanitizers:
# Release: static musl binaries, gcc with -Werror.
# dist/cst-agent-VERSION-linux-ARCH.tar.gz
linux:
needs: tcc-check
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- arch: x86_64
check: 1
- arch: x86
check: 1
- arch: aarch64
check: 0
steps:
- uses: actions/checkout@v4
- name: Install dependencies
- name: Checkout
env:
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_SHA: ${{ github.sha }}
run: |
curl -fsSL "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/raw/commit/$GITHUB_SHA/scripts/ci_checkout.sh" \
-o /tmp/ci_checkout.sh
bash /tmp/ci_checkout.sh
- name: Install host tools
run: |
SUDO=$(command -v sudo || true)
$SUDO bash scripts/ci_pkg_mirror.sh
$SUDO apt-get update
# ncurses-bin/-term: tic, infocmp and the fallback terminal entries
$SUDO apt-get install -y --no-install-recommends \
build-essential pkg-config libcurl4-openssl-dev libncurses-dev
- name: ASan + UBSan
run: make asan
build-essential cmake ninja-build pkg-config python3 perl \
curl ca-certificates xz-utils bzip2 file ncurses-bin ncurses-term
if [ "${{ matrix.arch }}" = aarch64 ]; then
$SUDO apt-get install -y --no-install-recommends qemu-user-static
fi
- name: Build (static, -Werror)
env:
CST_LINUX_CHECK: ${{ matrix.check }}
run: bash scripts/package_linux.sh ${{ matrix.arch }}
- name: Smoke test
run: |
d=$(ls -d dist/cst-agent-*-linux-${{ matrix.arch }})
file "$d/cst-agent"
run=""
[ "${{ matrix.arch }}" = aarch64 ] && run=qemu-aarch64-static
$run "$d/cst-agent" --version
$run "$d/cst-agent" --help >/dev/null
$run "$d/cst-agent" --workspace "$PWD" --list-sessions
cat dist/cst-agent-*-linux-${{ matrix.arch }}.tar.gz.sha256
# Release vVERSION (from include/cst_version.h) on Gitea. Prefer repo
# secret RELEASE_TOKEN (write:repository); fall back to the job token.
- name: Publish Gitea Release
if: github.event_name == 'push'
env:
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CST_GITEA: ${{ github.server_url }}
CST_REPO: ${{ github.repository }}
GITHUB_REF: ${{ github.ref }}
CI: "true"
run: |
export GITEA_TOKEN="${RELEASE_TOKEN:-${GITHUB_TOKEN:-}}"
if [[ -z "${GITEA_TOKEN}" ]]; then
echo "::warning::no RELEASE_TOKEN / GITHUB_TOKEN; skip auto-release"
exit 0
fi
bash scripts/release.sh dist/cst-agent-*-linux-${{ matrix.arch }}.tar.gz
musl:
# Release: mingw-w64 UCRT gcc (Debian trixie) with -Werror.
# dist/cst-agent-VERSION-windows-x86_64.zip and -setup.exe (NSIS)
windows:
needs: tcc-check
runs-on: ubuntu-latest
container: alpine:3.20
timeout-minutes: 120
container: debian:trixie
steps:
- name: Install dependencies
run: apk add --no-cache build-base pkgconf curl-dev ncurses-dev git nodejs bash python3
- uses: actions/checkout@v4
- name: Build + unit tests
- name: Install tools
run: |
make CFLAGS="-std=c11 -Wall -Wextra -O2 -Werror" -j"$(nproc)"
make check
# Gitea instances rarely have macOS runners; set the repository variable
# HAVE_MACOS_RUNNER=true once one is registered.
macos:
if: ${{ vars.HAVE_MACOS_RUNNER == 'true' }}
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: brew install ncurses curl pkg-config
- name: Build + unit tests
# No CA certificates in the image yet, so the mirror is plain http.
sed -i 's#https\?://deb.debian.org#http://mirrors.tuna.tsinghua.edu.cn#g' \
/etc/apt/sources.list.d/debian.sources
apt-get update
apt-get install -y --no-install-recommends \
ca-certificates curl git build-essential cmake ninja-build make \
pkg-config python3 perl zip unzip xz-utils patch file \
gcc-mingw-w64-ucrt64 binutils-mingw-w64-ucrt64 nsis
- name: Checkout
env:
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_SHA: ${{ github.sha }}
run: |
make -j"$(sysctl -n hw.ncpu)"
make check
curl -fsSL "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/raw/commit/$GITHUB_SHA/scripts/ci_checkout.sh" \
-o /tmp/ci_checkout.sh
bash /tmp/ci_checkout.sh
- name: Cross-build zip and installer
env:
CST_WIN_CFLAGS: -Wall -Wextra -Werror
run: bash scripts/ci_windows_cross.sh
- name: Verify
run: |
ls -lah dist/windows-x86_64/
file dist/windows-x86_64/cst-agent.exe dist/cst-agent-*-windows-x86_64-setup.exe
cat dist/cst-agent-*-windows-x86_64.zip.sha256 dist/cst-agent-*-windows-x86_64-setup.exe.sha256
unzip -l dist/cst-agent-*-windows-x86_64.zip
- name: Publish Gitea Release
if: github.event_name == 'push'
env:
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CST_GITEA: ${{ github.server_url }}
CST_REPO: ${{ github.repository }}
GITHUB_REF: ${{ github.ref }}
CI: "true"
run: |
export GITEA_TOKEN="${RELEASE_TOKEN:-${GITHUB_TOKEN:-}}"
if [[ -z "${GITEA_TOKEN}" ]]; then
echo "::warning::no RELEASE_TOKEN / GITHUB_TOKEN; skip auto-release"
exit 0
fi
bash scripts/release.sh dist/cst-agent-*-windows-x86_64.zip \
dist/cst-agent-*-windows-x86_64-setup.exe
+7
View File
@@ -21,10 +21,17 @@ third_party/busybox/busybox
third_party/busybox/busybox.exe
third_party/curl/
third_party/PDCurses/
third_party/ncurses-*/
third_party/ncurses-*.tar.gz
third_party/busybox-w32/
third_party/busybox-src/
third_party/prefix-mingw/
third_party/prefix-linux-*/
third_party/build-linux-*/
third_party/src/
vendor/sashidown/*.o
vendor/sashidown/src/*.o
vendor/sashidown/libsashidown.a
vendor/sashidown/libsashidown-mingw.a
.gen/
.build-tmp/
+165 -19
View File
@@ -5,9 +5,36 @@ set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
option(CST_BUILD_TUI "Build ncurses/PDCurses TUI" ON)
option(CST_BUILD_WEBUI "Build the HTTP/SSE WebUI (--web)" ON)
option(CST_NATIVE_ARCH "Pass -march=native to sashidown" OFF)
option(CST_BUNDLE_BUSYBOX "Install placeholder/scripts for busybox beside binary" ON)
if(WIN32)
enable_language(RC)
# Cross builds link libgcc/winpthreads/curl statically; MSYS2's curl is a DLL.
if(CMAKE_CROSSCOMPILING)
set(_CST_STATIC_DEFAULT ON)
else()
set(_CST_STATIC_DEFAULT OFF)
endif()
option(CST_WIN_STATIC "Link Windows executables with -static" ${_CST_STATIC_DEFAULT})
endif()
# Windows: UTF-8 manifest (activeCodePage, longPathAware) and version resource
# on every executable, so tests run with the same process code page as the app.
function(cst_win_exe t)
if(NOT WIN32)
return()
endif()
target_sources(${t} PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/res/windows/cst-agent.rc")
target_include_directories(${t} PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}/include" "${CMAKE_CURRENT_SOURCE_DIR}/res/windows")
target_compile_definitions(${t} PRIVATE _WIN32_WINNT=0x0A00)
if(CST_WIN_STATIC)
target_link_options(${t} PRIVATE -static)
endif()
endfunction()
set(SD_DIR "${CMAKE_CURRENT_SOURCE_DIR}/vendor/sashidown")
if(CST_NATIVE_ARCH)
set(_SD_MAKE_EXTRA CST_NATIVE_ARCH=1)
@@ -15,6 +42,9 @@ else()
set(_SD_MAKE_EXTRA "")
endif()
# sashidown has a plain Makefile; CMAKE_MAKE_PROGRAM may be ninja.
find_program(CST_MAKE NAMES make gmake mingw32-make REQUIRED)
# Separate archive for MinGW so Linux host builds keep libsashidown.a
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
set(SASHIDOWN_A "${SD_DIR}/libsashidown-mingw.a")
@@ -26,19 +56,19 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
set(_MINGW_RANLIB "${CMAKE_RANLIB}")
add_custom_command(
OUTPUT "${SASHIDOWN_A}"
COMMAND ${CMAKE_MAKE_PROGRAM} -C "${SD_DIR}" clean
COMMAND ${CMAKE_MAKE_PROGRAM} -C "${SD_DIR}" libsashidown.a
COMMAND ${CST_MAKE} -C "${SD_DIR}" clean
COMMAND ${CST_MAKE} -C "${SD_DIR}" libsashidown.a
CC=${CMAKE_C_COMPILER} AR=${_MINGW_AR} RANLIB=${_MINGW_RANLIB}
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${SD_DIR}/libsashidown.a" "${SASHIDOWN_A}"
COMMAND ${CMAKE_MAKE_PROGRAM} -C "${SD_DIR}" clean
COMMAND ${CST_MAKE} -C "${SD_DIR}" clean
COMMENT "Building libsashidown for MinGW"
VERBATIM)
else()
set(SASHIDOWN_A "${SD_DIR}/libsashidown.a")
add_custom_command(
OUTPUT "${SASHIDOWN_A}"
COMMAND ${CMAKE_MAKE_PROGRAM} -C "${SD_DIR}" libsashidown.a CC=${CMAKE_C_COMPILER} ${_SD_MAKE_EXTRA}
COMMAND ${CST_MAKE} -C "${SD_DIR}" libsashidown.a CC=${CMAKE_C_COMPILER} ${_SD_MAKE_EXTRA}
COMMENT "Building libsashidown.a"
VERBATIM)
endif()
@@ -54,6 +84,7 @@ set(CST_CORE_SOURCES
src/util.c
src/ctl_parse.c
src/os_compat.c
src/os_win.c
src/sandbox.c
src/sandbox_exec.c
src/keymap.c
@@ -62,6 +93,9 @@ set(CST_CORE_SOURCES
src/plugins.c
src/mcp.c
src/subagent.c
src/attach.c
src/clipboard.c
src/pdf_text.c
src/session.c
src/tools/tools.c
src/tools/patch.c
@@ -78,10 +112,75 @@ set(CST_CORE_SOURCES
src/agent_store.c
src/json_baseline.c
src/creds.c
src/provider_switch.c
src/mm_tool.c
src/models.c
src/model_registry.c
src/metrics.c
src/net.c
src/http_srv.c
src/web_md.c
)
# Built-in model data as a C byte list (included by model_registry.c).
set(CST_GEN_DIR "${CMAKE_CURRENT_BINARY_DIR}/gen")
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/src/models_builtin.json")
# The WebUI shell (web/*) as C byte arrays, the same trick as the model data.
file(GLOB CST_WEB_ASSETS "${CMAKE_CURRENT_SOURCE_DIR}/web/*")
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${CST_WEB_ASSETS})
set(_cst_web_inc "/* Generated by CMake from web/ - do not edit. */\n")
set(_cst_web_table "")
foreach(_f ${CST_WEB_ASSETS})
if(IS_DIRECTORY "${_f}")
continue()
endif()
get_filename_component(_name "${_f}" NAME)
string(REGEX REPLACE "[^A-Za-z0-9]" "_" _var "wui_asset_${_name}")
file(READ "${_f}" _hex HEX)
string(REGEX REPLACE "([0-9a-f][0-9a-f])" "0x\\1," _hex "${_hex}")
set(_ctype "application/octet-stream")
if(_name MATCHES "\\.html$")
set(_ctype "text/html; charset=utf-8")
elseif(_name MATCHES "\\.js$")
set(_ctype "text/javascript; charset=utf-8")
elseif(_name MATCHES "\\.css$")
set(_ctype "text/css; charset=utf-8")
elseif(_name MATCHES "\\.svg$")
set(_ctype "image/svg+xml")
endif()
string(APPEND _cst_web_inc "static const unsigned char ${_var}[] = {${_hex}0x00};\n")
string(APPEND _cst_web_table " {\"${_name}\", \"${_ctype}\", ${_var}, sizeof(${_var}) - 1},\n")
endforeach()
string(APPEND _cst_web_inc "static const wui_asset_t wui_assets[] = {\n${_cst_web_table}};\n")
string(APPEND _cst_web_inc "static const int wui_asset_count = (int)(sizeof(wui_assets) / sizeof(wui_assets[0]));\n")
file(WRITE "${CST_GEN_DIR}/web_assets.inc" "${_cst_web_inc}")
if(NOT CST_BUILD_WEBUI)
list(APPEND CST_CORE_SOURCES src/webui_stub.c)
else()
list(APPEND CST_CORE_SOURCES src/webui.c)
endif()
file(READ "${CMAKE_CURRENT_SOURCE_DIR}/src/models_builtin.json" _cst_models_hex HEX)
string(REGEX REPLACE "([0-9a-f][0-9a-f])" "0x\\1," _cst_models_hex "${_cst_models_hex}")
file(WRITE "${CST_GEN_DIR}/models_builtin.inc" "${_cst_models_hex}\n")
include_directories("${CST_GEN_DIR}")
if(NOT CST_BUILD_WEBUI)
add_compile_definitions(CST_NO_WEBUI=1)
endif()
if(WIN32)
# MinGW has no <regex.h>: musl's TRE engine, UTF-8 aware (vendor/tre-regex)
include_directories(vendor/tre-regex)
set(_CST_TRE_SOURCES
vendor/tre-regex/regcomp.c vendor/tre-regex/regexec.c
vendor/tre-regex/regerror.c vendor/tre-regex/tre-mem.c)
list(APPEND CST_CORE_SOURCES ${_CST_TRE_SOURCES})
# Upstream code, kept as is: CST_WIN_CFLAGS=-Werror must not apply to it.
set_source_files_properties(${_CST_TRE_SOURCES} PROPERTIES COMPILE_OPTIONS "-w")
endif()
set(CST_SOURCES ${CST_CORE_SOURCES} src/main.c)
if(CST_BUILD_TUI)
@@ -92,6 +191,7 @@ endif()
add_executable(cst-agent ${CST_SOURCES})
add_dependencies(cst-agent sashidown)
cst_win_exe(cst-agent)
target_include_directories(cst-agent PRIVATE
include
@@ -113,9 +213,26 @@ if(NOT WIN32)
target_link_libraries(cst-agent PRIVATE m)
endif()
if(CST_BUILD_TUI)
set(CST_WIN_CURSES "ncurses" CACHE STRING "Windows TUI backend: ncurses or pdcurses")
set_property(CACHE CST_WIN_CURSES PROPERTY STRINGS ncurses pdcurses)
if(CST_BUILD_TUI AND WIN32 AND CST_WIN_CURSES STREQUAL "ncurses")
# ncursesw built by scripts/build_third_party_mingw.sh ncurses (or MSYS2's
# mingw-w64-ucrt-x86_64-ncurses); headers live in include/ncursesw.
find_path(NCURSESW_INCLUDE_DIR ncurses.h PATH_SUFFIXES ncursesw include/ncursesw)
find_library(NCURSESW_LIBRARY NAMES ncursesw)
if(NOT NCURSESW_INCLUDE_DIR OR NOT NCURSESW_LIBRARY)
message(FATAL_ERROR "CST_WIN_CURSES=ncurses needs ncursesw "
"(scripts/build_third_party_mingw.sh ncurses), or -DCST_WIN_CURSES=pdcurses")
endif()
# ncurses.h includes <ncursesw/ncurses_dll.h>, so the parent is needed too
get_filename_component(NCURSESW_PARENT_DIR ${NCURSESW_INCLUDE_DIR} DIRECTORY)
target_include_directories(cst-agent PRIVATE ${NCURSESW_INCLUDE_DIR} ${NCURSESW_PARENT_DIR})
target_link_libraries(cst-agent PRIVATE ${NCURSESW_LIBRARY})
target_compile_definitions(cst-agent PRIVATE CST_USE_NCURSES=1 NCURSES_STATIC=1 NCURSES_WIDECHAR=1)
message(STATUS "TUI: ncursesw (${NCURSESW_LIBRARY})")
elseif(CST_BUILD_TUI)
if(WIN32)
# Prefer PDCurses (pdcurses / curses). Users may set PDCurses_ROOT.
# PDCurses (pdcurses / curses). Users may set PDCurses_ROOT.
# PATH_SUFFIXES also covers the layout MSYS2's mingw-w64-pdcurses uses,
# which puts the headers under include/pdcurses/ rather than include/.
find_path(PDCURSES_INCLUDE_DIR curses.h
@@ -156,17 +273,27 @@ endif()
if(CMAKE_SYSTEM_NAME STREQUAL "Windows" OR WIN32)
# curl Schannel + networking; winmm is needed by PDCurses (PlaySoundW)
target_link_libraries(cst-agent PRIVATE
ws2_32 crypt32 secur32 bcrypt advapi32 winmm)
ws2_32 crypt32 secur32 bcrypt advapi32 user32 gdi32 winmm)
endif()
# --- tests ---
enable_testing()
set(_CST_SMALL_TESTS test_ctl_frontier test_ctl_heredoc test_json_baseline
test_replay_starvation test_replay_real test_mm_tool test_textsel
test_markdown)
test_pdf_text test_markdown)
foreach(t ${_CST_SMALL_TESTS})
if(t STREQUAL test_textsel)
add_executable(${t} tests/${t}.c src/textsel.c)
add_executable(${t} tests/${t}.c src/textsel.c src/os_compat.c src/os_win.c
src/util.c)
if(WIN32)
target_link_libraries(${t} PRIVATE advapi32 user32)
endif()
elseif(t STREQUAL test_pdf_text)
add_executable(${t} tests/${t}.c src/pdf_text.c src/os_compat.c src/os_win.c
src/util.c)
if(WIN32)
target_link_libraries(${t} PRIVATE advapi32 user32)
endif()
elseif(t STREQUAL test_markdown)
add_executable(${t} tests/${t}.c)
target_include_directories(${t} PRIVATE "${SD_DIR}/include")
@@ -177,33 +304,52 @@ foreach(t ${_CST_SMALL_TESTS})
elseif(t STREQUAL test_json_baseline)
add_executable(${t} tests/${t}.c src/json_baseline.c vendor/cjson/cJSON.c)
elseif(t STREQUAL test_replay_starvation OR t STREQUAL test_replay_real)
add_executable(${t} tests/${t}.c src/ctl_parse.c src/json_baseline.c vendor/cjson/cJSON.c)
# cst_test_util.h static helpers reference os_*; gcc -O0 keeps unused ones.
# On Windows os_win.c also needs util.c (cst_buf_*, cst_utf8_valid).
add_executable(${t} tests/${t}.c src/ctl_parse.c src/json_baseline.c
vendor/cjson/cJSON.c src/os_compat.c src/os_win.c src/util.c)
if(WIN32)
target_link_libraries(${t} PRIVATE advapi32 user32)
endif()
else()
add_executable(${t} tests/test_mm_tool.c src/mm_tool.c)
endif()
target_include_directories(${t} PRIVATE include "${CJSON_DIR}")
target_compile_definitions(${t} PRIVATE _GNU_SOURCE)
if(NOT WIN32)
target_link_libraries(${t} PRIVATE m)
target_link_libraries(${t} PRIVATE m Threads::Threads)
endif()
cst_win_exe(${t})
add_test(NAME ${t} COMMAND ${t} WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}")
endforeach()
# Tests that link the whole core (no TUI).
foreach(t test_llm_body test_sse_decode test_apply_patch test_session_roundtrip
test_shell test_diff test_tools_fs test_registry
test_undo test_config test_models test_sandbox
test_keymap test_shell_compact test_skills
test_plugins test_mcp test_subagent)
set(_CST_CORE_TESTS test_llm_body test_prefix_stable test_sse_decode
test_apply_patch test_session_roundtrip test_attach
test_shell test_diff test_tools_fs test_registry
test_undo test_config test_models test_model_registry test_sandbox
test_keymap test_shell_compact test_skills
test_plugins test_mcp test_subagent test_lowil
test_win_utf8 test_os_dir test_tmpdir)
if(CST_BUILD_WEBUI)
# test_webui drives the real server; the stub has nothing to test.
list(APPEND _CST_CORE_TESTS test_webui)
endif()
foreach(t ${_CST_CORE_TESTS})
add_executable(${t} tests/${t}.c ${CST_CORE_SOURCES})
target_include_directories(${t} PRIVATE include "${CJSON_DIR}")
# The core now contains src/web_md.c, which needs sashidown's header and
# archive exactly like the Makefile build does.
target_include_directories(${t} PRIVATE include "${CJSON_DIR}" "${SD_DIR}/include")
target_compile_definitions(${t} PRIVATE _GNU_SOURCE)
target_link_libraries(${t} PRIVATE CURL::libcurl Threads::Threads)
target_link_libraries(${t} PRIVATE
"${SASHIDOWN_A}" CURL::libcurl Threads::Threads)
add_dependencies(${t} sashidown)
if(NOT WIN32)
target_link_libraries(${t} PRIVATE m)
else()
target_link_libraries(${t} PRIVATE ws2_32 crypt32 secur32 bcrypt advapi32)
target_link_libraries(${t} PRIVATE ws2_32 crypt32 secur32 bcrypt advapi32 user32)
endif()
cst_win_exe(${t})
add_test(NAME ${t} COMMAND ${t} WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}")
endforeach()
+96 -24
View File
@@ -7,6 +7,7 @@ PREFIX ?= /usr/local
DESTDIR ?=
UNAME_S := $(shell uname -s 2>/dev/null)
UNAME_O := $(shell uname -o 2>/dev/null)
comma := ,
# Android (Termux): pthreads live in libc.
PTHREAD_LIBS := -lpthread
@@ -26,6 +27,15 @@ CPPFLAGS += -D_DARWIN_C_SOURCE
endif
PKGC = $(PKG_ENV) $(PKG_CONFIG)
# STATIC=1: fully static link (the musl release builds).
STATIC ?= 0
LINK_STATIC :=
ifeq ($(STATIC),1)
PKGC += --static
LINK_STATIC := -static
endif
LDFLAGS += $(LINK_STATIC)
# Wide-character curses: ncursesw, else ncurses (macOS/BSD build it wide).
CURSES_PC := $(shell for p in ncursesw ncurses; do $(PKGC) --exists $$p 2>/dev/null && { echo $$p; exit 0; }; done)
ifneq ($(CURSES_PC),)
@@ -43,23 +53,50 @@ ifeq ($(strip $(CURL_LIBS)),)
CURL_LIBS := -lcurl
endif
# WebUI (HTTP + SSE front end): src/net.c src/http_srv.c src/web_md.c
# src/webui.c. CST_WEBUI=0 swaps in the stub (--web then prints a hint).
CST_WEBUI ?= 1
PKG_CFLAGS := $(CURL_CFLAGS) $(CURSES_CFLAGS)
PKG_LIBS := $(CURL_LIBS) $(CURSES_LIBS)
CPPFLAGS += -Iinclude -I$(SD_DIR)/include -I$(CJSON_DIR) -D_XOPEN_SOURCE_EXTENDED $(PKG_CFLAGS)
# tcc writes no rpath of its own, so shared libs found via pkg-config -L
# (Nix store, /opt prefixes) would not load at run time.
RPATH_LIBS :=
ifneq ($(findstring tcc,$(notdir $(CC))),)
ifneq ($(STATIC),1)
RPATH_LIBS := $(patsubst -L%,-Wl$(comma)-rpath$(comma)%,$(filter -L%,$(PKG_LIBS)))
endif
endif
# Keep POSIX declarations visible for shared test helpers (notably symlink()).
CPPFLAGS += -Iinclude -I.gen -I$(SD_DIR)/include -I$(CJSON_DIR) -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE_EXTENDED $(PKG_CFLAGS)
LDFLAGS += -L$(SD_DIR)
LDLIBS += -lsashidown -lm $(PKG_LIBS) $(PTHREAD_LIBS)
LDLIBS += -lsashidown -lm $(PKG_LIBS) $(PTHREAD_LIBS) $(RPATH_LIBS)
WEB_DIR := web
WEB_ASSETS := $(wildcard $(WEB_DIR)/*.html) $(wildcard $(WEB_DIR)/*.js) \
$(wildcard $(WEB_DIR)/*.css) $(wildcard $(WEB_DIR)/*.svg)
ifeq ($(CST_WEBUI),0)
WEB_SRC := src/net.c src/http_srv.c src/web_md.c src/webui_stub.c
# The stub body is guarded by this, so the switch has to define it too.
CPPFLAGS += -DCST_NO_WEBUI=1
else
WEB_SRC := src/net.c src/http_srv.c src/web_md.c src/webui.c
endif
CORE_SRC = \
$(CJSON_DIR)/cJSON.c \
src/util.c \
src/ctl_parse.c \
src/os_compat.c \
src/os_win.c \
src/sandbox.c \
src/sandbox_exec.c \
src/keymap.c \
src/skills.c \
src/extensions.c src/plugins.c src/mcp.c src/subagent.c \
src/extensions.c src/plugins.c src/mcp.c src/subagent.c src/attach.c src/clipboard.c \
src/pdf_text.c \
src/session.c \
src/tools/tools.c \
src/tools/patch.c \
@@ -75,14 +112,16 @@ CORE_SRC = \
src/agent_loop.c \
src/agent_store.c \
src/json_baseline.c \
src/creds.c \
src/creds.c src/provider_switch.c \
src/mm_tool.c \
src/models.c \
src/metrics.c
src/models.c src/model_registry.c \
src/metrics.c \
$(WEB_SRC)
SRC = $(CORE_SRC) src/textsel.c src/tui.c src/main.c
OBJ = $(SRC:.c=.o)
HDRS = $(wildcard include/*.h) $(wildcard src/tools/*.h) $(CJSON_DIR)/cJSON.h
HDRS = $(wildcard include/*.h) $(wildcard src/tools/*.h) $(CJSON_DIR)/cJSON.h \
.gen/models_builtin.inc .gen/web_assets.inc
TESTS = \
tests/test_ctl_frontier \
@@ -92,9 +131,12 @@ TESTS = \
tests/test_replay_real \
tests/test_mm_tool \
tests/test_llm_body \
tests/test_prefix_stable \
tests/test_sse_decode \
tests/test_apply_patch \
tests/test_session_roundtrip \
tests/test_attach \
tests/test_pdf_text \
tests/test_shell \
tests/test_diff \
tests/test_tools_fs \
@@ -102,6 +144,7 @@ TESTS = \
tests/test_undo \
tests/test_config \
tests/test_models \
tests/test_model_registry \
tests/test_sandbox \
tests/test_keymap \
tests/test_textsel \
@@ -110,9 +153,21 @@ TESTS = \
tests/test_skills \
tests/test_plugins \
tests/test_mcp \
tests/test_subagent
tests/test_subagent \
tests/test_lowil \
tests/test_win_utf8 \
tests/test_os_dir \
tests/test_tmpdir
# test_webui drives the real server; there is nothing to test with the stub.
ifneq ($(CST_WEBUI),0)
TESTS += tests/test_webui
endif
TEST_LIBS = -lm $(CURL_LIBS) $(PTHREAD_LIBS)
TEST_SYS_LIBS = -lm $(PTHREAD_LIBS) $(LINK_STATIC)
TEST_LIBS = -L$(SD_DIR) -lsashidown $(CURL_LIBS) $(RPATH_LIBS) $(TEST_SYS_LIBS)
# cst_test_util.h has static os_* helpers; gcc drops them when unused, tcc
# keeps them, so the small tests link the OS layer too.
TEST_OS_SRC = src/os_compat.c src/os_win.c src/util.c
.PHONY: all clean check demo sashidown-lib FORCE cmake-gcc cmake-clang asan install uninstall
@@ -139,32 +194,45 @@ FORCE:
cst-agent: $(OBJ) $(SD_DIR)/libsashidown.a
$(CC) $(CFLAGS) -o $@ $(OBJ) $(LDFLAGS) $(LDLIBS)
# Built-in model data as a C byte list (included by model_registry.c).
.gen/models_builtin.inc: src/models_builtin.json
@mkdir -p .gen
od -An -v -tx1 $< | sed 's/\([0-9a-f][0-9a-f]\)/0x\1,/g' > $@.tmp && mv $@.tmp $@
# The WebUI shell is embedded the same way, one array per file.
.gen/web_assets.inc: $(WEB_ASSETS) scripts/embed_web.sh
@mkdir -p .gen
@sh scripts/embed_web.sh $@ $(WEB_DIR)
%.o: %.c $(HDRS)
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<
src/tui.o: src/tui.c $(HDRS) $(SD_DIR)/include/sashidown.h
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<
tests/test_ctl_frontier: tests/test_ctl_frontier.c src/ctl_parse.c include/ctl.h
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_ctl_frontier.c src/ctl_parse.c
tests/test_ctl_frontier: tests/test_ctl_frontier.c src/ctl_parse.c include/ctl.h $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_ctl_frontier.c src/ctl_parse.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_ctl_heredoc: tests/test_ctl_heredoc.c src/ctl_parse.c include/ctl.h
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_ctl_heredoc.c src/ctl_parse.c
tests/test_ctl_heredoc: tests/test_ctl_heredoc.c src/ctl_parse.c include/ctl.h $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_ctl_heredoc.c src/ctl_parse.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_json_baseline: tests/test_json_baseline.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(HDRS)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_json_baseline.c src/json_baseline.c $(CJSON_DIR)/cJSON.c -lm
tests/test_json_baseline: tests/test_json_baseline.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(HDRS) $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_json_baseline.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_replay_starvation: tests/test_replay_starvation.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_replay_starvation.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c -lm
tests/test_replay_starvation: tests/test_replay_starvation.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_replay_starvation.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_replay_real: tests/test_replay_real.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_replay_real.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c -lm
tests/test_replay_real: tests/test_replay_real.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_replay_real.c src/ctl_parse.c src/json_baseline.c $(CJSON_DIR)/cJSON.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_mm_tool: tests/test_mm_tool.c src/mm_tool.c include/mm_tool.h
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_mm_tool.c src/mm_tool.c
tests/test_mm_tool: tests/test_mm_tool.c src/mm_tool.c include/mm_tool.h $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_mm_tool.c src/mm_tool.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_textsel: tests/test_textsel.c src/textsel.c include/cst_textsel.h
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_textsel.c src/textsel.c
tests/test_textsel: tests/test_textsel.c src/textsel.c $(TEST_OS_SRC) include/cst_textsel.h
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_textsel.c src/textsel.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_pdf_text: tests/test_pdf_text.c src/pdf_text.c include/cst_pdf.h $(TEST_OS_SRC)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_pdf_text.c src/pdf_text.c $(TEST_OS_SRC) $(TEST_SYS_LIBS)
tests/test_markdown: tests/test_markdown.c $(SD_DIR)/libsashidown.a
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ tests/test_markdown.c $(LDFLAGS) -lsashidown -lm
@@ -172,10 +240,14 @@ tests/test_markdown: tests/test_markdown.c $(SD_DIR)/libsashidown.a
tests/test_%: tests/test_%.c $(CORE_SRC) $(HDRS)
$(CC) $(CFLAGS) $(CPPFLAGS) -o $@ $< $(CORE_SRC) $(TEST_LIBS)
$(TESTS): sashidown-lib
check: $(TESTS)
@set -e; for t in $(TESTS); do echo "== $$t"; ./$$t; done
# AddressSanitizer + UBSan build of the agent and all tests.
# Under a ptrace-based sandbox (some Termux setups) LeakSanitizer aborts with
# "Tracer caught signal 11"; add ASAN_OPTIONS=detect_leaks=0 there.
asan:
$(MAKE) clean
$(MAKE) sashidown-lib
@@ -185,7 +257,7 @@ demo: cst-agent
./cst-agent --demo-ctl --workspace .
clean:
rm -f $(OBJ) cst-agent $(TESTS)
rm -f $(OBJ) cst-agent $(TESTS) .gen/models_builtin.inc .gen/web_assets.inc
install: cst-agent
install -d "$(DESTDIR)$(PREFIX)/bin"
+109 -25
View File
@@ -27,6 +27,20 @@ cst-agent --version
cd ~/my-project && cst-agent
```
不想编译:[Release](https://git.whcself.xyz/WHC/CST-agent/releases) 里有静态链接的 Linux 单文件
(`cst-agent-VERSION-linux-{x86_64,x86,aarch64}.tar.gz`,musl,任何发行版都能直接跑),解压后把 `cst-agent`
放进 `PATH` 即可;每个文件旁边有 `.sha256`。
Windows 10 1903+(PowerShell,不需要管理员):
```powershell
irm https://git.whcself.xyz/WHC/CST-agent/raw/branch/main/scripts/install_windows.ps1 | iex
# 新开终端后:cd 到项目目录运行 cst-agent;升级用 cst-agent-update
```
也可以下载 Release 里的安装包 `cst-agent-VERSION-windows-x86_64-setup.exe`(按用户安装、自动加 PATH、
开始菜单有 WebUI 入口),或便携版 zip。详见 [docs/README-windows.md](docs/README-windows.md)。
一键脚本可以重复运行:已有仓库会 `git pull --ff-only`,代码有更新时干净重编,然后重新安装;
同时装一个 `cst-agent-update` 命令(等同重跑脚本)。NixOS 上会自动进 `nix-shell` 编译。
@@ -39,8 +53,10 @@ cd ~/my-project && cst-agent
|---|---|
| Linux | [`scripts/bootstrap_linux.sh`](scripts/bootstrap_linux.sh) |
| Termux | [`scripts/bootstrap_termux.sh`](scripts/bootstrap_termux.sh) |
| Windows 交叉构建(在 Linux 上) | [`scripts/bootstrap_windows.sh`](scripts/bootstrap_windows.sh) → `dist/windows-x86_64/` |
| Windows 运行 | 拷贝 dist 后 [`scripts/run_windows.bat`](scripts/run_windows.bat) |
| Windows 安装 / 升级 | [`scripts/install_windows.ps1`](scripts/install_windows.ps1)(从 Release 安装),之后 `cst-agent-update` |
| Linux 静态发行包(musl 交叉) | [`scripts/package_linux.sh`](scripts/package_linux.sh) `x86_64\|x86\|aarch64` → `dist/*.tar.gz` |
| Windows 交叉构建(在 Linux 上) | [`scripts/bootstrap_windows.sh`](scripts/bootstrap_windows.sh) → `dist/windows-x86_64/`、zip 和安装包 |
| Windows 原生构建(MSYS2 UCRT64) | [`scripts/bootstrap_msys2.sh`](scripts/bootstrap_msys2.sh) → `dist/windows-x86_64/` |
### 迷你验收工程(论文案例)
@@ -76,6 +92,7 @@ cd CST-agent
make # 依赖 pkg-config 找 libcurl / ncursesw(找不到时退回 ncurses)
make check # 单元测试 + 截断实验(含真实录制流)
make asan # AddressSanitizer + UBSan 构建并跑全部测试
make CC=tcc AR=ar check # TinyCC:整套构建 + 测试十几秒(CI 的校验就是这条)
make install PREFIX=$HOME/.local # 或 sudo make install(默认 /usr/local)
```
@@ -97,14 +114,11 @@ cmake --build build-clang
# Headless only(无 ncurses)
cmake -B build-hd -DCST_BUILD_TUI=OFF && cmake --build build-hd
# Windows MinGW 交叉(需安装 mingw-w64)
cmake -B build-win -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
-DCST_BUILD_TUI=OFF
cmake --build build-win
# TUI:安装 PDCurses 后 -DCST_BUILD_TUI=ON -DPDCurses_ROOT=...
# Windows:用一键脚本(见下方“Windows 发行包”),会先编译静态依赖
./scripts/bootstrap_windows.sh
```
依赖:`gcc` 或 `clang`、`libcurl`、Linux TUI 需 `ncursesw`;Windows TUI 需 **PDCurses**。
依赖:`gcc`、`clang` 或 `tcc`(0.9.27 mob 以上)、`libcurl`、TUI 需 `ncursesw`(Windows 版用打过补丁的 ncurses 6.6,脚本自动编译)。
Sashidown(`vendor/sashidown`)与 cJSON(`vendor/cjson`,MIT)已随仓库提供。
API Key:TUI 里输入并按 provider 保存到配置目录的 `credentials`(权限 0600);`/key` 更换,`--ask-key` 在命令行输入。
`--api-key` 只用于本次进程、不落盘;`DEEPSEEK_API_KEY` 等环境变量和 `CST_API_KEY` 仅作为没有保存 key 时的兜底(方便 CI)。
@@ -113,11 +127,28 @@ API Key:TUI 里输入并按 provider 保存到配置目录的 `credentials`(
| OS | Arch | 编译器 | Headless | TUI | Shell |
|---|---|---|---|---|---|
| Linux(glibc / musl) | x86_64 / aarch64 | GCC, Clang | yes | ncursesw | `/bin/sh` 或旁路 `busybox` |
| Linux(glibc / musl) | x86_64 / x86 / aarch64 | GCC, Clang, TinyCC | yes | ncursesw | `/bin/sh` 或旁路 `busybox` |
| WSL2 | x86_64 | 同 Linux | yes | ncursesw | `/bin/sh`(工作区建议放在 Linux 文件系统内) |
| macOS | arm64 / x86_64 | Apple Clang | yes | ncurses(Homebrew) | `/bin/sh` |
| Android Termux | aarch64 / … | Clang (Termux) | yes | ncurses(Termux 包) | `$PREFIX/bin/sh`;可旁路 busybox |
| Windows | x86_64 / aarch64 | MinGW GCC/Clang | yes | PDCurses | **捆绑 busybox-w32** |
| Windows 10 1903+ | x86_64 | MinGW-w64 GCC(UCRT) | yes | ncurses 6.6(Windows 控制台) | **捆绑 busybox-w32** |
### WebUI(`--web`)
同一个二进制内置一个 HTTP + SSE 前端:`cst-agent --web`(默认 `http://127.0.0.1:8787/`)。
它和 TUI 是同一引擎的两个前端,浏览器里能流式看回答、点审批、切会话、贴图。
协议是一份稳定的 JSON + SSE 接口,供以后的专用客户端复用。详见 [docs/webui.md](docs/webui.md)。
```bash
cst-agent --web # 本机浏览器打开 http://127.0.0.1:8787/
cst-agent --web --web-port 0 # 端口交给系统,启动日志里给 URL
cst-agent --web --web-dev ./web # 开发前端:直接读目录,改完刷新
cst-agent --web --web-host 0.0.0.0 --web-token 口令 # 手机/局域网访问
sh scripts/e2e_webui.sh # 用录制流跑完整轮次,不需要 API key
```
不带 TUI 的构建也可以开 WebUI(`cmake -DCST_BUILD_WEBUI=ON -DCST_BUILD_TUI=OFF`);
`-DCST_BUILD_WEBUI=OFF` 时 `--web` 只打印一条提示。
### Termux
@@ -137,20 +168,40 @@ make && make install # 装到 Termux 的 $PREFIX/bin,之后任意目录
```
环境:`PREFIX` 自动用于找 `sh`;`CST_EXE_DIR` / `argv[0]` 用于定位旁路 busybox(`/proc/self/exe` 受限时)。
临时目录:依次试 `$TMPDIR`、`$PREFIX/tmp`、`/tmp`、`/var/tmp`、`~/.cache/cst-agent/tmp`,用第一个真能建文件的,
并写回 `TMPDIR` 给 shell 子进程和沙箱用(Termux 的 `/tmp` 常常不存在或被 SELinux 拒绝)。
### Linux 静态发行包
```bash
scripts/package_linux.sh x86_64 # 或 x86 / aarch64;CST_LINUX_CHECK=1 顺便跑 make check
```
用 Bootlin 的 musl 交叉 gcc(固定版本、校验 sha256)静态编译 mbedTLS、curl 8.22(只开 HTTP/HTTPS)和宽字符 ncurses,
再以 `STATIC=1` 和 `-Werror` 编出 `cst-agent`,产物是 `dist/cst-agent-VERSION-linux-ARCH.tar.gz`(+ `.sha256`)。
宿主需要 `cmake ninja make perl` 以及 `tic`/`infocmp` 和终端条目(Debian:`ncurses-bin ncurses-term`)。
ncurses 会去常见目录找 terminfo,找不到时用编进去的 `xterm-256color`、`xterm`、`screen-256color`、`tmux-256color`、`linux`、`vt100`。
CA 证书:`CST_CA_BUNDLE` → `SSL_CERT_FILE` → 各发行版的常见位置(Debian/Ubuntu、Fedora/RHEL、openSUSE、Alpine)。
### Windows 发行包
见 [docs/README-windows.md](docs/README-windows.md)。交叉构建:
见 [docs/README-windows.md](docs/README-windows.md)(安装、编码、沙箱、构建、已知限制)。
```bash
./scripts/build_third_party_mingw.sh all # curl+PDCurses+busybox-w32
# 源码:https://github.com/rmyorston/busybox-w32.git
# https://github.com/mirror/busybox.git (Unix)
cmake -B build-win-tui -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
-DCST_BUILD_TUI=ON -DCMAKE_PREFIX_PATH=$PWD/third_party/prefix-mingw
./scripts/bootstrap_windows.sh # Linux 交叉编译;Windows 上在 MSYS2 UCRT64 里用 scripts/bootstrap_msys2.sh
```
产物示例:`dist/windows-x86_64/`(含 `cst-agent.exe` + `busybox.exe` + `libwinpthread-1.dll`)。
产物:`dist/windows-x86_64/`(静态链接的 `cst-agent.exe` + `busybox.exe`,无其他 DLL)、
`dist/cst-agent-VERSION-windows-x86_64.zip`,以及装了 `makensis` 时的安装包 `dist/cst-agent-VERSION-windows-x86_64-setup.exe`。
### CI 与 Release
CI 用 TinyCC 校验(构建、`make check`、WebUI 端到端),通过后用 gcc(`-Werror`)出发行包:
Linux x86_64 / x86 / aarch64 静态包(x86 两种跑 `make check`,aarch64 用 qemu 冒烟),
Windows x86_64 的 zip 和安装包(Debian 的 mingw-w64 UCRT gcc + NSIS)。
推送到 `main` 或 `v*` tag 时每个任务把自己的产物上传到 Gitea Release `vVERSION`(同名资源会被替换);
仓库可设 `RELEASE_TOKEN`(`write:repository`),否则用任务自带 token。
本地:`GITEA_TOKEN=... scripts/release.sh dist/cst-agent-*.tar.gz dist/cst-agent-*.zip dist/cst-agent-*-setup.exe`。
环境变量:`CST_SHELL`、`CST_EXE_DIR`、`CST_AUTO_APPROVE` 等。
@@ -163,6 +214,7 @@ cmake -B build-win-tui -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
| `minimax`(默认) | `https://api.minimax.cn/v1` | `MiniMax-M2.7` | `MINIMAX_API_KEY` |
| `deepseek` | `https://api.deepseek.com/v1` | `deepseek-flash`(另有 `deepseek-v4-pro`) | `DEEPSEEK_API_KEY` |
| `openai` | `https://api.openai.com/v1` | `gpt-4.1-mini` | `OPENAI_API_KEY` |
| `teamorouter` | `https://api.teamorouter.com/v1` | 登录后自动取 `/models` 第一个(`/model` 可换) | `TEAMOROUTER_API_KEY` |
| 用途 | 来源(优先级从高到低) |
|---|---|
@@ -181,18 +233,20 @@ cmake -B build-win-tui -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
|---|---|
| `CST_PROTOCOL` / `--protocol` | 工具调用协议(`native-eos`) |
| `CST_TOOLS` / `--tools` | 工具集:`default`(全部)/ `minimal`(论文 4 件套) |
| `CST_MODE` / `--mode` | 工作模式:`coding`(默认,代码口径)/ `general`(通用助手口径) |
| `CST_EFFORT` / `--effort` | 推理强度 `low` / `medium` / `high` / `max` |
| `CST_CONFIRM` / `--confirm` | 哪些类别要确认:`shell,write,net`(默认)或 `none` |
| `CST_MAX_STEPS` / `--max-steps` | 每个用户回合最多调用模型几次(30) |
| `CST_TOOL_OUT_CAP` | 单个工具输出发给模型的字节上限(16384,保留头 60% + 尾 40%) |
| `CST_CTX_WINDOW` | 发给模型的最近非 system 消息条数(80) |
| `CST_CTX_TOKENS` / `CST_AUTO_COMPACT=0` | 覆盖模型上下文窗口大小 / 关闭自动压缩(用到 80% 时自动 `/compact`) |
| `CST_CTX_TOKENS` / `CST_AUTO_COMPACT=0` | 覆盖模型上下文窗口大小 / 关闭自动压缩(用到 75% 时自动 `/compact`) |
| `CST_NO_PROJECT_NOTES=1` | 不把工作区的 `AGENTS.md` 注入 system prompt |
| `NO_COLOR` / `CST_ASCII=1` | 关闭颜色 / 强制 ASCII 图形(非 UTF-8 终端自动退化) |
| `CST_SHELL_TIMEOUT` | `run_shell` 超时秒数(120),超时整组进程被杀,exit=124 |
| `CST_SHELL_TIMEOUT` | `run_shell` 超时秒数(默认不限),设了才在超时后整组杀进程,exit=124 |
| `CST_METRICS=0` / `CST_METRICS_PATH` | 关闭 / 指定计时埋点 JSONL(默认 `.agent/metrics/<session>.jsonl`) |
| `CST_RECORD_SSE=DIR` / `CST_REPLAY_SSE=DIR` | 录制 / 回放原始 SSE 流(`NNNN.sse` + `NNNN.req.json`) |
| `CST_APPROVAL=ask\|auto\|yolo` / `CST_SANDBOX=off\|landlock\|seccomp` | 审批模式 / 关闭沙箱(回落到只读白名单规则)或只用指定沙箱 |
| `CST_CA_BUNDLE` | HTTPS 用的 CA 证书文件(默认:`SSL_CERT_FILE`、libcurl 自带路径、常见发行版位置;Windows 用系统证书库) |
| `CST_MODELS`、`CST_AUTO_APPROVE=1`、`CST_SHELL`、`CST_EXE_DIR` | 模型列表、自动批准(= yolo)、shell、旁路 busybox |
## Run
@@ -223,6 +277,7 @@ cst-agent --headless --prompt 'List files'
| `-y` / `--yolo` | 等同 `--approval yolo`,只对本次运行有效 |
| `-C LIST` / `--confirm LIST` | 需要确认的类别:`shell,write,net` 的任意组合或 `none` |
| `-T PROFILE` / `--tools PROFILE` | `default` 或 `minimal` |
| `-M MODE` / `--mode MODE` | 工作模式 `coding`(默认)/ `general`,也读 `CST_MODE`,并记进 `settings.json` |
| `-P NAME` / `-m NAME` / `-e LEVEL` | 本次使用的 provider / 模型 / 推理强度(不传则用上次的) |
| `-a` / `--ask-key` | 在命令行输入并保存当前 provider 的 key |
@@ -244,6 +299,8 @@ cst-agent --headless --prompt 'List files'
| `delete_file` / `move_file` | 写 | 删除 / 移动(重命名)文件 |
| `run_shell` | shell | 非交互命令,超时整组进程被杀,Esc 可中断 |
| `web_fetch` | 网络 | 抓取 URL,HTML 转纯文本 |
| `clipboard_read` | 读 | 读系统剪贴板里的文本(`max_bytes` 默认 64 KiB,超出截断并说明) |
| `clipboard_write` | 写 | 把文本放进系统剪贴板(按写类权限确认;本机走 pbcopy / wl-copy / xclip / Win32 API) |
| `todo_write` | 元 | 维护任务清单(TUI 右侧面板,Ctrl+T 开关) |
| `ask_user` | 询问 | 向用户提问(可给选项);headless 下返回错误让模型自行决定 |
| `skill` | 元 | 加载一个 skill 的说明(`SKILL.md`)或其中的文件;有 skill 时才出现 |
@@ -255,6 +312,26 @@ cst-agent --headless --prompt 'List files'
`--tools minimal` 只保留 `read_file` / `run_shell` / `write_file` / `apply_patch`(论文实验口径)。
TUI 中 `/tools off NAME`、`/tools on NAME` 可临时开关单个工具。
### 工作模式(coding / general)
`--mode general`(或 TUI 里 `/mode general`、WebUI 运行设置里的「工作模式」、`CST_MODE=general`)把默认口径
从"代码助手"换成"通用助手":工作流从改代码/跑测试换成"先弄清要什么、再动手、给出处和不确定的地方",
工具说明从"仓库"改成"文件与数据",并说明 `web_fetch` 可以拿外部资料;`/status` 第一行显示当前模式,
切换后 history 里会留一条系统提示。默认仍是 `coding`,选择记在 `settings.json`;
`--tools minimal`(论文口径)的提示词按字节冻结,不受模式影响。
### 剪贴板与文档附件
- **剪贴板**:`clipboard_read` / `clipboard_write` 两个工具(写剪贴板是不可逆发射,要确认);
TUI 里 `/attach paste` 在剪贴板里没有图片时,把剪贴板的**文本**放进输入框(丢掉 CR 与控制字节,尊重输入长度上限)。
- **附件**(`/attach <path>`、拖入终端、剪贴板图片都算):图片按原样存进会话目录,发给模型时是 image part;
**PDF 在摄取时就抽出文本层**(零依赖解析器:Flate / ASCIIHex / ASCII85、对象流、CID 字体 ToUnicode、
内联图像跳过),存的是文本、`mime` 仍是 `application/pdf`;其余文件必须是 UTF-8 文本。
文档以 **text part 内联**发出,所以不受 `CST_NO_IMAGES` 或非视觉模型影响;图片照旧会被换成一行"未发送"说明。
上限:单个附件 ≤16 MB(文档文本 ≤4 MiB),每条消息最多 8 个。
加密 PDF、扫描件(无文本层)会被明确拒绝并说明原因;
`view_image` 仍只收图片,文本/PDF 走 `/attach`。
### 权限、撤销与项目说明
- **审批模式**(Shift+Tab 循环切换,`/approval`,`--approval`,`CST_APPROVAL`):
@@ -268,15 +345,19 @@ TUI 中 `/tools off NAME`、`/tools on NAME` 可临时开关单个工具。
`ask` / `auto` 的选择会记在 `settings.json`;`yolo` 只在本次运行有效。headless 行为不变(全部自动)。
- **沙箱**(C 实现,`src/sandbox_exec.c`):
- Linux:Landlock(内核 5.13+,直接 syscall,无需 libseccomp)。子进程只能写工作区、临时目录、`/dev/null`、`/dev/tty`;
读和执行不受限。Landlock ABI ≥ 4(内核 6.7+)时同时禁止 TCP 连接和监听。
读、执行和联网不受限。
- Linux 没有 Landlock 时(Android / Termux、WSL、老发行版;内核 5.5+,x86_64 / aarch64):seccomp user-notify
(界面显示 `auto · seccomp`)。子进程的写类系统调用(带写标志的 open、创建 / 删除 / 改名 / 链接 / chmod 等)先交给
agent 里的监督线程,按真实路径判断是否在工作区、临时目录或 `/dev/null` 等之内,不在就返回 `EACCES`;IPv4/IPv6 socket 一律拒绝。
agent 里的监督线程,按真实路径判断是否在工作区、临时目录或 `/dev/null` 等之内,不在就返回 `EACCES`。联网不受限。
读、执行和已打开文件的写入不检查。它防的是误操作而不是恶意程序:检查之后内核会再读一次路径,多线程程序可以趁机换掉。
探测还会确认能读直接子进程的内存;`ptrace_scope=2` 且无 `CAP_SYS_PTRACE` 时 seccomp 不会启用。
比 shell 活得久的后台进程:桌面发行版开着 Yama(`ptrace_scope` ≥ 1)时它们的受检写入会被拒;命令结束后一律失败。
- macOS:`sandbox_init` profile,同样只放行工作区和临时目录写入,禁网。
- Windows、更老的内核:没有沙箱,自动回落到规则判断(界面显示 `auto · rules`)。
- `CST_SANDBOX=off` 强制关闭沙箱;`CST_SANDBOX=landlock` / `seccomp` 只用指定的那种。
- macOS:`sandbox_init` profile,同样只放行工作区和临时目录写入,不限制联网。
- Windows 10 1903+:Low 完整性级别的受限令牌 + Job Object(界面显示 `auto · lowil`)。第一次在工作区运行时
给它打上 Low 标签,之后子进程只能写工作区和沙箱临时目录,读不到凭据文件;不限制联网。
`/sandbox` 查看、`/sandbox reset-labels` 撤销标签,详见 [docs/README-windows.md](docs/README-windows.md)。
- 更老的内核:没有沙箱,自动回落到规则判断(界面显示 `auto · rules`)。
- `CST_SANDBOX=off` 强制关闭沙箱;`CST_SANDBOX=landlock` / `seccomp` / `lowil` 只用指定的那种。
- 沙箱里的命令如果像是被拦了(`Permission denied` / `权限不够` / `Read-only file system` 等),会弹窗问
“Rerun outside the sandbox?”:`y` 在沙箱外重跑一次 · `p` 保存命令前缀规则并重跑 · `n` 保留沙箱结果(模型会收到说明)。
- **只读白名单**(无沙箱时 `auto` 用):`ls cat head tail wc grep rg find pwd echo which file stat du tree`
@@ -289,7 +370,7 @@ TUI 中 `/tools off NAME`、`/tools on NAME` 可临时开关单个工具。
`sudo`、`rm`、解释器(`python3 -c …`)等危险前缀不会被建议成规则。
- **撤销**:每轮写文件前自动快照到 `.agent/undo/`,`/undo` 恢复上一轮改动的全部文件(新建的文件会被删除)。
- **AGENTS.md**:工作区根目录的 `AGENTS.md` 会注入 system prompt;`/init` 让 agent 自己生成一份。
- **自动压缩**:上一轮 prompt 用到模型上下文 80% 时,下一轮开始前自动总结历史(`CST_AUTO_COMPACT=0` 关闭)。
- **自动压缩**:上一轮 prompt 用到模型上下文 75% 时,下一轮开始前自动总结历史(`CST_AUTO_COMPACT=0` 关闭)。
### 扩展:Skills、插件、MCP、子 agent
@@ -369,6 +450,7 @@ Thought(推理过程)默认折叠成一行“Thought · 3.2s”,生成中
| `/provider [name]`、`/model [name]`、`/effort [level]` | 切换 provider(缺 key 时当场输入并用 `/models` 校验)、模型(列表优先显示 API 实际返回的模型,标 `(api)`)、推理强度;都会记住 |
| `/key` | 输入或更换当前 provider 的 API key(遮罩输入并保存) |
| `/protocol [name]` | 切换工具调用协议 |
| `/mode [coding\|general]` | 工作模式(通用助手 / 代码助手口径) |
| `/tools [on\|off NAME\|minimal\|default]` | 查看或开关工具 |
| `/approval [ask\|auto\|yolo]` *、`/yolo` * | 审批模式;本次开关 yolo |
| `/permissions [reset]` *、`/max-steps [N]` * | 权限规则、每轮步数 |
@@ -378,6 +460,8 @@ Thought(推理过程)默认折叠成一行“Thought · 3.2s”,生成中
| `/diff [path]` * | `git status` + `git diff` |
| `/compact` | 手动压缩上下文 |
| `/export [file]` *、`/copy` * | 导出 Markdown 记录;复制上一条回复(OSC 52,需终端支持) |
| `/attach [path\|paste\|list\|clear]` | 给下一条消息带附件:图片、PDF(自动取文本层)、文本文件;`paste` 优先取剪贴板图片,没有就把剪贴板文本放进输入框 |
| `/web` * | 后台起一个 WebUI 进程(同 workspace、新会话,日志写到 `.agent/webui.log`) |
| `/init` | 让 agent 写 `AGENTS.md` |
| `/details` *、`/thinking [on\|off]`、`/keys` * | 展开全部 tool 输出;Thought 默认展开/折叠;快捷键列表 |
| `/skills [reload\|list]`、`/skill <name> [任务]` | 浏览 skills;带着某个 skill 开始一轮 |
+12 -4
View File
@@ -2,17 +2,25 @@
# ./scripts/build_third_party_mingw.sh curl
# cmake -B build-win -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
# -DCST_BUILD_TUI=OFF
# $MINGW_HOST picks the tool prefix (default x86_64-w64-mingw32; Debian's UCRT
# cross gcc is x86_64-w64-mingw32ucrt).
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR x86_64)
set(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc)
set(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++)
set(CMAKE_RC_COMPILER x86_64-w64-mingw32-windres)
if(DEFINED ENV{MINGW_HOST} AND NOT "$ENV{MINGW_HOST}" STREQUAL "")
set(_MINGW_HOST "$ENV{MINGW_HOST}")
else()
set(_MINGW_HOST x86_64-w64-mingw32)
endif()
set(CMAKE_C_COMPILER ${_MINGW_HOST}-gcc)
set(CMAKE_CXX_COMPILER ${_MINGW_HOST}-g++)
set(CMAKE_RC_COMPILER ${_MINGW_HOST}-windres)
get_filename_component(_AGENT_ROOT "${CMAKE_CURRENT_LIST_DIR}/.." ABSOLUTE)
set(_MINGW_PREFIX "${_AGENT_ROOT}/third_party/prefix-mingw")
list(APPEND CMAKE_PREFIX_PATH "${_MINGW_PREFIX}")
set(CMAKE_FIND_ROOT_PATH "/usr/x86_64-w64-mingw32" "${_MINGW_PREFIX}")
set(CMAKE_FIND_ROOT_PATH "/usr/${_MINGW_HOST}" "${_MINGW_PREFIX}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE BOTH)
+199 -73
View File
@@ -1,82 +1,208 @@
# CST-Agent on Windows
## Release layout
原生 Windows 版:一个静态链接的 `cst-agent.exe`(UCRT,内嵌 UTF-8 清单)加一个
`busybox.exe`(busybox-w32 Unicode 版)作为 shell,不依赖其他 DLL,也不需要管理员权限。
```
cst-agent.exe
busybox.exe # built from busybox-w32
libwinpthread-1.dll # MinGW runtime (required)
LICENSE.busybox
README-windows.md
## 安装
### 安装包
从 [Release](https://git.whcself.xyz/WHC/CST-agent/releases) 下载
`cst-agent-VERSION-windows-x86_64-setup.exe` 运行即可:按当前用户安装到
`%LOCALAPPDATA%\Programs\cst-agent`(不需要管理员),把这个目录加进**用户** PATH,
开始菜单里放一个“CST-Agent WebUI”(在用户目录启动 `cst-agent --web --web-open`)。
系统低于 Windows 10 1903 时安装包会直接拒绝。
卸载:设置 → 应用 → CST-Agent,或运行安装目录里的 `uninstall.exe`;会去掉 PATH 里的这一项。
API key 和设置在 `%APPDATA%\cst-agent`,卸载时保留。
安装包装的文件和 zip 一样,所以之后也能用 `cst-agent-update` 升级,或者重新运行新版安装包覆盖。
### PowerShell 脚本
在 PowerShell 里(Windows PowerShell 5.1 或 PowerShell 7 都行):
```powershell
irm https://git.whcself.xyz/WHC/CST-agent/raw/branch/main/scripts/install_windows.ps1 | iex
```
Prebuilt tree from Linux cross-compile: `agent/dist/windows-x86_64/`.
## Build (MinGW-w64 cross from Linux)
```bash
cd agent
# 1) third-party: curl (Schannel) + PDCurses + busybox-w32
./scripts/build_third_party_mingw.sh all
# sources:
# https://github.com/rmyorston/busybox-w32.git
# https://github.com/curl/curl.git
# https://github.com/wmcbrine/PDCurses.git
# Unix busybox (optional, Linux/Termux bundle):
# ./scripts/build_third_party_mingw.sh busybox-unix
# https://github.com/mirror/busybox.git
# 2) headless
cmake -B build-win -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
-DCST_BUILD_TUI=OFF -DCMAKE_PREFIX_PATH=$PWD/third_party/prefix-mingw
cmake --build build-win
# 3) TUI (PDCurses)
cmake -B build-win-tui -DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
-DCST_BUILD_TUI=ON -DCMAKE_PREFIX_PATH=$PWD/third_party/prefix-mingw \
-DPDCurses_ROOT=$PWD/third_party/prefix-mingw
cmake --build build-win-tui
```
Copy `build-win-tui/cst-agent.exe`, `third_party/busybox/busybox.exe`, and
`libwinpthread-1.dll` into one folder.
## Build natively on Windows (MSYS2)
The CMake build also works directly on Windows with MSYS2's mingw-w64
toolchain -- no separate Linux host needed:
```bash
# in an MSYS2 MINGW64 shell
pacman -S --needed --noconfirm \
mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake mingw-w64-x86_64-make \
mingw-w64-x86_64-curl mingw-w64-x86_64-pdcurses
cd agent
cmake -B build-win -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release
cmake --build build-win --target cst-agent -j"$(nproc)"
```
`CST_BUILD_TUI=ON` (the default) picks up MSYS2's PDCurses automatically,
including its `include/pdcurses/` layout; set `-DPDCurses_ROOT=...` only for
a custom PDCurses.
Two differences from the cross-build release layout above:
- **Extra DLLs.** MSYS2's curl package is the shared OpenSSL build, so the
executable needs its whole dependency chain beside it. Collect it with
`ldd build-win/cst-agent.exe | grep mingw64`.
- **Only build the `cst-agent` target.** Some sources under `tests/` are
POSIX-only (two-argument `mkdir`, `setenv`, `arpa/inet.h`, `symlink`) and
do not compile with MinGW. The main program is unaffected.
## Run
脚本从 Gitea Release 下载 `cst-agent-VERSION-windows-x86_64.zip`,校验 `.sha256`,
装到 `%LOCALAPPDATA%\Programs\cst-agent`,并把这个目录加进**用户** PATH。装完新开一个终端:
```bat
set CST_API_KEY=...
cst-agent.exe --headless --workspace . --prompt "echo hi"
cst-agent.exe --workspace .
cd C:\code\my-project
cst-agent
```
Shell: bundled `busybox.exe sh -c ...` unless `CST_SHELL` is set.
Credentials: `%APPDATA%\cst-agent\credentials` (or console prompt).
首次启动会让你选 provider 并输入 API key(遮罩输入,按 provider 保存)。
以后升级:
```bat
cst-agent-update :: 最新版
cst-agent-update -Version 0.6.0 :: 指定版本
```
正在运行的 `cst-agent.exe` 不会挡住升级:旧文件会先改名为 `.old`,下次升级时删掉。
安装脚本的参数(下载后用 `powershell -ExecutionPolicy Bypass -File install_windows.ps1 ...` 运行):
| 参数 | 环境变量 | 作用 |
|---|---|---|
| `-Version 0.6.0` | `CST_VERSION` | 指定版本,默认最新 |
| `-Prefix D:\tools\cst` | `CST_PREFIX` | 安装目录 |
| `-Zip FILE` | | 用本地 zip,不联网 |
| `-NoPath` | | 不改用户 PATH |
| | `CST_GITEA` / `CST_REPO` | 换 Gitea 服务器或仓库(默认 `WHC/CST-agent`) |
不想用脚本的话,把 zip 解压到任意目录直接运行 `cst-agent.exe` 也可以。
## 系统要求
- Windows 10 1903(build 18362)及以上、Windows 11、Windows Server 2022 / 2025。
更老的系统不认清单里的 `activeCodePage=UTF-8`,安装脚本会直接拒绝。
- x86_64。Windows 10 1903 起系统自带 UCRT,无需另装运行库。
- 终端:Windows Terminal 或传统控制台(conhost)都行。
## 发行包内容
```
windows-x86_64/
cst-agent.exe # 静态链接:curl(Schannel)、ncurses 6.6、winpthreads
busybox.exe # busybox-w32 Unicode 版,run_shell 用它的 sh
LICENSE.busybox
README.md # 本文件
VERSION
cst-agent-update.cmd # 调用旁边的 install_windows.ps1 升级本目录
install_windows.ps1
```
## 文件位置
| 内容 | 位置 |
|---|---|
| API key(每个 provider 一个) | `%APPDATA%\cst-agent\credentials`,只有当前用户可读,并带“禁止向上读”标签(沙箱内读不到) |
| 上次的 provider、模型、推理强度、确认模式 | `%APPDATA%\cst-agent\settings.json` |
| 沙箱打过标签的目录 | `%APPDATA%\cst-agent\sandbox-labels` |
| 会话、撤销快照、导出 | `<工作区>\.agent\` |
设置了绝对路径的 `XDG_CONFIG_HOME` 时用 `%XDG_CONFIG_HOME%\cst-agent`。
## 编码:全部 UTF-8
- exe 内嵌 `activeCodePage=UTF-8` 清单:进程的 ANSI 代码页就是 65001,命令行参数、
环境变量、文件名都以 UTF-8 进出;内部所有路径在系统调用边界转成 UTF-16(`*W` API)。
- 控制台输入输出代码页在启动时设为 65001,退出时恢复。
- 文件按字节读写,不做换行转换。`read_file` 给模型看的是去掉 `\r` 的文本;`edit_file`、
`apply_patch`、`write_file` 会保持原文件的 CRLF 换行和 UTF-8 BOM。
- 子进程输出:先按 UTF-8 解码;不是合法 UTF-8 时依次尝试控制台代码页、系统 OEM / ANSI
代码页(比如中文系统的 936),转成 UTF-8 再交给模型。
- 中文用户名、中文工作区路径(`C:\Users\测试\项目`)都已在中英文系统上测过。
## 终端界面(TUI)
用的是打过补丁的 ncurses 6.6 Windows 控制台驱动(补丁在
`scripts/patches/ncurses-6.6-win32-console.patch`):
- 中文按两列宽显示,光标位置正确;粘贴多行不会直接发送。
- 鼠标:点击、拖选复制(写入 Windows 剪贴板)、滚轮滚动;`/mouse` 关闭。
- `/copy` 和拖选复制走 Win32 剪贴板 API,不依赖终端的 OSC 52。
- 控制台是点阵字体时(显示不了中文,会变成 `?`),启动时自动换成同字号的 Consolas
(中文系统换成新宋体),退出时换回去。
- 传统控制台不做字形回退:当前字体缺 logo 用的方块字符时(比如中文系统默认的新宋体),
首页改用文字 logo。
## Shell 沙箱:Low 完整性级别
`auto` 确认模式下,`run_shell` 在 Low 完整性级别的受限令牌里运行(界面显示 `auto · lowil`):
- 子进程的令牌去掉了管理员组和特权,完整性级别为 Low(S-1-16-4096),放进 Job Object,
超时或按 Esc 时整个进程树一起结束。
- Low 进程只能写标为 Low 的位置。第一次在某个工作区跑沙箱命令时,CST-Agent 给工作区
加上可继承的 Low 标签(`S:(ML;OICI;NW;;;LW)`),并在输出里提示一次;
记录写进 `sandbox-labels`。沙箱的临时目录是 `%TEMP%\cst-agent-sandbox`(同样标为 Low),
子进程的 `TEMP` / `TMP` 指向这里。
- 不会给盘符根目录、用户目录本身、`%APPDATA%`、`%LOCALAPPDATA%`、`%TEMP%` 本身,
以及 Windows、Program Files、ProgramData 下面的目录打标签:在这些地方启动时,命令不会运行,
界面会问是否在沙箱外重跑。请在项目目录里启动。
- `/sandbox` 列出打过标签的目录;`/sandbox reset-labels` 恢复它们原来的标签。
- 沙箱里可以读、执行用户能访问的文件(凭据文件除外),**不限制联网**:`curl`、`git pull`、
`npm install` 等联网命令在 `auto` 模式下也直接在沙箱里运行。
- 命令像是被拦了(`Access is denied`、`拒绝访问`、`Permission denied` 等)时,会问
“Rerun outside the sandbox?”:`y` 在沙箱外重跑一次,`p` 保存命令前缀规则并重跑,`n` 保留结果。
- MSYS2 / Cygwin 的程序(包括 Git for Windows 自带的 `usr\bin` 工具)在 Low 级别下无法初始化
(`0xc0000142`),同样会触发上面的重跑询问。Git for Windows 的 `git.exe` 本身不受影响。
- `CST_SANDBOX=off` 关闭沙箱(`auto` 退回只读白名单规则);`CST_SANDBOX=lowil` 只用 Low IL。
它防的是误操作写坏工作区以外的文件,不是对抗恶意程序的安全边界。
## 从源码构建
两种方式产物一样:`dist/windows-x86_64/` 和 `dist/cst-agent-VERSION-windows-x86_64.zip`;
装了 `makensis`(NSIS)时还有安装包 `dist/cst-agent-VERSION-windows-x86_64-setup.exe`
(脚本在 `scripts/windows/cst-agent.nsi`)。
第三方依赖(curl、ncurses、busybox-w32)都按固定版本从源码编译、静态链接。
### 在 Linux 上交叉编译
```bash
git clone https://git.whcself.xyz/WHC/CST-agent.git
cd CST-agent
scripts/bootstrap_windows.sh
```
NixOS / 有 Nix 时自动进 `shell-win.nix`(`pkgsCross.ucrt64`,带 NSIS);否则需要 UCRT 版的
mingw-w64 gcc(带 winpthreads)、cmake、ninja、git、perl。工具前缀由 `MINGW_HOST` 指定,默认
`x86_64-w64-mingw32`;Debian trixie 起的 `gcc-mingw-w64-ucrt64` 是 `x86_64-w64-mingw32ucrt`:
```bash
sudo apt install gcc-mingw-w64-ucrt64 binutils-mingw-w64-ucrt64 nsis cmake ninja-build perl
MINGW_HOST=x86_64-w64-mingw32ucrt scripts/bootstrap_windows.sh
```
`CST_WIN_CFLAGS` 给 `cst-agent.exe` 追加编译选项(CI 用 `-Wall -Wextra -Werror`)。
### 在 Windows 上用 MSYS2 原生编译
装好 [MSYS2](https://www.msys2.org/) 后,打开 **MSYS2 UCRT64** 终端(不是 MINGW64:
MINGW64 链接 msvcrt,UTF-8 清单不起作用):
```bash
pacman -S --needed git
git clone https://git.whcself.xyz/WHC/CST-agent.git
cd CST-agent
scripts/bootstrap_msys2.sh
```
脚本会用 pacman 装 UCRT64 的 gcc / cmake / ninja,编译依赖和 `cst-agent.exe`,跑 ctest
(`CST_SKIP_TESTS=1` 跳过),再打包。
### 测试
`scripts/win_test.sh` 在本机 QEMU 里的 Windows Server 虚拟机(英文 / 中文系统,管理员 /
普通用户 / 中文用户名)上交叉编译后跑全部测试;虚拟机的创建和控制见 `scripts/winvm/`。
### 发布
CI:推 `main` 或 `v*` tag 且 `windows` 任务成功后,自动把 zip、安装包和各自的 `.sha256` 发到
Gitea Release `vVERSION`(版本来自 `include/cst_version.h`;同名资源替换)。仓库 Secrets
可设 `RELEASE_TOKEN`(`write:repository`),没有则用 Actions 自带 token。
本地:
```bash
scripts/bootstrap_windows.sh
GITEA_TOKEN=... scripts/release.sh dist/cst-agent-*-windows-x86_64.zip dist/cst-agent-*-windows-x86_64-setup.exe
```
创建(或复用)`vVERSION` Release,上传给出的文件和旁边的 `.sha256`。token 只从环境变量读取,经 stdin
交给 curl,不会出现在命令行参数、磁盘或输出里。发版前先 bump `cst_version.h`;推
`vX.Y.Z` tag 时必须与头文件一致。
## 已知限制
- 英文等非中文系统的传统控制台默认用 Consolas,它没有中文字形,中文会显示成方框(列宽仍然正确)。
用 Windows Terminal,或在控制台“属性 → 字体”里选新宋体等中文字体即可。
- 传统控制台里输入表情符号(BMP 以外的字符)可能显示不对;中文输入没问题。
- 通过 SSH / ConPTY 远程运行时,鼠标事件不会转发过来,键盘操作不受影响。
- 很大的工作区第一次打标签要等一会儿(标签要继承到每个文件)。
- 只有 x86_64 发行包。
+45 -23
View File
@@ -21,11 +21,11 @@ The Makefile picks `ncursesw`, then `ncurses`, through pkg-config. On macOS it
adds Homebrew's keg-only `ncurses`/`curl` to `PKG_CONFIG_PATH`; without
Homebrew it falls back to the system `-lncurses` (wide-character capable).
Verified so far: NixOS x86_64 with gcc and clang (`-Werror`, `make check`,
`make asan`, ctest), and every source compiled `-fsyntax-only` with
`x86_64-unknown-linux-musl-gcc` without warnings. Full musl, macOS and WSL
builds run in CI (`.gitea/workflows/ci.yml`; the macOS job needs a runner and
the `HAVE_MACOS_RUNNER` variable) or by hand with the checklist below.
Verified so far: NixOS x86_64 with gcc, clang and TinyCC (`-Werror` except
tcc, `make check`, `make asan`, ctest). CI (`.gitea/workflows/ci.yml`) checks
with TinyCC on Debian and builds the static musl releases for x86_64, x86 and
aarch64 with gcc `-Werror` (`make check` on both x86 targets, a qemu smoke
test on aarch64). macOS and WSL are checked by hand with the list below.
## Files
@@ -42,18 +42,20 @@ the `HAVE_MACOS_RUNNER` variable) or by hand with the checklist below.
In the default `auto` approval mode, `run_shell` runs inside an OS sandbox
when one is available. Commands may read and execute anything but may only
write beneath the workspace, the temp directory, `/dev/null` and `/dev/tty`.
The network is not restricted.
| Platform | Sandbox | Notes |
|---|---|---|
| Linux 5.13+ | Landlock (raw syscalls, no extra library) | ABI ≥ 4 (6.7+) also blocks TCP connect/bind |
| Linux 5.5+ without Landlock (Android / Termux, WSL, older distros), x86_64 / aarch64 | seccomp user-notify | refuses IPv4/IPv6 sockets; see below |
| macOS | `sandbox_init` profile | also denies network |
| Older kernels, Windows | none | falls back to the read-only rules |
| Linux 5.13+ | Landlock (raw syscalls, no extra library) | |
| Linux 5.5+ without Landlock (Android / Termux, WSL, older distros), x86_64 / aarch64 | seccomp user-notify | see below |
| macOS | `sandbox_init` profile | |
| Windows 10 1903+ | Low integrity restricted token + Job Object | writes need a Low label (given to the workspace once); see `README-windows.md` |
| Older kernels | none | falls back to the read-only rules |
The status bar and sidebar show `auto · landlock`, `auto · seccomp`,
`auto · seatbelt` or `auto · rules`. `CST_SANDBOX=off` disables the
sandbox; `CST_SANDBOX=landlock` or `CST_SANDBOX=seccomp` allows only that
kind. When a sandboxed command looks blocked, the TUI asks whether to rerun
`auto · seatbelt`, `auto · lowil` or `auto · rules`. `CST_SANDBOX=off`
disables the sandbox; `CST_SANDBOX=landlock`, `seccomp` or `lowil` allows
only that kind. When a sandboxed command looks blocked, the TUI asks whether to rerun
it outside the sandbox. The policy is built in the parent; the forked child
only makes a few async-signal-safe syscalls before `exec`, so the agent
itself is never confined.
@@ -65,16 +67,19 @@ every create / remove / rename / link / chmod / chown / utimes / xattr call
stop until the supervisor has read the path from the child's memory,
resolved it (cwd, dirfd, `..`, symlinks, `/proc/self`, `/dev/stdout`) and
answered: continue when it lands in the allowed places, `EACCES` otherwise.
Reads, exec and writes to already-open files are not checked. Background
jobs that outlive their shell are no longer the agent's descendants: with
Yama `ptrace_scope` ≥ 1 (common on desktop distros, absent on Android) the
supervisor cannot read their memory and their checked writes fail with
`EACCES`; once the run has finished they get `ENOSYS`. This keeps
mistakes out of the rest of the disk but is not a security boundary: the
kernel reads the path again after the check, so a hostile multi-threaded
program can swap it in between. The availability check runs in a
throwaway child, so a platform seccomp policy that kills the caller cannot
take the agent down.
Reads, exec and writes to already-open files are not checked. The probe also
checks that the parent can peek a direct child's memory (`process_vm_readv`
or `/proc/<pid>/mem`); with Yama `ptrace_scope=2` and no `CAP_SYS_PTRACE`
that fails, so seccomp is not selected (forced `CST_SANDBOX=seccomp` then
falls back to unavailable). Background jobs that outlive their shell are no
longer the agent's descendants: with Yama `ptrace_scope` ≥ 1 (common on
desktop distros, absent on Android) the supervisor cannot read their memory
and their checked writes fail with `EACCES`; once the run has finished they
get `ENOSYS`. This keeps mistakes out of the rest of the disk but is not a
security boundary: the kernel reads the path again after the check, so a
hostile multi-threaded program can swap it in between. The availability
check runs in a throwaway child, so a platform seccomp policy that kills the
caller cannot take the agent down.
## Terminal notes
@@ -140,7 +145,7 @@ TUI:
- [ ] Tab toggles Build/Plan, Shift+Tab cycles ask/auto/yolo, Ctrl+P opens the palette
- [ ] `/model` lists the provider's `/models` (marked `(api)`); a wrong key is re-asked
- [ ] `auto` mode: `echo x > ~/outside` in run_shell asks "Rerun outside the
sandbox?" (Linux/Termux/macOS); with `CST_SANDBOX=off`, `ls` runs and `touch x` asks
sandbox?" (Linux/Termux/macOS/Windows); with `CST_SANDBOX=off`, `ls` runs and `touch x` asks
- [ ] Resize the terminal during streaming: layout recovers
- [ ] `NO_COLOR=1` and `CST_ASCII=1` render readable screens
@@ -149,6 +154,23 @@ WSL specifics:
- [ ] Workspace under `/mnt/c/...` works (slower IO is expected); `~/project` is recommended
- [ ] Windows Terminal: mouse wheel scroll and OSC 52 `/copy`
Windows specifics (`scripts/win_test.sh` runs the tests on English and
Chinese VMs as Administrator, a standard user and a Chinese user name):
- [ ] `install_windows.ps1 -Zip dist\cst-agent-*.zip` installs, adds the user
PATH; `cst-agent-update -Zip ...` works while `cst-agent.exe` runs
- [ ] conhost with a raster font: Chinese renders (font switched), and the
old font is back after exit; Windows Terminal renders the same
- [ ] Chinese input via IME, paste, mouse click / drag-copy / wheel
- [ ] Workspace under `C:\Users\<中文名>\项目`: read/write/edit/apply_patch
keep CRLF and BOM; `run_shell` output in Chinese is readable on a
936 system
- [ ] `auto · lowil`: first command reports the label; `echo x > C:\outside`
asks to rerun; `curl ...` runs without asking; `/sandbox reset-labels`
restores the label
- [ ] Esc during `sleep 30 & sleep 30` returns at once and leaves no
`busybox.exe` behind
macOS specifics:
- [ ] Builds with Homebrew ncurses and with the system ncurses (`brew unlink ncurses`)
+2 -1
View File
@@ -13,6 +13,7 @@ CST-Agent 使用 **OpenAI Chat Completions** 协议(`POST {base}/chat/completi
| `minimax`(默认) | `https://api.minimax.cn/v1` | `MiniMax-M2.7` | `MINIMAX_API_KEY` |
| `deepseek` | `https://api.deepseek.com/v1` | `deepseek-flash`(另有 `deepseek-v4-pro`) | `DEEPSEEK_API_KEY` |
| `openai` | `https://api.openai.com/v1` | `gpt-4.1-mini` | `OPENAI_API_KEY` |
| `teamorouter` | `https://api.teamorouter.com/v1` | 登录后自动取 `/models` 第一个(`/model` 可换) | `TEAMOROUTER_API_KEY` |
显式给出的 `--api-base` / `--model` / 下表中的环境变量优先于预设。
@@ -72,7 +73,7 @@ export OPENAI_MODEL='your-model'
| `CST_MAX_STEPS` | 每个用户回合最多调用模型几次(默认 30) |
| `CST_TOOL_OUT_CAP` | 单个工具输出发给模型的字节上限(默认 16384) |
| `CST_CTX_WINDOW` | 发给模型的最近非 system 消息条数(默认 80) |
| `CST_SHELL_TIMEOUT` | `run_shell` 超时秒数(默认 120) |
| `CST_SHELL_TIMEOUT` | `run_shell` 超时秒数(默认不限,Ctrl+C / 停止按钮仍可随时中断) |
| `CST_METRICS=0` / `CST_METRICS_PATH` | 关闭 / 指定计时埋点 JSONL |
| `CST_RECORD_SSE` / `CST_REPLAY_SSE` | 录制 / 回放原始 SSE 流目录;`CST_REPLAY_CHUNK` 控制回放分块字节数 |
| `CST_MODELS` | `/model` 可选列表,`:` 或 `,` 分隔 |
+292
View File
@@ -0,0 +1,292 @@
# WebUI(内置 HTTP + SSE 前端)
`cst-agent --web` 让同一个进程同时提供命令行以外的前端:浏览器(或任何 HTTP 客户端)
连上后能看到流式回答、工具卡片、审批弹窗和任务清单。它是 TUI 之外的第二个前端,
**引擎没有为它改一行**——两边用的都是 `cst_agent_hooks_t` 这组回调。
这份文档同时是**引擎 API v1 的规范**:以后专门客户端(Tauri / Qt / 移动端)直接照它实现,
不需要读 C 代码。
## 快速开始
```bash
# 默认只监听 127.0.0.1:8787
cst-agent --web
# 指定端口;0 = 让系统分配(启动日志里会打印实际 URL)
cst-agent --web --web-port 8787
# 让手机/另一台机器访问(必须给 token,否则拒绝启动在非 loopback 上)
cst-agent --web --web-host 0.0.0.0 --web-token 我自己定的口令
# 开发前端:直接从目录读 web/,改完刷新页面即可,不用重新编译
cst-agent --web --web-dev ./web
# 启动后自动打开浏览器
cst-agent --web --web-open
# 启动即跑一轮(不点页面也能用)
cst-agent --web -p "看看这个仓库有什么问题"
```
启动日志:
```
webui: http://127.0.0.1:8787/
webui: token 3f9a… (append #token=3f9a… to the URL)
```
带 token 时把 `#token=…` 贴到浏览器地址栏后面即可(`#` 之后不会发给服务器,
页面会用 `?token=` 再请求一次)。
环境变量:`CST_WEB_PORT`、`CST_WEB_HOST`、`CST_WEB_TOKEN`、`CST_WEB_ROOT`。
退出:终端里 Ctrl+C,或 `POST /api/quit`。
## 安全模型
| 场景 | 行为 |
|---|---|
| 默认 `127.0.0.1` | 不需要 token;`Host` 必须是 loopback(防 DNS rebinding);带 `Origin` 的请求必须来自本页面自己(`http://<Host>`),否则 401 —— 防止任意网页跨站 POST 驱动 agent |
| 静态资源 | 页面本身与 `app.js`/`style.css` 不需要 token;`/api/attach/<rel>` 图片仍要(页面会把 token 写进 `SameSite=Strict` cookie) |
| 请求解析 | 鉴权与大小检查在读请求体**之前**完成;`Content-Length` 必须是纯数字且只有一个值;请求头整体 15 s 内收完 |
| `--web-host 0.0.0.0` | 没给 `--web-token` 时自动生成随机 token 并打印;所有 `/api/*` 都要 token |
| token 传递 | `Authorization: Bearer <t>`、`Cookie: cst_token=<t>` 或 `?token=<t>`(`EventSource` 不能设 header,所以查询参数是必需的一条路) |
| 资源 | 只有白名单路由;路径里的 `..`/编码 NUL 一律 400;静态资源来自二进制内嵌(或 `--web-dev DIR`) |
| 输出渲染 | 模型与工具输出都当不可信内容:Markdown 由服务端渲染,并经过标签/属性白名单(`javascript:` 链接、站外图片、`<script>` 全部丢弃) |
| 请求体 | 默认 1 MiB,图片上传 32 MiB;超限 413 |
**不要**把 WebUI 直接暴露到公网:没有 TLS,token 会以明文出现在网络里。需要远程访问时
放在反向代理(https)后面,并确认 `X-Accel-Buffering: no` 之类的缓冲设置不会掐断 SSE。
## 引擎 API(协议 `cst-webui/1`)
所有响应都是 JSON(`Content-Type: application/json`)。`GET /api/state` 的
`protocol` 字段给出版本号,客户端应当先看它再决定支持哪些端点。
### 事件流
```
GET /api/events?since=<seq> → text/event-stream (chunked)
```
* 每条事件都有 `id: <seq>`(单调递增)与 `data: {"type":…,"seq":…, …}`。
* 断线重连带上 `since=<最后收到的 seq>`(浏览器用 `Last-Event-ID` 也行),
服务端保留最近 2000 条,超出后从可用的最老一条开始;比当前 seq 还大(服务端重启过)
就当作新连接。
* 新连接(不带 `since`)只收之后的事件;但若有尚未回答的 `permission`/`ask_user`,
会从那条事件开始补发,刷新页面不会丢掉正在等的问题。
* 心跳是 SSE 注释 `: ping`(每秒一次,客户端忽略)。
* `state` 事件在状态变化时整份重发,客户端可以直接替换本地状态。
事件类型:
| type | 字段 | 说明 |
|---|---|---|
| `state` | 见下表 | 会话/模型/审批/todo/用量快照 |
| `status` | `message` | 引擎自己的一句话状态("calling model (step 2)…") |
| `assistant_html` | `text` | **已渲染且已消毒的 HTML 片段**,按顺序追加 |
| `assistant_delta` | `text` | 原始文本增量(逐字节真相,做复制/导出时用) |
| `think_delta` | `text` | 推理过程增量(Thought) |
| `tool` | `id`,`name`,`args`,`phase`,`detail` | `phase` = `open`/`ready`/`run`/`progress`/`done`;`progress` 是子代理的最近几步(`detail` 形如 `explore · grep foo · read_file x`),可以出现多次 |
| `tool_result` | `id`,`name`,`out`,`exit`,`bytes`,`truncated` | 完整输出被截到 32 KiB,`truncated=true` |
| `todo` | `text` | `todo_write` 替换后的清单(一行一项) |
| `permission` | `id`,`tool`,`detail`,`preview`,`sandbox`,`timeout` | 需要回答(见下);`timeout` 秒后按拒绝处理 |
| `permission_done` | `id`,`answer` | 问题已结束(超时/取消也会发) |
| `ask_user` | `id`,`question`,`options[]` | 需要回答 |
| `session` | `id` | 切换/新建了会话,客户端应重新拉 `/api/session` |
| `turn_end` | `rc`,`error` | 本轮结束(`rc=0` 正常,非 0 见 `error`) |
`state` 字段:`workspace` `session` `provider` `model` `effort` `approval`
(`ask|auto|yolo`) `plan_mode` `protocol_tools` `status_busy` `todo`
`context_tokens` `undo_available` `pending_images[]` `usage{requests,prompt_tokens,
completion_tokens,cached_tokens,cache_miss_tokens,cost,priced,currency}`
`turn_usage{…}`(上一轮,字段同 `usage`)`api_base` `key_set` `key_from`
(`saved` / 环境变量名 / `session` / 空,**从不含 key 本身**)
`model_info{label,context,vision,reasoning,source,price?}` `effort_applies`
`sees_images`(1 / 0 / -1 未知)`max_steps` `tool_profile` `perm_timeout`
`show_thinking`(思考过程默认展开,存在 `settings.json`,和 TUI 的 `/thinking` 是同一个开关)。
`cached_tokens` / `cache_miss_tokens` 来自上游的 prompt cache 统计,前端用它算命中率。
`think_delta` / `assistant_delta` 的 `text` 总是完整的 UTF-8:上游在多字节字符中间
断开时,残缺的尾巴留到下一块再发。
### 请求
| 方法 | 路径 | 说明 |
|---|---|---|
| GET | `/api/state` | 上面那份快照 |
| GET | `/api/session?from=N` | 会话消息(`{messages:[…]}`,形状与 `.agent/sessions/*.jsonl` 一致,另加的字段见下) |
| GET | `/api/sessions` | `{current, sessions:[{id,title,messages,mtime}]}` |
| POST | `/api/sessions/new` | `{}` → `{id}` |
| POST | `/api/sessions/switch` | `{"id":"…"}` |
| POST | `/api/sessions/rename` | `{"id":"…"?,"title":"…"}`(不带 `id` = 当前会话);标题按字符截到 120 字节,空串 = 恢复默认(第一句提问);没这个会话 → 404 |
| POST | `/api/sessions/delete` | `{"id":"…"}` → `{ok,current}`;删的是当前会话时先开一个新会话(并发 `session` 事件);没这个会话 → 404 |
| POST | `/api/retry` | `{}` = 重新生成最后一轮;`{"text":"…"}` = 把最后一句提问换成新文字再跑(编辑重发)。先把最后一轮从会话里退掉,再开新一轮 → 202 `{ok,text}`;没有可重来的轮次 / 轮次在跑 → 409;空串或非字符串 → 400 |
| GET | `/api/export` | 当前会话的 Markdown(`text/markdown`,与 TUI `/export` 同一格式) |
| GET | `/api/diff[?path=p]` | 工作区的真实 diff:`{git,path,files:[{status,path}],diff,truncated}`。`files` 来自 `git status --short`(含未跟踪),`diff` 是 `git diff`(未暂存部分,限 512 KiB);不是 git 仓库 / 没装 git → `git:false`;10 s 超时 |
| GET | `/api/settings` | 运行设置:`protocol`,`protocols[]`,`max_steps`,`tool_profile`,`profiles[]`,`confirm{shell,write,net}`,`approval`,`tools[{name,perm,on,desc,…}]`,`sandbox{kind,active,labels?}` |
| POST | `/api/settings` | `{protocol?,max_steps?,tool_profile?,tools?:{名字:bool},confirm?:{shell?,write?,net?}}`;整体校验后才生效(任何一项不对 → 400,什么都不改),成功后发 `state` |
| GET | `/api/permissions` | 已保存的"总是允许"规则:`{text,tools[],shell_prefixes[]}`(来自 `<workspace>/.agent/permissions.json`) |
| DELETE | `/api/permissions` | 清空这些规则 |
| POST | `/api/init` | `{}` → 202 `{ok,text}`:开一轮,让 agent 分析仓库写 `AGENTS.md`(与 TUI `/init` 同一段提示词) |
| POST | `/api/skill` | `{"name":"…","task":"…"?}` → 202 `{ok,text}`:把 skill 正文内联进这条消息再开一轮(同 TUI `/skill`);没有这个 skill → 404;缺 `name` / 超过 63 字节 / `task` 不是字符串 → 400 |
| GET | `/api/extensions` | `{enabled,trusted,report,task_tool,mcp:[{name,source,command,state,tools,error}],plugins:[{name,source,desc,tools,mcp,skills,error}],skills:[{name,source,desc}],roles:[{name,source,desc,model,read_only}],runs:[{file,role,desc,status,steps,tools,ms}]}`;`state` = `starting/ready/failed/exited/off`,`runs` 是当前会话的子代理运行 |
| POST | `/api/extensions/reload` | 停掉 MCP 服务器和插件,重新读取它们和 skills → `{ok,trusted,report}` |
| POST | `/api/extensions/trust` | `{"on":true\|false}`:信任(或取消)当前工作区——决定它的 `.agent/plugins`、`.agent/mcp.json` 是否加载;写进 `settings.json` 后重新加载,回复同上 |
| POST | `/api/workspace` | `{"path":"/abs/dir"}` —— 换工作目录(权限规则/埋点/AGENTS.md 跟着换,扩展按新目录重新加载),并**在新目录里开一个新会话**(会话是按目录存的);返回 `{id}`,坏路径 → 400 |
| POST | `/api/turn` | `{"text":"…"}` → 202 `{ok:true}`;已有轮次在跑时 **409** |
| POST | `/api/cancel` | `{}` —— 取消当前轮(不会杀进程);挂起的权限问题按拒绝、提问按忽略处理 |
| POST | `/api/permission` | `{"id":N,"answer":0\|1\|2\|3}` → deny / once / always-tool / always-prefix |
| POST | `/api/ask_user` | `{"id":N,"text":"…"}` 或 `{"id":N,"dismiss":true}` |
| POST | `/api/config` | `{model?,effort?,approval?,plan_mode?,show_thinking?}`;先整体校验再生效,model/effort/approval(`yolo` 除外)写进 `settings.json`;当前模型不吃 effort 时回 `warning`;带 `provider` → 400(改用 `/api/provider`) |
| GET | `/api/providers` | `{current, providers:[{name,api_base,model,has_key,key_from,key_env,source,current}]}` |
| GET | `/api/models[?refresh=1]` | 当前提供商的模型:`{provider,current,api_base,key_set,api_count,models:[{id,api,label,context,vision,reasoning,source,price?}]}`;先用缓存,过期/缺失/`refresh=1` 时向 `/v1/models` 拉;key 被拒 → 401(仍带内置模型和 `auth:true`),其它失败放在 `api_error` |
| POST | `/api/provider` | `{"name":"…","key":"…"?}`:切换提供商。缺 key → 400 `{need_key:true}`;key 被拒 → 401;没这个提供商 → 404;成功 → `{ok,provider,model,warning?}`,key 验证后存进 `credentials`(0600) |
| POST | `/api/key` | `{"key":"…"}`:换当前提供商的 key,验证与回复同上 |
| POST | `/api/undo` | 回滚最近一轮的文件改动,回 200 + 报告文本 |
| POST | `/api/compact` | 手动压缩上下文 |
| POST | `/api/attach?name=x.png` | 请求体是图片原始字节(≤32 MiB),进入"待发送"列表 |
| DELETE | `/api/attach` | 清空待发送图片 |
| GET | `/api/attach/<rel>` | 读回已存进会话的图片(`rel` 由消息的 `attach[].rel` 给出) |
| POST | `/api/quit` | 关闭进程 |
未命中路由 → `404 {"error":"not found"}`;需要 token 而没给 → `401`。
轮次运行中(或另一个修改正在进行)时,`sessions/new`、`sessions/switch`、`sessions/delete`、
`workspace`、`undo`、`compact`、`attach`(上传/清空)、`provider`、`key`、`retry`、`diff`、
`POST /api/settings`、`DELETE /api/permissions`、`init`、`skill`、`extensions/reload`、
`extensions/trust` 以及改 `model/effort` 的 `config` 一律 **409**;
`approval`/`plan_mode`/`show_thinking` 可以中途改,`sessions/rename`、`export` 与
`GET /api/extensions` 在轮次进行中也能用。
`/api/session` 的每条消息在会话文件的字段之外还带:
| 字段 | 出现在 | 说明 |
|---|---|---|
| `html` | assistant | 已消毒的 Markdown 渲染结果(文本协议时只渲染说明文字,不含调用帧) |
| `text` | 文本协议的 assistant | 去掉调用帧后的说明文字(`content` 仍是原文) |
| `calls` | 带工具调用的 assistant | `[{id,name,args}]`:原生协议来自 `tool_calls`,文本协议从 `content` 解析 |
| `tool_result` | 工具结果 | `true`;原生协议 role 是 `tool`,文本协议 role 是 `user`(凭 `tool_call_id` 区分) |
| `out`,`exit`,`truncated` | 工具结果 | 去掉 `[tool …]` / `result … end` 外框后的输出(截到 32 KiB,按字符边界)与退出码 |
| `note` | 引擎插入的提示 | `true`;如中途切换计划模式时的 `[cst] …` 说明,不是用户说的话 |
客户端据此在刷新后重建"本轮工具调用"列表:`calls` 建行,`tool_call_id` 相同的结果填进去。
key 只出现在请求体里:回复、事件、日志都不会带它。`/api/provider` 与 `/api/key`
跟其它 `/api/*` 一样受 loopback 同源检查或 token 保护。
### 客户端接入示例(10 行)
```python
import json, urllib.request
base = "http://127.0.0.1:8787"
def api(path, body=None, token=None):
req = urllib.request.Request(base + path, method="POST" if body is not None else "GET")
if token: req.add_header("Authorization", "Bearer " + token)
data = json.dumps(body).encode() if body is not None else None
return urllib.request.urlopen(req, data).read()
api("/api/turn", {"text": "hello"}) # 之后读 /api/events 收流
```
### 空 body 的 POST
`POST /api/cancel`、`/api/undo`、`/api/compact`、`/api/quit` 不需要 body。
没有 `Content-Length` 头的 POST 按 RFC 9112 §6.3 视为**空 body**(`curl -X POST`
正是这样发的),但该连接不再复用(服务端随后关闭它):后面是否跟着未声明的
字节无法判断。
## 前端实现(web/)
没有构建步骤:`web/index.html` + `web/app.js` + `web/style.css` 是纯 ES module,
编译时被 `scripts/embed_web.sh` 塞进二进制(`.gen/web_assets.inc`,跟内置模型数据同一招),
所以 Windows 发布包依旧是单个 exe。`--web-dev DIR` 可以绕过内嵌版本直接读目录。
Markdown 在**服务端**渲染(复用 TUI 用的 sashidown 流式 API),客户端只贴 HTML;
因此不需要引入任何 JS 依赖,也顺带保证了"只发送已确认的 HTML"。
界面本身:左栏可收(顶栏按钮 / `?side=0`),右栏可收(`?panel=0`),主题 `?theme=dark|light`;
三个开关都会记进 `localStorage`,带查询参数时以参数为准。空状态只留一句提示,第一轮
对话开始后自动隐去;右栏只给三件事——任务清单、本轮文件改动行数(`+N −M`)、上下文
占用与 prompt cache 命中率。布局类在页面脚本之前就写进 `<body>`,避免首帧闪一下再收起。
对话列:正文、过程块、提示行和输入框共用同一条左边线,正文宽度上限 720px(`--content`);
左右两栏随屏宽在 `clamp()` 范围内变宽,超宽屏下中间列不会无限拉开。
过程时间线:一段**连续**的思考和工具调用合成一个过程块,折叠时只有一行
("思考 4 次 · 工具 6 次";运行中显示当前动作,比如"执行 run_shell…"),展开后是一条
细竖线串起的时间线,每次思考、每个工具调用各占一行,思考只露第一行,点开看全文。
出现可见的正文(或引擎提示、下一句提问)时过程块就此结束,后面的思考/工具另起一块——
历史重建(`/api/session`)和实时事件流用的是同一条规则,所以刷新前后看到的一样。
`/thinking on|off`(`show_thinking`)决定过程块和其中的思考默认展开还是收起,流式输出时也照此执行;
`/details` 只管工具行是否展开。
图片:提问附带的图片显示在气泡**上方**、靠右,一张保持原比例(最长 260px),多张是
152px 的圆角方块;点开是大图预览(Esc 关闭,←/→ 切换)。从本页上传/粘贴/拖入的图片在
输入框上方显示缩略图,发送后立刻出现在新提问上(本地预览,本轮结束后换成 `/api/attach/<rel>`);
从别的客户端加的待发图片仍然显示为文字条。
### 命令与快捷键
输入框以 `/` 开头时是命令,在页面里执行,不发给模型:打 `/` 弹出列表(↑↓ 选,Tab 补全,
Enter 执行),`/model`、`/effort`、`/approval`、`/provider`、`/skill`、`/thinking`、`/details`
还会补全参数。`/usr/bin 是什么` 这种第一个词里带 `/` 的照常发送;未知命令留在输入框里并提示,
开头加个空格即可当普通消息发出。
| 命令 | 作用 |
|---|---|
| `/new` `/sessions` `/rename [标题]` | 新建 / 会话列表 / 重命名 |
| `/retry` `/edit` `/stop` `/undo` `/compact` | 重新生成 / 编辑重发 / 停止 / 撤销写入 / 压缩上下文 |
| `/export` `/copy` `/diff [文件]` | 导出 Markdown / 复制最后一条回答 / 工作区 diff |
| `/model [id]` `/effort [级别]` `/provider [名字]` `/key` | 模型、推理强度、提供商、API key |
| `/approval [ask\|auto\|yolo]` `/plan` `/build` | 审批权限、计划/构建模式 |
| `/settings`(`/protocol` `/tools` `/max-steps` `/permissions`) | 运行设置 |
| `/init` `/skill 名字 [任务]` `/skills` `/extensions`(`/mcp` `/plugins` `/agents`) | 写 AGENTS.md、用 skill、扩展状态 |
| `/thinking [on\|off]` `/details [on\|off]` | 思考过程默认展开(存 `settings.json`)、工具输出全部展开(存浏览器) |
| `/theme` `/palette` `/help` | 主题、命令面板、帮助 |
快捷键:Ctrl+P 命令面板,Ctrl+K 会话列表,Shift+Tab 循环审批(每次询问 → 自动 → 完全放行),
F1 或 `?` 帮助,Esc 关闭弹窗 / 取消编辑 / 停止当前一轮,Ctrl+B / Ctrl+J 收起左栏 / 右栏。
引擎的英文状态、错误和会话里的 `[cst]` 提示在页面上换成中文;一轮失败时(`turn_end.rc≠0`)
显示可读的原因(比如"API key 无效或已过期(HTTP 401)"、"用满了 30 步上限"),不再是 `rc=-1`。
TUI 里打 `/web` 会另起一个 WebUI 进程(`--web --workspace <当前目录> --new`),
日志在 `<workspace>/.agent/webui.log`:TUI 不能被 `cst_webui_run()` 的阻塞循环占住,
而且两个进程同时写一个会话文件会把它写坏,所以子进程总是开新会话。
自动压缩的门槛是模型上下文窗口的 **75%**(`CST_CTX_BUDGET` 可覆盖);到点后下一轮
开始前先总结历史,而不是等到塞满。
## 测试
```bash
make check # 含 tests/test_webui(HTTP/SSE/鉴权/消毒)
sh scripts/e2e_webui.sh # 用录制流跑完整轮次 + 权限往返,不需要 API key
# (含 e2e_webui_settings.py:假 /v1/models、换提供商/key、中文思考流;
# e2e_webui_sessions.py:重新生成/编辑重发/导出/重命名/删除/历史字段;
# e2e_webui_workspace.py:diff、运行设置、权限规则;
# e2e_webui_ext.py:/init、/skill、扩展、信任、思考开关)
sh scripts/e2e_webui.sh tests/fixtures/sse/deepseek-ctl-heredoc ctl-heredoc
# (deepseek-ctl 录的是一次契约错误连发的失败运行,不能拿来跑这个)
```
手工验收清单(浏览器):
1. 打开首页 → 顶部显示 workspace/模型/审批模式,右下状态"就绪"。
2. 输入一句话 → 助手气泡逐字出现;工具卡片自动展开、结束后带 `exit=` 标记。
3. 切到 `ask` 审批 → 写文件时弹窗;点"允许一次"后继续,点"拒绝"则该工具返回失败但对话继续。
4. `会话` 按钮:新建/切换后历史正确重建(刷新页面也一样):连续的思考和工具调用折叠成一行
过程块,展开是时间线,工具行点开能看到参数和输出;搜索、重命名、删除可用。
`/thinking off` 时流式输出的思考也保持收起。
4a. 最后一句提问上有"编辑重发",最后一个回答上有"重新生成";右栏文件行点开是真实 diff。
5. 拖入一张图片 → 输入框上方出现缩略图,发送后图片立刻出现在提问气泡上方,点开是大图预览。
6. 关掉标签页再打开 → 事件流自动续传,不丢已完成的消息。
## 限制与已知问题
* 同一会话同时只能有一轮(`POST /api/turn` 返回 409);TUI 与 WebUI 目前是
**二选一**(`--web` 时不启动 TUI),同进程并存留待后续的 hooks 分发层。
* 一个会话文件只能由一个进程写:不要同时用 TUI 打开 WebUI 正在用的那个 workspace。
* 没有 WebSocket;服务端→客户端用 SSE,客户端→服务端用 POST。够用且穿透代理更容易。
* 事件环只有 2000 条:断线太久(超过这个窗口)的客户端会从当前可用的最老事件开始,
这时应当重新拉一次 `/api/session`。
+6 -8
View File
@@ -1,11 +1,9 @@
# Windows self-test checklist
Run on a Windows machine (or wine) after MinGW build + busybox.exe beside the binary.
Automated: `scripts/win_test.sh` cross-builds and runs every test on the
English and Chinese Windows Server VMs as Administrator, a standard user and
a Chinese user name (VM tooling in `scripts/winvm/`).
1. `cst-agent.exe --help` exits 0
2. `set CST_API_KEY=...` then `--headless --workspace . --prompt "run echo hello"` — tool output contains hello
3. `write_file` under workspace succeeds; path outside workspace rejected
4. Credentials saved under `%APPDATA%\cst-agent\credentials`
5. With PDCurses build: launch without `--headless`, confirm TUI draws, `/help` works, Esc cancels
Shell must resolve to bundled busybox unless `CST_SHELL` is set.
Manual: the "Windows specifics" list in [compat.md](compat.md), plus the
general CLI and TUI items there. Install and sandbox details are in
[README-windows.md](README-windows.md).
+128 -4
View File
@@ -4,6 +4,7 @@
#include "ctl.h"
#include <stddef.h>
#include <time.h>
#ifdef __cplusplus
extern "C" {
@@ -11,7 +12,6 @@ extern "C" {
#define CST_AGENT_MAX_MSG 256
#define CST_AGENT_MAX_CONTENT (512 * 1024)
#define CST_AGENT_CTX_WINDOW 80 /* max non-system msgs sent to LLM */
#define CST_DEFAULT_MAX_STEPS 30
#define CST_DEFAULT_TOOL_OUT_CAP (16 * 1024)
@@ -47,6 +47,22 @@ int cst_protocol_is_text(cst_protocol_t p);
/* Eager protocols execute a call during the stream, at confirmation time. */
int cst_protocol_is_eager(cst_protocol_t p);
/* One file carried by a message (see cst_attach.h): an image, or a document
* whose stored payload is its text. The payload lives beside the session;
* this is the metadata plus the name of the stored copy. */
typedef struct {
char id[32]; /* unique inside the session ("img-1a2b3c4d") */
char name[256]; /* display name (original file name) */
char mime[32]; /* image/png, ...; application/pdf for a PDF whose text
* layer the stored file holds; text/plain for text */
char rel[256]; /* file name inside the session's attachment directory */
long long bytes; /* stored size (the text, for documents) */
int width, height; /* pixels, 0 when the header could not be read */
int pages; /* PDF page count, 0 for anything else */
char detail[8]; /* auto | low | high (wire hint, "" = auto) */
char origin[32]; /* user | view_image | mcp (for display only) */
} cst_attachment_t;
typedef struct {
cst_role_t role;
char *content;
@@ -55,6 +71,10 @@ typedef struct {
char tool_name[CTL_MAX_NAME];
char *tool_calls_json; /* assistant: OpenAI tool_calls array JSON, or NULL */
char *meta_json; /* optional JSON object (kind, protocol, ...) */
cst_attachment_t *attach; /* images shown with this message, or NULL */
int nattach;
char *reasoning; /* assistant: reasoning_content as streamed, or NULL */
int prefix; /* wire only: assistant prefix the model continues */
} cst_message_t;
typedef struct {
@@ -65,6 +85,7 @@ typedef struct {
char model[128];
char provider[32];
char effort[16]; /* low | medium | high | max */
char mode[16]; /* coding | general (CST_DEFAULT_MODE) */
cst_protocol_t protocol;
int max_steps; /* model calls per user turn */
size_t tool_out_cap; /* bytes of one tool output sent to the model */
@@ -97,6 +118,14 @@ typedef enum {
const char *cst_approval_name(int a);
int cst_approval_parse(const char *s, int *out);
/* Working mode: how the system prompt frames the agent. "coding" (default)
* is code framed; "general" is for tasks that are not about a codebase
* (documents, data, research, everyday files). Orthogonal to plan mode. */
#define CST_DEFAULT_MODE "coding"
int cst_mode_valid(const char *m);
/* Canonical name of m, or the default when m is unknown / empty. */
const char *cst_mode_name(const char *m);
typedef struct cst_session cst_session_t;
typedef struct cst_agent cst_agent_t;
@@ -113,6 +142,12 @@ const cst_message_t *cst_session_get(const cst_session_t *s, int i);
const char *cst_session_path(const cst_session_t *s);
int cst_session_clear(cst_session_t *s, int keep_system);
int cst_session_rewrite(cst_session_t *s, const cst_message_t *msgs, int n);
/* Tool output: role=tool, or a text-protocol user message answering a call. */
int cst_message_is_tool_result(const cst_message_t *m);
/* The transcript as Markdown, malloc'd (NULL on OOM). Tool output and call
* arguments are cut at 4000 bytes. Takes the session lock. */
char *cst_session_export_md(cst_session_t *s, const char *title,
const char *sid, const char *model);
/* Readers on another thread must hold the lock while using pointers returned
* by cst_session_get (append may reallocate). Append locks internally. */
void cst_session_lock(cst_session_t *s);
@@ -149,6 +184,10 @@ typedef struct {
char *out;
size_t out_len;
int exit_code;
/* Images the tool produced for the model (view_image, image-returning MCP
* tools). cst_tool_result_free releases them. */
cst_attachment_t *attach;
int nattach;
} cst_tool_result_t;
void cst_tool_result_free(cst_tool_result_t *r);
@@ -200,7 +239,9 @@ typedef struct {
char finish_reason[32];
long prompt_tokens;
long completion_tokens;
long cached_tokens;
long cached_tokens; /* prompt tokens read from the provider's cache */
long cache_miss_tokens; /* prompt tokens billed at the full input price */
long cache_write_tokens; /* prompt tokens written to the cache (Anthropic) */
double t_first_byte_ms; /* relative to request start; -1 if none */
double t_end_ms;
int client_stopped; /* stream ended by stop_flag (not an error) */
@@ -222,8 +263,15 @@ typedef struct {
void *userdata;
int enable_tools; /* send tools[] schema */
const cst_agent_config_t *tool_cfg; /* active tool set; NULL = all */
const char *tools_json; /* tools[] sent verbatim (frozen); NULL = tool_cfg */
int tool_choice_none; /* "tool_choice":"none" */
int send_reasoning; /* assistant reasoning goes back as reasoning_content */
int cache_style; /* CST_CACHE_* hints to add */
const char *cache_key; /* prompt_cache_key (CST_CACHE_OPENAI) */
const char *cache_ttl; /* cache_control ttl (CST_CACHE_ANTHROPIC); NULL = 5m */
int images_off; /* the model does not take images: send a note instead */
size_t tool_out_cap; /* wire cap for role=tool content; 0 = default */
const char *session_path; /* where stored images live; NULL = none */
volatile int *cancel_flag; /* non-zero → abort (error "cancelled") */
volatile int *stop_flag; /* non-zero → stop reading, not an error */
cst_llm_stats_t *stats; /* optional */
@@ -242,6 +290,8 @@ int cst_llm_list_models(const char *api_base, const char *api_key,
char *cst_llm_build_body(const cst_llm_request_t *req);
/* Reset the CST_RECORD_SSE / CST_REPLAY_SSE sequence counter. */
void cst_llm_trace_reset(void);
/* Set CURLOPT_CAINFO on a CURL * when a CA bundle file is known. */
void cst_curl_apply_ca(void *curl);
/* on_permission answers */
#define CST_PERM_DENY 0
@@ -291,9 +341,34 @@ cst_agent_config_t *cst_agent_config_mut(cst_agent_t *a);
void cst_agent_request_cancel(cst_agent_t *a);
int cst_agent_new_session(cst_agent_t *a, char *id_out, size_t id_sz);
int cst_agent_switch_session(cst_agent_t *a, const char *id);
/* Re-root the agent at another directory. Per-workspace state (permission
* rules, metrics, project notes) follows, the system prompt is rebuilt and the
* token budget is re-measured. The session is not touched: the caller decides
* whether to keep it or start a new one. */
int cst_agent_set_workspace(cst_agent_t *a, const char *path, char *err,
size_t err_sz);
int cst_agent_compact(cst_agent_t *a, char *err, size_t err_sz);
/* Bare id of the current session (file name without .jsonl). */
void cst_agent_session_id(const cst_agent_t *a, char *out, size_t out_sz);
/* Transcript path of the current session (images are stored beside it). */
const char *cst_agent_session_path(const cst_agent_t *a);
/* Images picked for the next user message (/attach). They ride along with the
* next cst_agent_run_turn*() call and are then part of the session.
* cst_agent_attach_file copies the file into the session store; *err gets a
* short reason on failure (-1). */
int cst_agent_attach_file(cst_agent_t *a, const char *path, char *err,
size_t err_sz);
/* Same for bytes already in memory (clipboard); name is display-only. */
int cst_agent_attach_bytes(cst_agent_t *a, const char *name, const char *data,
size_t n, char *err, size_t err_sz);
/* Whether the current model is shown images: 1 yes, -1 unknown (tried; a
* 400 turns sending off), 0 no (images go as text notes). */
int cst_agent_sees_images(const cst_agent_t *a);
int cst_agent_pending_count(const cst_agent_t *a);
/* One line "<name> (<mime>, 12 KB)"; -1 when i is out of range. */
int cst_agent_pending_describe(const cst_agent_t *a, int i, char *out,
size_t out_sz);
void cst_agent_pending_clear(cst_agent_t *a);
/* Latest todo_write list of this session, or NULL. */
const char *cst_agent_todo(const cst_agent_t *a);
/* AGENTS.md (or CST.md) text injected into the system prompt, or NULL. */
@@ -311,6 +386,19 @@ void cst_agent_undo_key(const cst_agent_t *a, char *sid, size_t sid_sz,
char *tag, size_t tag_sz);
/* Last prompt_tokens reported by the model (0 if unknown). */
long cst_agent_last_prompt_tokens(const cst_agent_t *a);
/* Token use since the session was opened, and of the latest turn. */
typedef struct {
int requests;
long prompt, cached, miss, write, out;
double cost; /* in currency; valid when priced */
int priced; /* every request had a known price */
char currency[8];
} cst_usage_t;
void cst_agent_usage(const cst_agent_t *a, cst_usage_t *session,
cst_usage_t *turn);
/* "[cst] ..." notes the agent appends when Plan mode, approval, branch,
* date or AGENTS.md change mid-session (the system prompt stays frozen). */
int cst_message_is_note(const cst_message_t *m);
const char *cst_agent_system_prompt(void);
const char *cst_agent_system_prompt_for(cst_protocol_t p);
@@ -330,13 +418,14 @@ int cst_creds_resolve(const char *provider, char *out, size_t out_sz,
/* Hidden console prompt (no echo). */
int cst_creds_prompt_console(const char *provider, char *out, size_t out_sz);
/* Last used provider, model per provider and effort, in
/* Last used provider, model per provider, effort and working mode, in
* <config>/settings.json. */
#define CST_SETTINGS_MAX_MODELS 16
#define CST_SETTINGS_MAX_TRUSTED 32
typedef struct {
char provider[32];
char effort[16];
char mode[16]; /* coding | general, "" = not chosen yet */
char approval[8]; /* "" = not chosen yet */
int show_thinking; /* TUI: Thought blocks start expanded */
/* workspaces whose .agent/plugins and .agent/mcp.json may run */
@@ -351,11 +440,13 @@ typedef struct {
int cst_settings_load(cst_settings_t *s);
int cst_settings_save(const cst_settings_t *s);
const char *cst_settings_model(const cst_settings_t *s, const char *provider);
/* Record cfg's provider, model and effort as the last used ones. */
/* Record cfg's provider, model, effort and mode as the last used ones. */
int cst_settings_remember(const cst_agent_config_t *cfg);
/* Record an approval mode the user picked (CST_APPROVAL_*). */
int cst_settings_remember_approval(int approval);
int cst_settings_remember_thinking(int show_thinking);
/* Record the working mode the user picked (coding | general). */
int cst_settings_remember_mode(const char *mode);
/* Workspace trust for project plugins / MCP servers (by real path). */
int cst_settings_trusted(const char *workspace);
int cst_settings_trust(const char *workspace, int on);
@@ -384,6 +475,39 @@ const cst_provider_t *cst_provider_at(int i);
/* Context window of a model in tokens (CST_CTX_TOKENS overrides). */
long cst_model_context_tokens(const char *model);
/* How a provider caches prompt prefixes.
* auto : implicit prefix cache, nothing to send (DeepSeek, MiniMax...)
* openai : implicit, routed by "prompt_cache_key"
* anthropic : explicit "cache_control" breakpoints on content parts */
typedef enum {
CST_CACHE_AUTO = 0,
CST_CACHE_OPENAI,
CST_CACHE_ANTHROPIC
} cst_cache_style_t;
typedef struct {
int cache_style; /* cst_cache_style_t */
int reasoning_passback; /* history keeps reasoning_content (DeepSeek) */
int prefix_completion; /* assistant "prefix": true via <base>/beta */
int vision; /* 1 takes images, 0 text only, -1 unknown */
} cst_model_caps_t;
/* CST_NO_CACHE_HINTS=1 forces CST_CACHE_AUTO. */
void cst_model_caps(const char *provider, const char *api_base,
const char *model, cst_model_caps_t *out);
/* Prices per million tokens. CST_PRICE="miss,hit,out[,write[,currency]]"
* overrides the built-in table. */
typedef struct {
char currency[8]; /* "¥", "$" */
double miss, hit, write, out;
int offpeak_half; /* DeepSeek: half price outside Beijing peak hours */
} cst_price_t;
/* 0 when a price is known for model, -1 otherwise. */
int cst_model_price(const char *model, cst_price_t *out);
/* DeepSeek peak hours: Beijing time Mon-Fri 9:00-12:00 and 14:00-18:00. */
int cst_price_is_peak(time_t when);
double cst_price_cost(const cst_price_t *p, long miss, long hit, long write,
long out, time_t when);
/* Reasoning effort: low | medium | high | max. */
#define CST_DEFAULT_EFFORT "medium"
int cst_effort_valid(const char *effort);
+91
View File
@@ -0,0 +1,91 @@
#ifndef CST_ATTACH_H
#define CST_ATTACH_H
#include "cst_agent.h"
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* --- attachments ---------------------------------------------------------
* An attachment is a validated copy of a file, stored beside the session so
* what a model is shown stays what was picked. Images are stored as they are
* and encoded into the request when it is built; a PDF is reduced to its text
* layer and a text file is stored as is, both of which travel inline as text
* (so a text-only model still gets them). Only the metadata goes in the
* JSONL. */
/* Supported: PNG, JPEG, GIF, WebP, BMP (static images), PDF, UTF-8 text. */
#define CST_ATTACH_MAX_BYTES (16 * 1024 * 1024) /* one picked file */
#define CST_ATTACH_MAX_TOTAL (32 * 1024 * 1024) /* per request */
#define CST_ATTACH_MAX_COUNT 8 /* per message */
/* Text stored and inlined per document (a PDF also caps its text here). */
#define CST_ATTACH_TEXT_MAX (4 * 1024 * 1024)
/* Images go on the wire as image parts; everything else is inlined as text. */
int cst_attach_is_image(const cst_attachment_t *a);
int cst_attach_is_text(const cst_attachment_t *a);
/* Magic-byte test: 0 and a canonical mime when data holds a supported
* image, -1 otherwise (an extension alone never decides). */
int cst_attach_sniff(const void *data, size_t n, char *mime, size_t mime_sz);
/* Pixel size from the header when it is readable; 0 for unknown. */
int cst_attach_dimensions(const void *data, size_t n, int *w, int *h);
/* A stored file name as store() makes it ("<id>.<ext>"); rejects anything
* that could leave the attachment directory. */
int cst_attach_rel_ok(const char *rel);
/* Directory holding this session's stored images (created on demand). */
int cst_attach_dir(const char *session_path, char *out, size_t out_sz);
/* Absolute path of a stored image (0 / -1). */
int cst_attach_full_path(const char *session_path, const cst_attachment_t *a,
char *out, size_t out_sz);
/* Read src_path and copy it into the session's attachment directory: an image
* as it is, a PDF as its extracted text, anything else only when it is text.
* Returns a malloc'd entry the caller owns (cst_attach_free), or NULL with
* err filled. */
cst_attachment_t *cst_attach_import_file(const char *session_path,
const char *src_path, char *err,
size_t err_sz);
/* Store already-read image bytes (clipboard, downloads); name is
* display-only. */
cst_attachment_t *cst_attach_import_bytes(const char *session_path,
const char *name, const char *data,
size_t n, char *err, size_t err_sz);
/* Validate and read back a stored payload, or NULL when it is gone. */
char *cst_attach_read(const char *session_path, const cst_attachment_t *a,
size_t *out_len, char *err, size_t err_sz);
/* "data:image/png;base64,...." for the wire; images only, NULL otherwise. */
char *cst_attach_data_url(const char *session_path, const cst_attachment_t *a);
/* "1920x1080 png 412 KB" / "pdf 1.2 MB, 12 pages" one-liner for prompts. */
void cst_attach_describe(const cst_attachment_t *a, char *out, size_t out_sz);
void cst_attach_free(cst_attachment_t *a);
/* Free a list of n entries allocated as separate blocks. */
void cst_attachments_free(cst_attachment_t *list, int n);
/* Deep copy (NULL on OOM or when n <= 0). */
cst_attachment_t *cst_attachments_dup(const cst_attachment_t *list, int n);
/* Append src to *list (grows with realloc). 0 / -1. */
int cst_attachments_push(cst_attachment_t **list, int *n, int *cap,
const cst_attachment_t *src);
/* JSON array of n attachments for the session JSONL (malloc'd), or NULL. */
char *cst_attachments_to_json(const cst_attachment_t *list, int n);
/* Parse a JSON array; caller frees with cst_attachments_free. */
cst_attachment_t *cst_attachments_from_json(const void *array_json, int *n_out);
/* Base64 of n bytes (malloc'd, no line breaks). */
char *cst_base64_encode(const void *data, size_t n);
/* Standard or URL-safe base64 (whitespace ignored) to bytes; malloc'd,
* NULL on a bad character. */
char *cst_base64_decode(const char *s, size_t n, size_t *out_len);
#ifdef __cplusplus
}
#endif
#endif
+49
View File
@@ -0,0 +1,49 @@
#ifndef CST_CLIPBOARD_H
#define CST_CLIPBOARD_H
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* Image bytes on the system clipboard (PNG, or BMP from a Windows DIB).
* Wayland: wl-paste; X11: xclip; macOS: pngpaste or osascript; Windows: the
* clipboard API. Returns 0 with *data malloc'd, or -1 with err filled in
* ("no image on the clipboard", "install wl-clipboard", ...). */
int cst_clipboard_image(char **data, size_t *n, char *err, size_t err_sz);
/* Cap on text moved through the clipboard (read and write). */
#define CST_CLIPBOARD_TEXT_MAX (1024 * 1024)
/* UTF-8 text on the system clipboard: Wayland (wl-paste), X11 (xclip),
* macOS (pbpaste), Windows (CF_UNICODETEXT). Returns 0 with *data malloc'd
* and NUL-terminated, or -1 with err filled in ("no text on the clipboard",
* "reading the clipboard needs wl-clipboard", ...). */
int cst_clipboard_text(char **data, size_t *n, char *err, size_t err_sz);
/* Replace the clipboard with UTF-8 text: wl-copy / xclip / pbcopy, or the
* Windows clipboard API. Returns 0 on success, else -1 with err. */
int cst_clipboard_set_text(const char *text, size_t n, char *err,
size_t err_sz);
/* Windows CF_DIB bytes (BITMAPINFOHEADER + pixels) as an image file: a PNG
* for uncompressed 24/32-bit bitmaps, else a BMP. NULL when malformed. */
char *cst_dib_to_image(const void *dib, size_t sz, size_t *n);
#define CST_PASTE_MAX_PATHS 8
#define CST_PASTE_PATH_MAX 1024
/* Split pasted text into file paths the way terminals paste dropped files:
* whitespace- or newline-separated, '...' / "..." quoting, file:// URIs
* (percent-decoded) and, outside Windows, backslash escapes. Returns the
* count, or -1 when the text is empty or holds more than max paths. Whether
* the paths exist is up to the caller. */
int cst_paste_paths(const char *text, size_t len,
char out[][CST_PASTE_PATH_MAX], int max);
#ifdef __cplusplus
}
#endif
#endif
+108
View File
@@ -0,0 +1,108 @@
#ifndef CST_HTTP_H
#define CST_HTTP_H
/* A small HTTP/1.1 server for the built-in WebUI (src/http_srv.c).
*
* Only what a local single-page app needs: exact-match routes, keep-alive,
* fixed-length bodies and chunked streaming responses (SSE). No TLS, no
* HTTP/2, no chunked *request* bodies, no CGI. Every connection is served
* by its own thread; a handler may keep the connection as long as it wants
* (that is how the event stream works). */
#include "cst_net.h"
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct cst_http_conn cst_http_conn_t;
typedef struct {
char method[8]; /* GET / POST / DELETE ... */
char path[1024]; /* percent-decoded, without the query string */
char query[1024]; /* raw, without '?' */
const char *body; /* NUL-terminated; may hold binary bytes */
size_t body_len;
/* Headers of interest, NULL when absent (pointers into the request). */
const char *authorization;
const char *cookie;
const char *origin;
const char *host;
const char *content_type;
const char *user_agent;
const char *last_event_id; /* an EventSource reconnecting */
int keep_alive;
} cst_http_req_t;
/* 0 = handled (a response was sent), -1 = not found (the server replies). */
typedef int (*cst_http_fn)(cst_http_conn_t *c, void *ud);
typedef struct {
const char *method; /* "*" matches any */
const char *path; /* exact; "*" = fallback (try last) */
cst_http_fn fn;
void *ud;
int auth; /* ask the auth hook before the handler */
size_t max_body; /* request body cap; 0 = server default */
} cst_http_route_t;
typedef struct {
const char *host; /* NULL/"" = 127.0.0.1 */
int port; /* 0 = pick a free port */
size_t max_body; /* default 1 MiB */
int max_conn; /* default 32 */
int read_timeout_ms; /* waiting for the request headers (default 15000) */
int idle_timeout_ms; /* keep-alive pause between requests (default 60000) */
const cst_http_route_t *routes;
int nroutes;
/* 0 = allow, -1 = the server answers 401 before the handler runs. */
int (*auth)(const cst_http_req_t *req, void *ud);
void *auth_ud;
/* One line per request, e.g. "GET /api/state -> 200". */
void (*on_log)(const char *line, void *ud);
void *log_ud;
} cst_http_opts_t;
typedef struct cst_http_server cst_http_server_t;
/* Start listening. NULL on failure with a reason in err. */
cst_http_server_t *cst_http_start(const cst_http_opts_t *o, char *err,
size_t err_sz);
int cst_http_port(const cst_http_server_t *s);
/* Non-zero once stop() was called: long-lived handlers must return. */
int cst_http_stopping(const cst_http_server_t *s);
/* Close the listener and every connection, then wait (30 s at most) for the
* handler threads. Returns how many still run: their ud must stay alive. */
int cst_http_stop(cst_http_server_t *s);
/* Does nothing while handlers still run on s. */
void cst_http_free(cst_http_server_t *s);
/* --- inside a handler --- */
const cst_http_req_t *cst_http_request(cst_http_conn_t *c);
int cst_http_send(cst_http_conn_t *c, int status, const char *ctype,
const void *body, size_t n);
int cst_http_send_str(cst_http_conn_t *c, int status, const char *ctype,
const char *s);
/* json may be NULL (then "null" is sent); it is copied, not owned. */
int cst_http_send_json(cst_http_conn_t *c, int status, const char *json);
/* {"error":"..."} */
int cst_http_send_err(cst_http_conn_t *c, int status, const char *msg);
/* True once a response (or stream) began for this request. */
int cst_http_sent(const cst_http_conn_t *c);
/* Chunked response: begin(), any number of write()s, end(). */
int cst_http_stream_begin(cst_http_conn_t *c, const char *ctype);
int cst_http_stream_write(cst_http_conn_t *c, const void *data, size_t n);
int cst_http_stream_end(cst_http_conn_t *c);
/* Query parameter, percent-decoded ('+' = space). 0 when found. */
int cst_http_query(const cst_http_req_t *req, const char *key, char *out,
size_t out_sz);
#ifdef __cplusplus
}
#endif
#endif /* CST_HTTP_H */
+1
View File
@@ -64,6 +64,7 @@ typedef enum {
CST_KA_COMPACT,
CST_KA_COPY,
CST_KA_EXPORT,
CST_KA_PASTE_IMAGE,
/* model */
CST_KA_MODEL,
CST_KA_PROVIDER,
+92
View File
@@ -0,0 +1,92 @@
#ifndef CST_MODELS_H
#define CST_MODELS_H
/* Model registry: what cst-agent knows about providers and models.
*
* Three layers, highest first, merged field by field:
* 1. <config>/models.json user overrides and extra providers
* 2. <config>/models-cache/<provider>.json GET /models metadata, exact id
* 3. built-in data (src/models_builtin.json, embedded at build time)
* then defaults (128000 tokens, vision unknown). Model rules match the name
* without its "vendor/" prefix, case-insensitively: a plain "match" is a
* prefix, one with '*' is a glob, one with '/' is tested against the full
* id. Within a layer the most specific rule (most literal characters) that
* sets a field wins, so table order does not matter.
*
* CST_CTX_TOKENS, CST_PRICE, CST_MODELS and CST_NO_CACHE_HINTS still
* override everything. */
#include "cst_agent.h"
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
long context; /* tokens */
int vision; /* 1 takes images, 0 text only, -1 unknown */
int cache_style; /* cst_cache_style_t */
int reasoning; /* takes OpenAI-style reasoning_effort */
int has_price;
cst_price_t price;
/* Layers that supplied anything, highest first: "user+api+built-in",
* "default", "env+built-in"... */
char source[48];
} cst_model_info_t;
/* provider may be NULL: then every provider's fetched cache is searched. */
void cst_model_info(const char *provider, const char *model,
cst_model_info_t *out);
/* Short picker label: "1M · vision", "200K · text only", "128K". */
void cst_model_info_label(const cst_model_info_t *mi, char *out, size_t n);
/* One entry of GET /models, with the optional OpenRouter-style metadata. */
typedef struct {
char id[128];
long context; /* 0 = not given */
int vision; /* -1 = not given */
int has_price; /* prices in USD per 1M tokens */
double miss, hit, write, out;
} cst_model_meta_t;
/* Like cst_llm_list_models, keeping context_length,
* architecture.input_modalities (or modalities) and pricing when present. */
int cst_llm_list_models_meta(const char *api_base, const char *api_key,
cst_model_meta_t *out, int max_out, char *err,
size_t err_sz);
#define CST_MODELS_CACHE_TTL (24L * 3600)
/* Store a fetched list as <config>/models-cache/<provider>.json and use it
* from now on. 0 / -1. */
int cst_models_cache_save(const char *provider, const char *api_base,
const cst_model_meta_t *list, int n);
/* The cached list for provider: count (up to max), or -1 when there is
* none. *age_s gets the seconds since it was fetched. */
int cst_models_cache_load(const char *provider, cst_model_meta_t *out, int max,
long *age_s);
/* The "models" a provider entry lists (built-in or user file). */
int cst_provider_models(const cst_provider_t *p, char out[][128], int max);
/* Where a provider entry came from: "built-in" or "user". */
const char *cst_provider_source(const cst_provider_t *p);
/* The user file's path (it may not exist). */
int cst_models_user_path(char *out, size_t out_sz);
/* Drop everything and read the built-in data, the user file and the caches
* again. Provider pointers stay valid but may name another provider. */
void cst_models_reload(void);
/* Tests: load these documents instead of the built-in data and the user
* file (NULL = none). Caches still come from <config>. 0, or -1 with err. */
int cst_models_load_json(const char *builtin_json, const char *user_json,
char *err, size_t err_sz);
/* The embedded built-in document. */
const char *cst_models_builtin_json(void);
/* Problems found while loading (bad user file...), or "". */
const char *cst_models_load_error(void);
#ifdef __cplusplus
}
#endif
#endif
+68
View File
@@ -0,0 +1,68 @@
#ifndef CST_NET_H
#define CST_NET_H
/* Minimal TCP sockets for the embedded HTTP server (src/net.c).
*
* The point of this layer is that src/http_srv.c and src/webui.c never
* include a socket header: Unix file descriptors and Windows SOCKETs differ
* in type, close() and error reporting, and Windows needs WSAStartup before
* anything works. */
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
#ifdef _WIN32
typedef intptr_t os_socket_t;
#define OS_SOCKET_BAD ((os_socket_t)-1)
#else
typedef int os_socket_t;
#define OS_SOCKET_BAD (-1)
#endif
/* Longest "address:port" os_net_peer writes (IPv6 hosts are bracketed so a
* URL built from it stays unambiguous). */
#define OS_NET_ADDR_MAX 64
/* Winsock startup (no-op on Unix, which also ignores SIGPIPE so a client
* that hangs up cannot kill the process). 0 / -1. */
int os_net_init(void);
void os_net_cleanup(void);
/* Bind and listen on host:port. host may be NULL/"" (then "127.0.0.1"),
* "0.0.0.0"/"::" (every interface) or any literal address / name; port 0
* picks a free one. Returns the listening socket, or OS_SOCKET_BAD with a
* short reason in err. */
os_socket_t os_net_listen(const char *host, int port, char *err, size_t err_sz);
/* The port actually bound (0 when unknown). */
int os_net_port(os_socket_t s);
/* A one-line "host:port" for messages. */
void os_net_peer(os_socket_t s, char *out, size_t out_sz);
/* Connect to host:port, waiting up to timeout_ms: the socket, or
* OS_SOCKET_BAD. Used by the tests and by anything that drives the WebUI
* API from inside the process. */
os_socket_t os_net_connect(const char *host, int port, int timeout_ms);
/* Accept one connection, waiting up to timeout_ms (<0 = forever).
* Returns the socket, OS_SOCKET_BAD on error, or OS_SOCKET_BAD with
* *timed_out set when the wait ran out. */
os_socket_t os_net_accept(os_socket_t s, int timeout_ms, int *timed_out);
/* Read at most n bytes, waiting up to timeout_ms: bytes read, 0 at EOF,
* -1 on error, -2 when nothing arrived in time. */
long os_net_recv(os_socket_t s, void *buf, size_t n, int timeout_ms);
/* Write everything (timeout_ms per call): 0 / -1. */
int os_net_send_all(os_socket_t s, const void *buf, size_t n, int timeout_ms);
/* Wake up a thread blocked in recv/send on this socket (shutdown, no close). */
void os_net_shutdown(os_socket_t s);
void os_net_close(os_socket_t s);
#ifdef __cplusplus
}
#endif
#endif /* CST_NET_H */
+27
View File
@@ -0,0 +1,27 @@
#ifndef CST_PDF_H
#define CST_PDF_H
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* Text layer of a PDF, in page order, with no external dependency (no
* poppler, no zlib): classic objects and object streams, FlateDecode plus
* ASCIIHex/ASCII85, PNG/TIFF predictors, Tj/TJ/'/" text with per-font
* ToUnicode CMaps and a WinAnsi / Latin-1 fallback. Scanned pages have no
* text layer and are reported as such.
* Returns a malloc'd NUL-terminated UTF-8 string, or NULL with err filled in
* ("PDF is encrypted ...", "no extractable text ..."). *pages (optional) gets
* the page count; the result is truncated at CST_PDF_TEXT_MAX bytes. */
char *cst_pdf_text(const void *data, size_t n, int *pages, char *err,
size_t err_sz);
#define CST_PDF_TEXT_MAX (4u * 1024u * 1024u)
#ifdef __cplusplus
}
#endif
#endif
+3
View File
@@ -26,6 +26,9 @@ void cst_perms_suggest_prefix(const char *cmd, char *out, size_t out_sz);
/* 1 when every segment of cmd (split on | || && ;) is a read-only command
* from a fixed whitelist, with no output redirection or substitution. */
int cst_shell_is_safe(const char *cmd);
/* 1 when cmd looks like it talks to the network (curl, ssh, git push,
* npm install, ...). Errs on the side of yes. */
int cst_shell_uses_network(const char *cmd);
#ifdef __cplusplus
}
+79
View File
@@ -0,0 +1,79 @@
#ifndef CST_PROVIDER_SWITCH_H
#define CST_PROVIDER_SWITCH_H
/* Model lists, API keys and provider switches, shared by the TUI and the
* WebUI so both front ends validate and persist the same way.
*
* A key is checked with GET /models: a 401/403 (CST_MODELS_AUTH) rejects
* it, any other failure (offline, no /models route) lets it through
* unverified. A key that passes is saved to <config>/credentials. Keys are
* never copied into err. */
#include "cst_agent.h"
#include "cst_models.h"
#ifdef __cplusplus
extern "C" {
#endif
#define CST_PS_MAX_MODELS 512
#define CST_PS_UNKNOWN (-1) /* no such provider */
#define CST_PS_NEED_KEY (-10) /* no key given, none saved or in the env */
/* Non-negative results are flags: */
#define CST_PS_UNVERIFIED 1 /* /models failed for another reason; err says */
#define CST_PS_UNSAVED 2 /* the credentials file could not be written */
typedef struct {
char id[128];
int api; /* listed by the provider's /models (else built-in only) */
cst_model_info_t info;
} cst_ps_model_t;
/* GET /models; a good answer refreshes models-cache/<provider>.json. The
* count, CST_MODELS_AUTH, or another negative value with err set. */
int cst_ps_fetch(const char *provider, const char *api_base, const char *key,
char ids[][128], int max, char *err, size_t err_sz);
/* The cached list: count, or -1; *fresh when younger than the TTL. */
int cst_ps_cached(const char *provider, char ids[][128], int max, int *fresh);
/* Base URL for provider: cfg's own when it is the current provider (it may
* be overridden), else the preset's; "" when unknown. */
const char *cst_ps_base(const cst_agent_config_t *cfg, const char *provider);
/* The model picker list for cfg's provider: the API's ids (the cache while
* it is fresh, else a fetch; refresh always fetches) followed by the
* built-in names it did not list. *api_rc gets the fetch result (count,
* CST_MODELS_AUTH, -1 with err) or the cached count. Returns the count. */
int cst_ps_models(const cst_agent_config_t *cfg, int refresh,
cst_ps_model_t *out, int max, int *api_rc, char *err,
size_t err_sz);
/* Checks key against /models and saves it: CST_MODELS_AUTH (err set), or
* the CST_PS_* flags. When ids is given it gets the fetched list and *nids
* the raw fetch result. */
int cst_ps_verify_save(const char *provider, const char *api_base,
const char *key, char ids[][128], int max, int *nids,
char *err, size_t err_sz);
/* Point cfg at provider p with key: base URL, the remembered model (else
* the preset's), settings saved. */
void cst_ps_apply_provider(cst_agent_config_t *cfg, const cst_provider_t *p,
const char *key);
/* The whole switch without a prompt: key NULL/"" uses the saved or env
* key (else CST_PS_NEED_KEY); a given key is verified and saved first. A
* preset with no model takes the first one the API lists. The CST_PS_*
* flags, or CST_PS_UNKNOWN / CST_PS_NEED_KEY / CST_MODELS_AUTH. */
int cst_ps_switch_provider(cst_agent_config_t *cfg, const char *name,
const char *key, char *err, size_t err_sz);
/* Replace the current provider's key: verified, saved, put in cfg. Same
* results as cst_ps_verify_save. */
int cst_ps_set_key(cst_agent_config_t *cfg, const char *key, char *err,
size_t err_sz);
#ifdef __cplusplus
}
#endif
#endif
+25 -10
View File
@@ -2,16 +2,19 @@
#define CST_SANDBOX_H
/*
* OS sandbox for run_shell: reads and exec stay allowed everywhere, writes
* are limited to the workspace, the temp dir, /dev/null and /dev/tty, and
* the network is cut where the OS can do it.
* Linux Landlock (network needs ABI 4, kernel 6.7+)
* OS sandbox for run_shell: reads, exec and the network stay allowed;
* writes are limited to the workspace, the temp dir, /dev/null and /dev/tty.
* Linux Landlock
* Linux seccomp user-notify when Landlock is missing (Android/Termux,
* older kernels): a supervisor thread checks the path of every
* write-type syscall and IPv4/IPv6 sockets are refused. Guards
* write-type syscall. Guards
* against mistakes, not hostile code (the path is checked before
* the kernel reads it again).
* macOS sandbox_init (Seatbelt profile)
* Windows Low integrity level: a restricted token (no admin privileges,
* Low IL) can only write objects labeled Low, so the workspace and
* a temp dir get that label; the credentials file is labeled
* no-read-up.
* other none: callers fall back to rule-based approval
*/
@@ -27,17 +30,18 @@ typedef enum {
CST_SANDBOX_NONE = 0,
CST_SANDBOX_LANDLOCK,
CST_SANDBOX_SEATBELT,
CST_SANDBOX_SECCOMP
CST_SANDBOX_SECCOMP,
CST_SANDBOX_LOWIL
} cst_sandbox_kind_t;
/* What this process can use; cached. CST_SANDBOX=off|0|none forces NONE,
* CST_SANDBOX=landlock|seccomp allows only that kind. */
* CST_SANDBOX=landlock|seccomp|lowil allows only that kind. */
int cst_sandbox_probe(void);
/* Drops the cache and probes again (tests that change CST_SANDBOX). */
int cst_sandbox_reprobe(void);
/* "landlock" | "seatbelt" | "seccomp" | "none" */
/* "landlock" | "seatbelt" | "seccomp" | "lowil" | "none" */
const char *cst_sandbox_name(int kind);
/* Landlock ABI version (0 when unavailable); ≥4 also blocks TCP. */
/* Landlock ABI version (0 when unavailable). */
int cst_sandbox_landlock_abi(void);
typedef struct {
@@ -45,6 +49,9 @@ typedef struct {
int ruleset_fd; /* Landlock, built in the parent */
char *profile; /* Seatbelt */
void *sc; /* seccomp: filter program and supervisor thread */
void *token; /* lowil: primary token for CreateProcessAsUserW */
char *tmpdir; /* lowil: Low-labeled TEMP for the child */
char *note; /* lowil: shown once when a directory was just labeled */
} cst_sandbox_plan_t;
/* Parent side: build the policy. Returns 0, or -1 (plan->kind = NONE). */
@@ -61,9 +68,17 @@ int cst_sandbox_run_shell(const char *workspace, const char *cmd,
size_t max_out, os_run_result_t *r);
/* Output of a failed sandboxed run that looks like the sandbox got in the
* way (EACCES / EPERM / EROFS / blocked network). */
* way (EACCES / EPERM / EROFS). */
int cst_sandbox_blocked(const char *out, size_t n, int exit_code);
/* lowil: directories this user's cst-agent labeled Low (recorded in the
* config dir), one per line; malloc'd, NULL when there are none. */
char *cst_sandbox_labels(void);
/* Gives every recorded directory its old label back. Returns how many were
* restored (-1 when some failed; those stay recorded); *report gets a
* malloc'd summary. */
int cst_sandbox_reset_labels(char **report);
#ifdef __cplusplus
}
#endif
+2
View File
@@ -45,6 +45,8 @@ char *cst_skill_read(const char *workspace, const char *name,
* task. NULL when the skill does not exist. malloc'd. */
char *cst_skill_prompt(const char *workspace, const char *name,
const char *task);
/* User message for `/init`: have the agent write AGENTS.md. */
const char *cst_init_prompt(void);
/* Register the `skill` tool (idempotent). */
int cst_skills_register_tool(void);
+32
View File
@@ -25,6 +25,38 @@ char *cst_buf_steal(cst_buf_t *b);
size_t cst_utf8_floor(const char *s, size_t n);
/* Copy s[0..n) replacing invalid UTF-8 with '?'. malloc'd, NUL-terminated. */
char *cst_utf8_sanitize(const char *s, size_t n, size_t *out_len);
int cst_utf8_valid(const char *s, size_t n);
/* Re-chunks a byte stream so no piece ends inside a UTF-8 sequence: the
* incomplete tail (at most 3 bytes) waits in the hold for the next feed. */
typedef struct {
char b[4];
size_t n;
} cst_utf8_hold_t;
/* Returns the complete prefix of hold+s[0..n) (malloc'd, NUL-terminated,
* *out_len may be 0), or NULL on OOM. */
char *cst_utf8_hold_feed(cst_utf8_hold_t *h, const char *s, size_t n,
size_t *out_len);
/* Text files may start with a UTF-8 BOM (Windows editors): its length (0/3),
* and an in-place strip of the NUL-terminated s (*len updated when given). */
size_t cst_bom_len(const char *s, size_t n);
void cst_strip_bom(char *s, size_t *len);
/* Line endings: "\r\n" -> "\n" in place (returns the new length); whether
* every line of s ends in "\r\n" (and at least one does); and a malloc'd
* copy of s with bare "\n" turned into "\r\n" when crlf, its BOM dropped and
* a BOM added when bom. Edits keep a Windows file's style with these. */
size_t cst_crlf_to_lf(char *s, size_t n);
int cst_text_is_crlf(const char *s, size_t n);
char *cst_text_restyle(const char *s, size_t n, int crlf, int bom,
size_t *out_len);
/* fnmatch(3) without FNM_PATHNAME ('*' also crosses '/'): * ? [..] [!..]
* [[:class:]] and backslash escapes over UTF-8 code points. Returns 0 on a
* match. Always case-insensitive on Windows. */
#define CST_FNM_PERIOD 1 /* a leading '.' must be matched literally */
#define CST_FNM_CASEFOLD 2
int cst_fnmatch(const char *pat, const char *str, int flags);
uint64_t cst_fnv1a64(uint64_t h, const void *data, size_t n);
#define CST_FNV_INIT 1469598103934665603ULL
+9 -1
View File
@@ -1,6 +1,14 @@
#ifndef CST_VERSION_H
#define CST_VERSION_H
#define CST_VERSION "0.5.0"
#define CST_VERSION_MAJOR 0
#define CST_VERSION_MINOR 8
#define CST_VERSION_PATCH 6
#define CST_VERSION_STR_(x) #x
#define CST_VERSION_STR(x) CST_VERSION_STR_(x)
#define CST_VERSION \
CST_VERSION_STR(CST_VERSION_MAJOR) \
"." CST_VERSION_STR(CST_VERSION_MINOR) "." CST_VERSION_STR(CST_VERSION_PATCH)
#endif
+48
View File
@@ -0,0 +1,48 @@
#ifndef CST_WEB_MD_H
#define CST_WEB_MD_H
/* Markdown for the WebUI: the same sashidown renderer the TUI uses, but
* HTML output with a strict allowlist.
*
* sashidown renders straight to HTML and escapes text, quotes and '&', so
* no raw markup can reach the browser. Two holes remain, and both matter
* because every byte rendered here comes from a model or a tool: a link's
* href keeps any scheme ("javascript:") and an image's src may point
* anywhere (a tracking pixel, or worse). cst_web_md_* therefore rewrites
* tags instead of passing them through: unknown tags are dropped, hrefs
* are limited to http/https/mailto/relative, and images to the session's
* own attachment endpoint.
*
* The streaming form renders confirmed HTML as the model talks, which is
* the CST property the project is built on: nothing is emitted before it
* is safe to emit. A tag split across two deltas is held back until its
* '>' arrives. */
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* Called with chunks of sanitized HTML; return non-zero to stop. */
typedef int (*cst_web_md_emit_fn)(const char *html, size_t len, void *ud);
typedef struct cst_web_md cst_web_md_t;
cst_web_md_t *cst_web_md_new(cst_web_md_emit_fn emit, void *ud);
/* Feed model output. Returns 0, or -1 on OOM. */
int cst_web_md_feed(cst_web_md_t *s, const char *data, size_t len);
/* Flush the pending input and close the renderer. */
int cst_web_md_finish(cst_web_md_t *s);
void cst_web_md_free(cst_web_md_t *s);
/* One-shot: markdown -> sanitized HTML (malloc'd, never NULL on success). */
char *cst_web_md_html(const char *md, size_t len);
/* The sanitizer alone (tests, and any HTML that did not come from here). */
char *cst_web_md_sanitize(const char *html, size_t len);
#ifdef __cplusplus
}
#endif
#endif /* CST_WEB_MD_H */
+44
View File
@@ -0,0 +1,44 @@
#ifndef CST_WEBUI_H
#define CST_WEBUI_H
/* The built-in WebUI: an HTTP + SSE front end for one agent process.
*
* It is a second implementation of the same interface the TUI uses
* (cst_agent_hooks_t), which is why the engine needs no changes: the turn
* runs on a worker thread and every live event goes out as a Server-Sent
* Event. The protocol is documented in docs/webui.md and is meant to be
* the contract a dedicated client speaks later. */
#include "cst_agent.h"
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
/* What --web listens on when --web-port/CST_WEB_PORT says nothing (the
* README and --help quote this number, so keep them in sync). */
#define CST_WEBUI_DEFAULT_PORT 8787
typedef struct {
const char *host; /* NULL = 127.0.0.1; "0.0.0.0" = every interface */
int port; /* 0 = pick a free port */
const char *token; /* NULL = generate one when needed */
const char *root; /* serve the UI from this directory instead of the
copy embedded in the binary (development) */
int open_browser; /* best-effort xdg-open / open / start */
/* Prompt to run once at startup (--web -p ...); NULL = wait for the UI. */
const char *prompt;
int perm_timeout_s; /* how long a permission question waits (0 = 600) */
} cst_webui_opts_t;
/* Serve until interrupted (Ctrl+C) or the event stream asks to quit.
* Returns the process exit code. */
int cst_webui_run(cst_agent_t *agent, const cst_webui_opts_t *opts);
#ifdef __cplusplus
}
#endif
#endif /* CST_WEBUI_H */
+96 -5
View File
@@ -22,11 +22,36 @@ int os_realpath(const char *path, char *out, size_t out_sz);
/* True if path is under root (after realpath/norm). Trailing slash rules apply. */
int os_path_under(const char *root, const char *path);
/* --- directory listing / file type --- */
typedef struct os_dir os_dir_t;
typedef struct {
const char *name; /* UTF-8; valid until the next os_dir_next */
int is_dir;
int is_link; /* symlink, or a junction on Windows (not followed) */
} os_dirent_t;
os_dir_t *os_dir_open(const char *path);
/* Next entry, skipping "." and ".."; NULL at the end. */
const os_dirent_t *os_dir_next(os_dir_t *d);
void os_dir_close(os_dir_t *d);
typedef struct {
int is_dir, is_reg, is_link;
long long size;
double mtime; /* seconds since the epoch */
} os_stat_t;
/* Like lstat(2): a link is reported as a link, not followed. */
int os_lstat(const char *path, os_stat_t *st);
/* Replace `to` if it exists (atomic where the OS allows). */
int os_rename_replace(const char *from, const char *to);
/* --- dirs / config --- */
int os_mkdir(const char *path, int mode);
int os_mkdir_p(const char *path, int mode);
int os_config_dir(char *out, size_t out_sz);
int os_temp_dir(char *out, size_t out_sz);
/* Point TMPDIR at os_temp_dir() so child processes use the same writable
* directory. Call once at startup, before any threads. */
void os_temp_dir_export(void);
/* Create unique temp file; writes path into out; returns fd or -1. Caller closes. */
int os_mkstemp(char *out, size_t out_sz, const char *prefix);
@@ -67,9 +92,32 @@ int os_run_shell_ex(const char *cwd, const char *cmd, size_t max_out,
os_run_result_t *r, os_child_hook_fn child_hook,
void *ud);
/* Child processes get stdin=/dev/null, their own process group, and are
* killed after the timeout (CST_SHELL_TIMEOUT seconds, default 120) or when
* *flag becomes non-zero. Exit code 124 = timeout, 130 = cancelled. */
#ifdef _WIN32
/* Windows: run under `token` (a primary token, e.g. the Low-IL sandbox
* token; NULL = ours) with env_add "KEY=VALUE" entries layered on the
* inherited environment. Every child runs in a kill-on-close Job Object, so
* timeouts and cancels end the whole process tree. */
int os_run_capture_token(const char *cwd, const char *const argv[],
size_t max_out, os_run_result_t *r, void *token,
const char *const env_add[]);
/* os_run_shell_c under token, with env_add. */
int os_run_shell_token(const char *cwd, const char *cmd, size_t max_out,
os_run_result_t *r, void *token,
const char *const env_add[]);
/* "Windows 10.0.26100 x86_64" (the real version, not the compat shim's). */
void os_windows_version(char *out, size_t n);
/* Sets the mandatory-label SACL, e.g. "S:(ML;OICI;NW;;;LW)"; 0 / -1. */
int os_win_set_label(const char *path, const char *sddl);
/* The explicit (not inherited) label as SDDL, "S:" when there is none. */
int os_win_get_label(const char *path, char *out, size_t n);
/* CF_UNICODE_TEXT clipboard (the console may not do OSC 52). */
int os_clipboard_set(const char *utf8, size_t n);
#endif
/* Child processes get stdin=/dev/null and their own process group. They run
* with no time limit unless one is asked for (CST_SHELL_TIMEOUT seconds, or
* os_set_run_timeout()); either way a non-zero *flag cancels them.
* Exit code 124 = killed by that timeout, 130 = cancelled. */
void os_set_cancel_flag(volatile int *flag);
volatile int *os_get_cancel_flag(void);
void os_set_run_timeout(int seconds); /* this thread; 0 = default */
@@ -80,7 +128,7 @@ int os_get_run_timeout(void);
* the inherited environment (NULL-terminated, may be NULL). */
typedef struct {
#ifdef _WIN32
void *proc, *in_w, *out_r;
void *proc, *in_w, *out_r, *job;
#else
int pid, in_fd, out_fd;
#endif
@@ -106,10 +154,53 @@ void os_run_parallel(os_task_fn fn, void *const args[], int n);
/* Monotonic milliseconds (arbitrary epoch). */
double os_now_ms(void);
/* --- text encoding ---
* cst-agent is UTF-8 throughout. On Windows the manifest makes UTF-8 the
* process code page; os_console_init() also switches the console to UTF-8
* (restored at exit), sets a UTF-8 C locale and asks Python children for
* UTF-8. Returns -1 when the process code page is not UTF-8 (pre-1903
* Windows); no-op returning 0 elsewhere. */
int os_console_init(void);
/* Child output that is not valid UTF-8, decoded from the console/OEM/ANSI
* code page: malloc'd UTF-8, or NULL when s is already UTF-8 (or not on
* Windows). */
char *os_output_to_utf8(const char *s, size_t n, size_t *out_len);
#ifdef _WIN32
#include <wchar.h>
/* malloc'd conversions; NULL on invalid input or allocation failure */
wchar_t *os_utf8_to_wide(const char *s);
char *os_wide_to_utf8(const wchar_t *w);
#endif
/* --- misc --- */
void os_localtime(const time_t *t, struct tm *out);
int os_getpid(void);
int os_chmod_private(const char *path); /* best-effort 0600; no-op/ok on Win */
/* 0600; on Windows a Medium no-read-up label, so Low-IL children (the
* shell sandbox) cannot read the file. */
int os_chmod_private(const char *path);
/* --- UTF-8 --------------------------------------------------------------
* Encode one code point as UTF-8, independent of the C locale.
*
* wcrtomb() follows setlocale(). On Linux the environment normally selects a
* UTF-8 locale, but on Windows the C runtime maps setlocale(LC_ALL, "") to the
* ANSI code page -- 936 (GBK) on a Chinese system -- so wcrtomb() hands back
* GBK bytes where the protocol requires UTF-8, and the model receives
* mojibake. Windows' C runtime has no ".UTF-8" locale to switch to.
*
* Returns the number of bytes written (1..4), or 0 if the code point cannot be
* encoded. Surrogate halves (as they arrive on 16-bit wchar_t platforms for
* non-BMP code points) return 0 and are skipped. */
int os_utf8_encode(unsigned long cp, char out[4]);
/* Decode one UTF-8 sequence from s (at most avail bytes).
* Returns the number of bytes consumed (1..4) and stores the code point in
* *cp, or 0 if the bytes are not a valid, complete, non-overlong sequence.
*
* The counterpart of the encoder above, and needed for the same reason:
* mbrtowc() also follows setlocale(), so on Windows it would decode UTF-8
* text as GBK and the screen would show mojibake. */
int os_utf8_decode(const char *s, size_t avail, unsigned long *cp);
/* --- libc gaps on Windows ------------------------------------------------
* MinGW-w64 provides neither strcasestr() (a GNU extension) nor
+23
View File
@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1"
xmlns:asmv3="urn:schemas-microsoft-com:asm.v3">
<assemblyIdentity type="win32" name="cst-agent" version="1.0.0.0"/>
<asmv3:application>
<asmv3:windowsSettings>
<activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
<longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
</asmv3:windowsSettings>
</asmv3:application>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="asInvoker" uiAccess="false"/>
</requestedPrivileges>
</security>
</trustInfo>
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
</application>
</compatibility>
</assembly>
+29
View File
@@ -0,0 +1,29 @@
#include <winver.h>
#include "cst_version.h"
1 24 "cst-agent.manifest"
VS_VERSION_INFO VERSIONINFO
FILEVERSION CST_VERSION_MAJOR, CST_VERSION_MINOR, CST_VERSION_PATCH, 0
PRODUCTVERSION CST_VERSION_MAJOR, CST_VERSION_MINOR, CST_VERSION_PATCH, 0
FILEOS VOS_NT_WINDOWS32
FILETYPE VFT_APP
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "040904B0"
BEGIN
VALUE "CompanyName", "WHC"
VALUE "FileDescription", "CST agent"
VALUE "FileVersion", CST_VERSION
VALUE "InternalName", "cst-agent"
VALUE "OriginalFilename", "cst-agent.exe"
VALUE "ProductName", "CST agent"
VALUE "ProductVersion", CST_VERSION
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x409, 1200
END
END
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
# Native Windows build from an MSYS2 UCRT64 shell (https://www.msys2.org/).
# scripts/bootstrap_msys2.sh [repo-dir]
# Installs the UCRT64 toolchain with pacman, builds the same pinned static
# deps as the Linux cross build (curl with Schannel, ncurses with the Windows
# console patches, busybox-w32 Unicode), then a static cst-agent.exe with the
# UTF-8 manifest, runs the tests and packages dist/windows-x86_64/.
# CST_SKIP_TESTS=1 skips ctest.
set -euo pipefail
if [[ "${MSYSTEM:-}" != UCRT64 ]]; then
echo "run this from the MSYS2 UCRT64 shell: the UTF-8 manifest needs the" \
"UCRT runtime (MINGW64 links msvcrt)" >&2
exit 1
fi
REPO_URL="${CST_AGENT_REPO:-https://git.whcself.xyz/WHC/CST-agent.git}"
DIR="${1:-}"
if [[ -z "$DIR" ]]; then
if [[ -f CMakeLists.txt && -d src ]]; then
DIR="$(pwd)"
else
DIR="CST-agent"
fi
fi
pacman -S --needed --noconfirm \
mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-cmake \
mingw-w64-ucrt-x86_64-ninja make git patch tar perl zip diffutils
if [[ ! -f "$DIR/CMakeLists.txt" ]]; then
echo "==> git clone $REPO_URL -> $DIR"
git clone "$REPO_URL" "$DIR"
fi
cd "$DIR"
# third-party script: unprefixed native tools
export MINGW_HOST= BUILD_CC=gcc
[[ -f third_party/prefix-mingw/lib/libcurl.a ]] || scripts/build_third_party_mingw.sh curl
[[ -f third_party/prefix-mingw/lib/libncursesw.a ]] || scripts/build_third_party_mingw.sh ncurses
if [[ ! -f third_party/busybox/busybox.exe ]]; then
scripts/build_third_party_mingw.sh busybox-w32 || scripts/fetch_busybox_w32.sh
fi
echo "==> cmake (MSYS2 UCRT64, static, TUI)"
cmake -S . -B build-msys2 -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_PREFIX_PATH="$PWD/third_party/prefix-mingw" \
-DCST_WIN_STATIC=ON -DCST_BUILD_TUI=ON
cmake --build build-msys2
if [[ -z "${CST_SKIP_TESTS:-}" ]]; then
echo "==> ctest"
cp -f third_party/busybox/busybox.exe build-msys2/
ctest --test-dir build-msys2 --output-on-failure
fi
echo "==> package"
CST_WIN_BUILD=build-msys2 scripts/package_windows.sh
+29 -25
View File
@@ -1,11 +1,14 @@
#!/usr/bin/env bash
# One-shot on a Linux host: cross-build Windows x86_64 release (MinGW).
# Output: dist/windows-x86_64/ — copy that folder to a Windows machine.
# Linux host: cross-build the Windows x86_64 release (mingw-w64, UCRT, static).
# scripts/bootstrap_windows.sh [repo-dir]
# With Nix the toolchain comes from shell-win.nix; otherwise $MINGW_HOST-gcc
# (default x86_64-w64-mingw32; UCRT flavour, with winpthreads), cmake, ninja,
# git and perl must be installed. CST_WIN_CFLAGS adds C flags (CI: -Werror).
# Output: dist/windows-x86_64/ and dist/cst-agent-VERSION-windows-x86_64.zip
set -euo pipefail
REPO_URL="${CST_AGENT_REPO:-https://git.whcself.xyz/WHC/CST-agent.git}"
DIR="${1:-}"
if [[ -z "$DIR" ]]; then
if [[ -f CMakeLists.txt && -d src ]]; then
DIR="$(pwd)"
@@ -13,36 +16,37 @@ if [[ -z "$DIR" ]]; then
DIR="CST-agent"
fi
fi
if [[ ! -f "$DIR/CMakeLists.txt" ]]; then
echo "==> git clone $REPO_URL -> $DIR"
git clone "$REPO_URL" "$DIR"
fi
cd "$DIR"
need() { command -v "$1" >/dev/null || { echo "missing $1 (install mingw-w64 gcc cmake git)" >&2; exit 1; }; }
need git; need cmake; need x86_64-w64-mingw32-gcc
H="${MINGW_HOST:-x86_64-w64-mingw32}"
if ! command -v "$H-gcc" >/dev/null && [[ -z "${IN_NIX_SHELL:-}" ]] &&
command -v nix-shell >/dev/null; then
exec nix-shell shell-win.nix --run "$(printf '%q ' "$0" "$PWD")"
fi
echo "==> third-party (curl Schannel + PDCurses + busybox-w32)"
chmod +x scripts/*.sh 2>/dev/null || true
./scripts/build_third_party_mingw.sh all
need() { command -v "$1" >/dev/null || { echo "missing $1 (mingw-w64 UCRT gcc, cmake, ninja, git, perl)" >&2; exit 1; }; }
export MINGW_HOST="$H"
need git; need cmake; need ninja; need perl; need "$H-gcc"
if ! printf '#include <stdio.h>\n' | "$H-gcc" -dM -E - | grep -qw _UCRT; then
echo "$H-gcc targets msvcrt; the UTF-8 manifest needs a UCRT toolchain" >&2
exit 1
fi
echo "==> cmake Windows TUI"
cmake -B build-win-tui \
[[ -f third_party/prefix-mingw/lib/libcurl.a ]] || scripts/build_third_party_mingw.sh curl
[[ -f third_party/prefix-mingw/lib/libncursesw.a ]] || scripts/build_third_party_mingw.sh ncurses
[[ -f third_party/busybox/busybox.exe ]] || scripts/build_third_party_mingw.sh busybox-w32
echo "==> cmake (Windows, TUI)"
cmake -S . -B build-win-tui -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/mingw-w64-x86_64.cmake \
-DCST_BUILD_TUI=ON \
-DCMAKE_PREFIX_PATH="$PWD/third_party/prefix-mingw" \
-DPDCurses_ROOT="$PWD/third_party/prefix-mingw"
cmake --build build-win-tui -j"$(nproc 2>/dev/null || echo 4)"
-DCMAKE_BUILD_TYPE=Release -DCST_BUILD_TUI=ON \
-DCMAKE_C_FLAGS="${CST_WIN_CFLAGS:-}"
# Package only needs the agent binary; skip host-unrunnable Win32 tests.
cmake --build build-win-tui --target cst-agent
echo "==> package"
./scripts/package_windows.sh
echo
echo "OK: dist/windows-x86_64/"
echo "Copy that folder to Windows, then:"
echo " set OPENAI_API_KEY=sk-..."
echo " set OPENAI_BASE_URL=https://api.openai.com/v1"
echo " set OPENAI_MODEL=gpt-4.1-mini"
echo " cst-agent.exe --workspace ."
echo "See docs/env_openai.md and docs/README-windows.md"
scripts/package_windows.sh
+164
View File
@@ -0,0 +1,164 @@
#!/usr/bin/env bash
# Build pinned static deps for a Linux release into third_party/prefix-linux-ARCH:
# mbedTLS, libcurl (HTTP(S) only, mbedTLS) and wide ncurses.
# scripts/build_third_party_linux.sh x86_64|x86|aarch64 [all|mbedtls|curl|ncurses]
# Uses LINUX_HOST-gcc from PATH, else fetches it via ci_linux_toolchain.sh.
# Host tools: cmake ninja make perl, and tic/infocmp with the fallback
# terminal entries (Debian: ncurses-bin ncurses-term).
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
ARCH="${1:?usage: $0 x86_64|x86|aarch64 [all|mbedtls|curl|ncurses]}"
TP="$ROOT/third_party"
PREFIX="$TP/prefix-linux-$ARCH"
SRC="$TP/src"
WORK="$TP/build-linux-$ARCH"
JOBS="${JOBS:-$(nproc 2>/dev/null || echo 4)}"
MBEDTLS_VER=3.6.7
MBEDTLS_SHA=a7e8bcbec0e6f761b4af24f25677626b35f762f68eef79c08677a363212d11f6
CURL_VER=8.22.0
CURL_SHA=f7ef3ae8a22e521f289803fe93543eb64c329b58aa73a9e224dfd915a2a5f4f7
NCURSES_VER=6.6
NCURSES_SHA=355b4cbbed880b0381a04c46617b7656e362585d52e9cf84a67e2009b749ff11
# Compiled in, so the TUI starts even where no terminfo database is found.
NCURSES_FALLBACKS=xterm-256color,xterm,screen-256color,tmux-256color,linux,vt100
if [[ -z "${LINUX_HOST:-}" ]] || ! command -v "$LINUX_HOST-gcc" >/dev/null; then
read -r LINUX_HOST bindir < <(bash "$ROOT/scripts/ci_linux_toolchain.sh" "$ARCH")
export PATH="$bindir:$PATH"
fi
T="$LINUX_HOST-"
need() { command -v "$1" >/dev/null || { echo "missing: $1" >&2; exit 1; }; }
need "${T}gcc"
need cmake
need ninja
need make
need tic
need infocmp
mkdir -p "$PREFIX" "$SRC" "$WORK"
fetch() { # file sha url...
local file="$1" sha="$2" u
shift 2
if [[ -f "$SRC/$file" ]] && echo "$sha $SRC/$file" | sha256sum -c --quiet - 2>/dev/null; then
return 0
fi
for u in "$@"; do
echo " try $u"
if curl -fsSL --retry 2 --connect-timeout 15 --max-time 600 -o "$SRC/$file.part" "$u" &&
echo "$sha $SRC/$file.part" | sha256sum -c --quiet -; then
mv "$SRC/$file.part" "$SRC/$file"
return 0
fi
done
rm -f "$SRC/$file.part"
echo "download failed: $file" >&2
exit 1
}
unpack() { # file dir
rm -rf "${WORK:?}/$2"
tar -C "$WORK" -xf "$SRC/$1"
}
gh() { # path -> GitHub URL candidates (CN proxies first)
printf '%s\n' "https://ghfast.top/https://github.com/$1" \
"https://ghproxy.net/https://github.com/$1" "https://github.com/$1"
}
cmake_cross() { # src build args...
local s="$1" b="$2"
shift 2
cmake -S "$s" -B "$b" -G Ninja \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_C_COMPILER="${T}gcc" -DCMAKE_AR="$(command -v "${T}ar")" \
-DCMAKE_RANLIB="$(command -v "${T}ranlib")" \
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_PREFIX_PATH="$PREFIX" \
-DCMAKE_FIND_ROOT_PATH="$PREFIX" \
-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY \
-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY \
-DCMAKE_POSITION_INDEPENDENT_CODE=OFF \
"$@"
cmake --build "$b" -j"$JOBS"
cmake --install "$b"
}
build_mbedtls() {
local f="mbedtls-$MBEDTLS_VER.tar.bz2"
mapfile -t urls < <(gh "Mbed-TLS/mbedtls/releases/download/mbedtls-$MBEDTLS_VER/$f")
fetch "$f" "$MBEDTLS_SHA" "${urls[@]}"
unpack "$f" "mbedtls-$MBEDTLS_VER"
cmake_cross "$WORK/mbedtls-$MBEDTLS_VER" "$WORK/mbedtls-$MBEDTLS_VER/build" \
-DENABLE_TESTING=OFF -DENABLE_PROGRAMS=OFF -DGEN_FILES=OFF \
-DUSE_STATIC_MBEDTLS_LIBRARY=ON -DUSE_SHARED_MBEDTLS_LIBRARY=OFF \
-DMBEDTLS_FATAL_WARNINGS=OFF
echo "mbedTLS $MBEDTLS_VER -> $PREFIX"
}
build_curl() {
local f="curl-$CURL_VER.tar.xz"
mapfile -t urls < <(gh "curl/curl/releases/download/curl-${CURL_VER//./_}/$f")
fetch "$f" "$CURL_SHA" "https://curl.se/download/$f" "${urls[@]}"
unpack "$f" "curl-$CURL_VER"
# Cross builds skip CA detection; this is only libcurl's default, the
# program probes the usual distro paths when it is missing.
cmake_cross "$WORK/curl-$CURL_VER" "$WORK/curl-$CURL_VER/build" \
-DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON \
-DCURL_USE_MBEDTLS=ON -DCURL_USE_OPENSSL=OFF \
-DMBEDTLS_INCLUDE_DIR="$PREFIX/include" \
-DMBEDTLS_LIBRARY="$PREFIX/lib/libmbedtls.a" \
-DMBEDX509_LIBRARY="$PREFIX/lib/libmbedx509.a" \
-DMBEDCRYPTO_LIBRARY="$PREFIX/lib/libmbedcrypto.a" \
-DCURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt -DCURL_CA_PATH=none \
-DHTTP_ONLY=ON -DCURL_DISABLE_LDAP=ON -DCURL_USE_LIBPSL=OFF \
-DCURL_USE_LIBSSH2=OFF -DCURL_USE_LIBSSH=OFF -DUSE_LIBIDN2=OFF \
-DUSE_NGHTTP2=OFF -DCURL_ZLIB=OFF -DCURL_BROTLI=OFF -DCURL_ZSTD=OFF \
-DCURL_USE_GSSAPI=OFF -DENABLE_THREADED_RESOLVER=ON \
-DBUILD_CURL_EXE=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF \
-DBUILD_LIBCURL_DOCS=OFF -DBUILD_MISC_DOCS=OFF -DENABLE_CURL_MANUAL=OFF
echo "curl $CURL_VER -> $PREFIX"
}
build_ncurses() {
local f="ncurses-$NCURSES_VER.tar.gz"
fetch "$f" "$NCURSES_SHA" \
"https://invisible-island.net/archives/ncurses/$f" \
"https://mirrors.tuna.tsinghua.edu.cn/gnu/ncurses/$f" \
"https://ftp.gnu.org/gnu/ncurses/$f"
unpack "$f" "ncurses-$NCURSES_VER"
local b="$WORK/ncurses-$NCURSES_VER/build"
mkdir -p "$b"
cd "$b"
CC="${T}gcc" AR="${T}ar" RANLIB="${T}ranlib" STRIP="${T}strip" \
../configure --host="$LINUX_HOST" --prefix="$PREFIX" \
--without-ada --without-cxx --without-cxx-binding --without-manpages \
--without-progs --without-tests --without-debug --without-shared \
--with-normal --enable-widec --disable-db-install \
--with-default-terminfo-dir=/usr/share/terminfo \
--with-terminfo-dirs=/etc/terminfo:/lib/terminfo:/usr/share/terminfo:/usr/lib/terminfo \
--with-fallbacks="$NCURSES_FALLBACKS" \
--with-tic-path="$(command -v tic)" --with-infocmp-path="$(command -v infocmp)" \
--enable-pc-files --with-pkg-config-libdir="$PREFIX/lib/pkgconfig" \
--with-build-cc="${BUILD_CC:-cc}" >/dev/null
make -j"$JOBS" libs >/dev/null
make install.libs >/dev/null
echo "ncurses $NCURSES_VER -> $PREFIX"
}
case "${2:-all}" in
mbedtls) build_mbedtls ;;
curl) build_curl ;;
ncurses) build_ncurses ;;
all)
build_mbedtls
build_curl
build_ncurses
;;
*)
echo "usage: $0 ARCH [all|mbedtls|curl|ncurses]" >&2
exit 1
;;
esac
echo "DONE $ARCH"
+127 -112
View File
@@ -1,139 +1,154 @@
#!/usr/bin/env bash
# Build third-party deps for Windows MinGW cross + optional Unix busybox.
# Sources:
# busybox-w32: https://github.com/rmyorston/busybox-w32.git
# busybox: https://github.com/mirror/busybox.git
# curl: https://github.com/curl/curl.git (Windows via Schannel)
# PDCurses: https://github.com/wmcbrine/PDCurses.git
# Build pinned third-party deps for Windows x86_64 (mingw-w64, UCRT) into
# third_party/prefix-mingw, plus busybox-w32 into third_party/busybox.
# nix-shell shell-win.nix --run 'scripts/build_third_party_mingw.sh [all|curl|pdcurses|ncurses|busybox-w32]'
# Works the same from an MSYS2 UCRT64 shell (MINGW_HOST= uses unprefixed tools).
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
TP="$ROOT/third_party"
HOST="${MINGW_HOST:-x86_64-w64-mingw32}"
PREFIX="$TP/prefix-mingw"
HOST="${MINGW_HOST-x86_64-w64-mingw32}"
T="${HOST:+$HOST-}"
JOBS="${JOBS:-$(nproc 2>/dev/null || echo 4)}"
mkdir -p "$TP"
CURL_TAG=curl-8_22_0
PDCURSES_TAG=3.9
NCURSES_VER=6.6
BUSYBOX_W32_REV=9b50a1e
mkdir -p "$TP" "$PREFIX"
need_cmd() { command -v "$1" >/dev/null || { echo "missing: $1" >&2; exit 1; }; }
need_cmd git
need_cmd make
need_cmd "${HOST}-gcc"
need_cmd "${T}gcc"
clone_or_update() {
local url="$1" dir="$2"
if [[ -d "$dir/.git" ]]; then
git -C "$dir" fetch --depth 1 origin 2>/dev/null || true
git -C "$dir" pull --ff-only 2>/dev/null || true
else
git clone --depth 1 "$url" "$dir"
fi
# Clone a GitHub repo with CN-friendly mirrors; fall back if one proxy 502s.
github_mirrors() { # https://github.com/org/repo[.git] -> print candidate clone URLs
local url="$1" path
path="${url#https://github.com/}"
path="${path%.git}"
printf '%s\n' \
"https://ghfast.top/https://github.com/${path}.git" \
"https://ghproxy.net/https://github.com/${path}.git" \
"https://gitclone.com/github.com/${path}.git" \
"https://github.com/${path}.git"
}
# --- curl (static, Schannel) ---
build_curl_mingw() {
clone_or_update https://github.com/curl/curl.git "$TP/curl"
cd "$TP/curl"
if [[ ! -f configure ]]; then
autoreconf -fi 2>/dev/null || true
checkout() { # url dir rev
local url="$1" dir="$2" rev="$3" m
if [[ ! -d "$dir/.git" ]]; then
local ok=0
while IFS= read -r m; do
echo " clone $m"
rm -rf "$dir"
if git clone -q --filter=blob:none "$m" "$dir"; then
ok=1
break
fi
done < <(github_mirrors "$url")
[[ $ok -eq 1 ]] || { echo "clone failed: $url" >&2; exit 1; }
fi
# Prefer cmake for curl modern tree
rm -rf build-mingw
cmake -B build-mingw -G "Unix Makefiles" \
-DCMAKE_SYSTEM_NAME=Windows \
-DCMAKE_C_COMPILER="${HOST}-gcc" \
-DCMAKE_RC_COMPILER="${HOST}-windres" \
-DCMAKE_INSTALL_PREFIX="$TP/prefix-mingw" \
-DBUILD_SHARED_LIBS=OFF \
-DCURL_USE_SCHANNEL=ON \
-DCURL_USE_OPENSSL=OFF \
-DCURL_DISABLE_LDAP=ON \
-DCURL_USE_LIBPSL=OFF \
-DCURL_ZLIB=OFF \
-DCURL_BROTLI=OFF \
-DCURL_ZSTD=OFF \
-DUSE_NGHTTP2=OFF \
-DBUILD_CURL_EXE=OFF \
-DBUILD_TESTING=OFF \
-DHTTP_ONLY=OFF
cmake --build build-mingw -j"$JOBS"
cmake --install build-mingw
echo "curl installed -> $TP/prefix-mingw"
if ! git -C "$dir" rev-parse -q --verify "$rev^{commit}" >/dev/null; then
git -C "$dir" fetch -q --tags origin || true
git -C "$dir" fetch -q origin "$rev" || true
fi
git -C "$dir" checkout -q --detach "$rev"
}
# --- PDCurses ---
build_pdcurses_mingw() {
clone_or_update https://github.com/wmcbrine/PDCurses.git "$TP/PDCurses"
build_curl() {
checkout https://github.com/curl/curl.git "$TP/curl" "$CURL_TAG"
rm -rf "$TP/curl/build-mingw"
local tc=()
[[ -n "$HOST" ]] && tc=(-DCMAKE_SYSTEM_NAME=Windows -DCMAKE_C_COMPILER="${T}gcc" -DCMAKE_RC_COMPILER="${T}windres")
cmake -S "$TP/curl" -B "$TP/curl/build-mingw" -G Ninja "${tc[@]}" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON \
-DCURL_USE_SCHANNEL=ON -DCURL_USE_OPENSSL=OFF \
-DCURL_WINDOWS_SSPI=ON -DENABLE_UNICODE=ON \
-DCURL_DISABLE_LDAP=ON -DCURL_USE_LIBPSL=OFF -DCURL_USE_LIBSSH2=OFF \
-DUSE_LIBIDN2=OFF -DUSE_NGHTTP2=OFF \
-DCURL_ZLIB=OFF -DCURL_BROTLI=OFF -DCURL_ZSTD=OFF \
-DBUILD_CURL_EXE=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF \
-DBUILD_LIBCURL_DOCS=OFF -DBUILD_MISC_DOCS=OFF -DENABLE_CURL_MANUAL=OFF
cmake --build "$TP/curl/build-mingw" -j"$JOBS"
cmake --install "$TP/curl/build-mingw"
echo "curl $CURL_TAG -> $PREFIX"
}
build_pdcurses() {
checkout https://github.com/wmcbrine/PDCurses.git "$TP/PDCurses" "$PDCURSES_TAG"
cd "$TP/PDCurses/wincon"
make -f Makefile clean 2>/dev/null || true
make -f Makefile \
CC="${HOST}-gcc" \
LINK="${HOST}-gcc" \
CFLAGS="-O2 -I.. -DPDC_WIDE -DPDC_FORCE_UTF8" \
LIBEXE="${HOST}-ar" \
LIBFLAGS=rcv \
pdcurses.a
mkdir -p "$TP/prefix-mingw/include" "$TP/prefix-mingw/lib"
cp -f ../curses.h "$TP/prefix-mingw/include/"
[[ -f ../panel.h ]] && cp -f ../panel.h "$TP/prefix-mingw/include/" || true
cp -f pdcurses.a "$TP/prefix-mingw/lib/libpdcurses.a"
# Also install as curses for find_library
cp -f pdcurses.a "$TP/prefix-mingw/lib/libcurses.a"
echo "PDCurses -> $TP/prefix-mingw"
make clean >/dev/null 2>&1 || true
make -j"$JOBS" CC="${T}gcc" AR="${T}ar" STRIP="${T}strip" WIDE=Y UTF8=Y pdcurses.a
mkdir -p "$PREFIX/include/pdcurses" "$PREFIX/lib"
cp -f ../curses.h ../panel.h "$PREFIX/include/pdcurses/"
cp -f pdcurses.a "$PREFIX/lib/libpdcurses.a"
echo "PDCurses $PDCURSES_TAG -> $PREFIX"
}
build_ncurses() {
local src="$TP/ncurses-$NCURSES_VER" tgz="$TP/ncurses-$NCURSES_VER.tar.gz"
[[ -f "$tgz" ]] || curl -fL -o "$tgz" \
"https://invisible-island.net/archives/ncurses/ncurses-$NCURSES_VER.tar.gz"
[[ -d "$src" ]] || tar -C "$TP" -xzf "$tgz"
local pf
for pf in "$ROOT"/scripts/patches/ncurses-"$NCURSES_VER"-*.patch; do
[[ -e "$pf" ]] || continue
patch -d "$src" -p1 -R --dry-run -s -f <"$pf" >/dev/null 2>&1 && continue
if ! patch -d "$src" -p1 --dry-run -s -f <"$pf" >/dev/null 2>&1; then
echo "ncurses: $src has another version of $(basename "$pf"); re-extracting"
rm -rf "$src"
tar -C "$TP" -xzf "$tgz"
fi
patch -d "$src" -p1 -s <"$pf"
done
rm -rf "$src/build-mingw"
mkdir -p "$src/build-mingw"
cd "$src/build-mingw"
# --host stays canonical (config.sub may not know Debian's "mingw32ucrt");
# the tools themselves come from $HOST.
CC="${T}gcc" AR="${T}ar" RANLIB="${T}ranlib" STRIP="${T}strip" WINDRES="${T}windres" \
../configure --host=x86_64-w64-mingw32 --prefix="$PREFIX" \
--without-ada --without-cxx --without-cxx-binding --without-manpages \
--without-progs --without-tests --without-debug --without-shared \
--enable-widec --enable-sp-funcs --enable-term-driver \
--enable-interop --disable-home-terminfo --disable-db-install \
--with-fallbacks=ms-terminal,xterm-256color --without-pkg-config \
--disable-pc-files --with-build-cc="${BUILD_CC:-cc}" >/dev/null
make -j"$JOBS" libs >/dev/null
make install.libs >/dev/null
echo "ncurses $NCURSES_VER -> $PREFIX"
}
# --- busybox-w32 ---
build_busybox_w32() {
clone_or_update https://github.com/rmyorston/busybox-w32.git "$TP/busybox-w32"
checkout https://github.com/rmyorston/busybox-w32.git "$TP/busybox-w32" "$BUSYBOX_W32_REV"
cd "$TP/busybox-w32"
# configs: try mingw64 defconfig
if [[ -f configs/mingw64_defconfig ]]; then
make mingw64_defconfig
elif [[ -f configs/mingw_defconfig ]]; then
make mingw_defconfig
else
make defconfig
fi
# Cross compile
make -j"$JOBS" CROSS_COMPILE="${HOST}-" \
CONFIG_EXTRA_CFLAGS="-O2" \
busybox.exe 2>&1 || make -j"$JOBS" CROSS_COMPILE="${HOST}-"
make -s mingw64u_defconfig >/dev/null
make -s -j"$JOBS" CROSS_COMPILE="$T" </dev/null
mkdir -p "$TP/busybox"
if [[ -f busybox.exe ]]; then
cp -f busybox.exe "$TP/busybox/busybox.exe"
else
find . -name 'busybox.exe' | head -1 | xargs -I{} cp -f {} "$TP/busybox/busybox.exe"
fi
cat > "$TP/busybox/LICENSE.busybox" << 'EOF'
BusyBox is GPLv2. Source: https://github.com/rmyorston/busybox-w32
Upstream BusyBox: https://github.com/mirror/busybox
cp -f busybox.exe "$TP/busybox/busybox.exe"
cat >"$TP/busybox/LICENSE.busybox" <<EOF
BusyBox is GPLv2. This busybox.exe is busybox-w32 ($BUSYBOX_W32_REV, Unicode build):
https://github.com/rmyorston/busybox-w32
Upstream BusyBox: https://busybox.net/
EOF
ls -la "$TP/busybox/busybox.exe"
}
# --- Unix busybox (optional, for Linux/Termux side-by-side bundle) ---
build_busybox_unix() {
clone_or_update https://github.com/mirror/busybox.git "$TP/busybox-src"
cd "$TP/busybox-src"
make defconfig
# Keep applets reasonable; enable ash + diff
make -j"$JOBS" busybox
mkdir -p "$TP/busybox"
cp -f busybox "$TP/busybox/busybox"
echo "unix busybox -> $TP/busybox/busybox"
}
TARGETS="${1:-all}"
case "$TARGETS" in
curl) build_curl_mingw ;;
pdcurses) build_pdcurses_mingw ;;
busybox-w32) build_busybox_w32 ;;
busybox-unix) build_busybox_unix ;;
all)
build_curl_mingw
build_pdcurses_mingw
build_busybox_w32
;;
*)
echo "usage: $0 [all|curl|pdcurses|busybox-w32|busybox-unix]"
exit 1
;;
case "${1:-all}" in
curl) build_curl ;;
pdcurses) build_pdcurses ;;
ncurses) build_ncurses ;;
busybox-w32) build_busybox_w32 ;;
all)
build_curl
build_ncurses
build_busybox_w32
;;
*)
echo "usage: $0 [all|curl|pdcurses|ncurses|busybox-w32]"
exit 1
;;
esac
echo "DONE"
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
# Checkout this repo without downloading actions/checkout from GitHub.
# act_runner may already have placed sources in the workspace; otherwise
# clone from the same Gitea that started the job.
set -euo pipefail
SERVER="${GITHUB_SERVER_URL:-${GITEA_SERVER_URL:-}}"
REPO="${GITHUB_REPOSITORY:-${GITEA_REPOSITORY:-}}"
SHA="${GITHUB_SHA:-${GITEA_SHA:-}}"
if [[ -f CMakeLists.txt && -d src ]]; then
echo "==> workspace already has sources"
git rev-parse --short HEAD 2>/dev/null || true
exit 0
fi
if [[ -z "$SERVER" || -z "$REPO" ]]; then
echo "missing GITHUB_SERVER_URL / GITHUB_REPOSITORY" >&2
exit 1
fi
echo "==> git clone $SERVER/$REPO"
git clone --filter=blob:none "$SERVER/$REPO.git" _ci_src
if [[ -n "$SHA" ]]; then
git -C _ci_src fetch --depth 1 origin "$SHA" || git -C _ci_src fetch origin "$SHA"
git -C _ci_src checkout --force "$SHA"
fi
# Move into the workspace root (may already contain act_runner metadata).
shopt -s dotglob nullglob
mv _ci_src/* .
rmdir _ci_src 2>/dev/null || rm -rf _ci_src
test -f CMakeLists.txt
git rev-parse --short HEAD
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env bash
# Fetch a musl cross gcc for a static Linux release build.
# scripts/ci_linux_toolchain.sh x86_64|x86|aarch64
# Prints "HOST BINDIR" on stdout (e.g. "x86_64-buildroot-linux-musl
# /opt/x86-64--musl--stable-2025.08-1/bin"); progress goes to stderr.
# Bootlin's pinned stable release first, musl.cc as a fallback.
set -euo pipefail
ARCH="${1:?usage: $0 x86_64|x86|aarch64}"
BOOTLIN_VER="${CST_BOOTLIN_VER:-2025.08-1}"
case "$ARCH" in
x86_64)
bl=x86-64 bl_host=x86_64-buildroot-linux-musl mc=x86_64-linux-musl
sha=09fca3aa89540f1b01b5f4210d488cbeb00f522044c53e9989b1dd8a38076912
;;
x86)
bl=x86-i686 bl_host=i686-buildroot-linux-musl mc=i686-linux-musl
sha=3a340ebc386a057f5ddb24d8f4bbde07d2c013c99cebc02ac38565e2849bfda0
;;
aarch64)
bl=aarch64 bl_host=aarch64-buildroot-linux-musl mc=aarch64-linux-musl
sha=defba831ffa1175236f137069333e21ed46d4d19feb5080a90cf248b6fc2cb08
;;
*)
echo "unknown arch: $ARCH" >&2
exit 1
;;
esac
[[ "$BOOTLIN_VER" == 2025.08-1 ]] || sha="" # other releases: no pinned hash
if [[ -n "${CST_TOOLCHAIN_ROOT:-}" ]]; then
ROOT_DIR="$CST_TOOLCHAIN_ROOT"
elif [[ -w /opt || "$(id -u)" -eq 0 ]]; then
ROOT_DIR=/opt
else
ROOT_DIR="$HOME/.local/toolchains"
fi
mkdir -p "$ROOT_DIR"
fetch() { # out url...
local out="$1" u
shift
for u in "$@"; do
echo " try $u" >&2
if curl -fsSL --retry 2 --connect-timeout 15 --max-time 900 -o "$out" "$u"; then
return 0
fi
done
return 1
}
emit() { # host bindir
"$2/$1-gcc" -dumpmachine >&2
[[ -n "${GITHUB_PATH:-}" ]] && echo "$2" >>"$GITHUB_PATH"
echo "$1 $2"
}
name="$bl--musl--stable-$BOOTLIN_VER"
dir="$ROOT_DIR/$name"
if [[ -x "$dir/bin/$bl_host-gcc" ]]; then
emit "$bl_host" "$dir/bin"
exit 0
fi
echo "==> toolchain $name" >&2
tmp="$ROOT_DIR/$name.tar.xz"
if fetch "$tmp" "https://toolchains.bootlin.com/downloads/releases/toolchains/$bl/tarballs/$name.tar.xz" &&
{ [[ -z "$sha" ]] || echo "$sha $tmp" | sha256sum -c --quiet - >&2; }; then
tar -C "$ROOT_DIR" -xJf "$tmp"
rm -f "$tmp"
# Bootlin toolchains are relocatable but must be told once where they are.
[[ -x "$dir/relocate-sdk.sh" ]] && "$dir/relocate-sdk.sh" >&2
emit "$bl_host" "$dir/bin"
exit 0
fi
rm -f "$tmp"
echo "==> bootlin unavailable, falling back to musl.cc" >&2
mdir="$ROOT_DIR/$mc-cross"
if [[ ! -x "$mdir/bin/$mc-gcc" ]]; then
tgz="$ROOT_DIR/$mc-cross.tgz"
fetch "$tgz" "https://musl.cc/$mc-cross.tgz" "https://more.musl.cc/x86_64-linux-musl/$mc-cross.tgz" ||
{ echo "no musl toolchain for $ARCH" >&2; exit 1; }
tar -C "$ROOT_DIR" -xzf "$tgz"
rm -f "$tgz"
fi
emit "$mc" "$mdir/bin"
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Point package managers at a China mirror before apt/apk install.
# Gitea runners in CN are far too slow against deb.debian.org /
# archive.ubuntu.com / dl-cdn.alpinelinux.org.
#
# CST_CI_MIRROR=https://mirrors.tuna.tsinghua.edu.cn scripts/ci_pkg_mirror.sh
# Default mirror: Tsinghua TUNA. Override with CST_CI_MIRROR if needed.
set -euo pipefail
MIRROR="${CST_CI_MIRROR:-https://mirrors.tuna.tsinghua.edu.cn}"
MIRROR="${MIRROR%/}"
if [[ -f /etc/alpine-release ]]; then
echo "==> apk mirror -> $MIRROR/alpine"
if [[ -f /etc/apk/repositories ]]; then
sed -i \
-e "s|https\\?://dl-cdn.alpinelinux.org/alpine|${MIRROR}/alpine|g" \
-e "s|https\\?://[^/[:space:]]*/alpine|${MIRROR}/alpine|g" \
/etc/apk/repositories
fi
exit 0
fi
if ! command -v apt-get >/dev/null; then
echo "==> no apt/apk; skip mirror"
exit 0
fi
# Debian vs Ubuntu: different path under the mirror root.
id_like=""
. /etc/os-release 2>/dev/null || true
case "${ID:-}:${ID_LIKE:-}" in
ubuntu:*|*ubuntu*)
dist=ubuntu
;;
debian:*|*debian*|*)
dist=debian
;;
esac
echo "==> apt mirror -> $MIRROR/$dist (and ${dist}-security)"
rewrite() {
local f="$1"
[[ -f "$f" ]] || return 0
# Classic one-line sources
sed -i \
-e "s|https\\?://deb.debian.org/debian-security|${MIRROR}/debian-security|g" \
-e "s|https\\?://deb.debian.org/debian|${MIRROR}/debian|g" \
-e "s|https\\?://security.debian.org/debian-security|${MIRROR}/debian-security|g" \
-e "s|https\\?://security.debian.org|${MIRROR}/debian-security|g" \
-e "s|https\\?://archive.ubuntu.com/ubuntu|${MIRROR}/ubuntu|g" \
-e "s|https\\?://security.ubuntu.com/ubuntu|${MIRROR}/ubuntu|g" \
-e "s|https\\?://ports.ubuntu.com/ubuntu-ports|${MIRROR}/ubuntu-ports|g" \
"$f"
# DEB822 (.sources): URIs: http://deb.debian.org/debian
sed -i \
-e "s|^URIs: *https\\?://deb.debian.org/debian-security|URIs: ${MIRROR}/debian-security|g" \
-e "s|^URIs: *https\\?://deb.debian.org/debian|URIs: ${MIRROR}/debian|g" \
-e "s|^URIs: *https\\?://security.debian.org/debian-security|URIs: ${MIRROR}/debian-security|g" \
-e "s|^URIs: *https\\?://security.debian.org|URIs: ${MIRROR}/debian-security|g" \
-e "s|^URIs: *https\\?://archive.ubuntu.com/ubuntu|URIs: ${MIRROR}/ubuntu|g" \
-e "s|^URIs: *https\\?://security.ubuntu.com/ubuntu|URIs: ${MIRROR}/ubuntu|g" \
"$f"
}
rewrite /etc/apt/sources.list
shopt -s nullglob
for f in /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources; do
rewrite "$f"
done
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
# CI helper: build the pinned TinyCC used for the verification job.
# Distro tcc packages are too old (no <stdatomic.h>), so this builds the same
# mob snapshot nixpkgs ships. Installs to $CST_TCC_PREFIX (default ~/.local/tcc)
# and, under Actions, puts its bin/ on $GITHUB_PATH.
set -euo pipefail
COMMIT="${CST_TCC_COMMIT:-f6385c05308f715bdd2c06336801193a21d69b50}"
PREFIX="${CST_TCC_PREFIX:-$HOME/.local/tcc}"
WORK="${TMPDIR:-/tmp}/tcc-build"
if [[ -x "$PREFIX/bin/tcc" ]]; then
echo "==> tcc already at $PREFIX"
else
rm -rf "$WORK"
mkdir -p "$WORK"
TGZ="$WORK/tinycc.tar.gz"
# repo.or.cz is upstream but often unreachable from CN runners; the GitHub
# mirror carries the same commits.
urls=(
"https://ghfast.top/https://github.com/TinyCC/tinycc/archive/${COMMIT}.tar.gz"
"https://ghproxy.net/https://github.com/TinyCC/tinycc/archive/${COMMIT}.tar.gz"
"https://github.com/TinyCC/tinycc/archive/${COMMIT}.tar.gz"
"https://repo.or.cz/tinycc.git/snapshot/${COMMIT}.tar.gz"
)
ok=0
for u in "${urls[@]}"; do
echo " try $u"
if curl -fsSL --retry 2 --connect-timeout 15 --max-time 300 -o "$TGZ" "$u" &&
tar -tzf "$TGZ" 2>/dev/null | sed -n 1p | grep -q "${COMMIT:0:7}"; then
ok=1
break
fi
done
[[ $ok -eq 1 ]] || { echo "failed to download tinycc $COMMIT" >&2; exit 1; }
tar -C "$WORK" -xzf "$TGZ"
src=$(find "$WORK" -mindepth 1 -maxdepth 1 -type d -name 'tinycc*' | head -1)
cd "$src"
./configure --prefix="$PREFIX" --cc="${BUILD_CC:-cc}"
make -j"$(nproc)"
make install
fi
"$PREFIX/bin/tcc" -v
if [[ -n "${GITHUB_PATH:-}" ]]; then
echo "$PREFIX/bin" >>"$GITHUB_PATH"
fi
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# CI helper: Windows x86_64 release with Debian's mingw-w64 UCRT gcc.
# apt-get install gcc-mingw-w64-ucrt64 binutils-mingw-w64-ucrt64 nsis (trixie+)
# Checks the toolchain, then runs bootstrap_windows.sh, which builds the
# third-party libs, cst-agent.exe, the zip and the NSIS installer.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
export MINGW_HOST="${MINGW_HOST:-x86_64-w64-mingw32ucrt}"
export BUILD_CC="${BUILD_CC:-cc}"
export CST_REQUIRE_NSIS=1
need() { command -v "$1" >/dev/null || { echo "missing $1" >&2; exit 1; }; }
need "$MINGW_HOST-gcc"
need "$MINGW_HOST-windres"
need cmake; need ninja; need perl; need zip; need git; need make; need makensis
if ! printf '#include <stdio.h>\n' | "$MINGW_HOST-gcc" -dM -E - | grep -qw _UCRT; then
echo "$MINGW_HOST-gcc is not UCRT" >&2
exit 1
fi
"$MINGW_HOST-gcc" --version | head -1
# Do not set a global insteadOf: a single broken proxy (e.g. gitclone 502)
# would take down every third_party clone. build_third_party_mingw.sh tries
# several mirrors per repo instead.
echo "==> bootstrap_windows.sh"
exec bash scripts/bootstrap_windows.sh "$ROOT"
+299
View File
@@ -0,0 +1,299 @@
#!/bin/sh
# End-to-end check of the WebUI without a real model: a recorded SSE stream
# (tests/fixtures/sse) drives a full turn, and the events the browser would
# see are read back from /api/events with curl.
#
# scripts/e2e_webui.sh [replay-dir] [protocol]
#
# Needs: curl. python3 only for the JSON field checks (they are skipped
# without it). Exit code 0 = every assertion passed.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
REPLAY="${1:-$ROOT/tests/fixtures/sse/deepseek-native-eos}"
PROTO="${2:-native-eos}"
BIN="${CST_AGENT:-$ROOT/cst-agent}"
[ -x "$BIN" ] || { echo "build cst-agent first ($BIN)"; exit 1; }
WORK="$(mktemp -d "${TMPDIR:-/tmp}/cst-webui-XXXXXX")" 2>/dev/null || {
WORK="$ROOT/.build-tmp/e2e-webui"; rm -rf "$WORK"; mkdir -p "$WORK"; }
EVENTS="$WORK/events.txt"
LOG="$WORK/server.log"
cleanup() {
[ -n "${SRV_PID:-}" ] && kill "$SRV_PID" 2>/dev/null || true
rm -rf "$WORK"
}
trap cleanup EXIT INT TERM
echo "== start cst-agent --web (replay $REPLAY)"
# --yolo: the recorded stream calls write tools, and a permission question
# would wait for a browser nobody is driving here.
CST_REPLAY_SSE="$REPLAY" \
CST_REPLAY_CHUNK=512 \
"$BIN" --web --web-port 0 --workspace "$WORK" --new --api-key sk-e2e \
--protocol "$PROTO" --tools default --yolo > "$LOG" 2>&1 &
SRV_PID=$!
# The port is picked by the OS; the server prints the URL it bound.
i=0
PORT=""
while [ $i -lt 50 ]; do
PORT="$(sed -n 's#^webui: http://127.0.0.1:\([0-9]*\)/.*#\1#p' "$LOG" | head -1)"
[ -n "$PORT" ] && break
i=$((i + 1))
sleep 0.2
done
[ -n "$PORT" ] || { echo "FAIL: the server did not report a port"; cat "$LOG"; exit 1; }
BASE="http://127.0.0.1:$PORT"
echo " port $PORT"
fail() { echo "FAIL: $1"; echo "--- events"; cat "$EVENTS" 2>/dev/null | head -40; exit 1; }
# SIGTERM must shut the server down cleanly, even mid-turn (128+ = a crash)
stop_server() {
kill "$1" 2>/dev/null || return 0
n=0
while kill -0 "$1" 2>/dev/null && [ $n -lt 200 ]; do n=$((n + 1)); sleep 0.2; done
if kill -0 "$1" 2>/dev/null; then
kill -9 "$1" 2>/dev/null || true
fail "$2 hung on SIGTERM"
fi
st=0
wait "$1" 2>/dev/null || st=$?
[ "$st" -lt 128 ] || fail "$2 died with status $st on SIGTERM"
}
# 1. the shell page and the state endpoint
curl -fsS --max-time 5 "$BASE/" | grep -q "<title>CST-Agent" || fail "GET / is not the shell page"
curl -fsS --max-time 5 "$BASE/api/state" > "$WORK/state.json" || fail "GET /api/state"
grep -q '"protocol":"cst-webui/1"' "$WORK/state.json" || fail "state has no protocol tag"
grep -q '"session":' "$WORK/state.json" || fail "state has no session id"
# 2. an unknown route is a 404, an oversized body a 413
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "$BASE/api/does-not-exist")"
[ "$code" = "404" ] || fail "unknown route returned $code"
code="$(head -c 2000000 /dev/zero | tr '\0' 'a' |
curl -s -o /dev/null -w '%{http_code}' --max-time 20 \
-X POST --data-binary @- "$BASE/api/turn")"
[ "$code" = "413" ] || fail "2 MB body returned $code (want 413)"
# 3. stream the events while a recorded turn runs
curl -sN --max-time 120 "$BASE/api/events" > "$EVENTS" &
CURL_PID=$!
sleep 0.5
code="$(curl -s -o "$WORK/turn.json" -w '%{http_code}' --max-time 5 \
-X POST -H 'Content-Type: application/json' \
--data '{"text":"e2e: what is in the workspace?"}' "$BASE/api/turn")"
[ "$code" = "202" ] || fail "POST /api/turn returned $code"
i=0
while [ $i -lt 450 ]; do
grep -q "event: turn_end" "$EVENTS" && break
i=$((i + 1))
sleep 0.2
done
kill "$CURL_PID" 2>/dev/null || true
grep -q "event: turn_end" "$EVENTS" || fail "the turn never finished"
grep -q "event: assistant_html" "$EVENTS" || fail "no assistant markdown was streamed"
grep -q "event: tool" "$EVENTS" || fail "no tool event was streamed"
grep -q '"rc":0' "$EVENTS" || fail "the turn did not end with rc=0"
# 4. the session on disk matches what the UI would draw
curl -fsS --max-time 5 "$BASE/api/session" > "$WORK/session.json" || fail "GET /api/session"
grep -q '"role":"assistant"' "$WORK/session.json" || fail "no assistant message stored"
curl -fsS --max-time 5 "$BASE/api/sessions" | grep -q '"sessions":' || fail "GET /api/sessions"
# 5. a second turn while the first is running is refused with 409
curl -s -o /dev/null -w '' --max-time 5 -X POST -H 'Content-Type: application/json' \
--data '{"text":"x"}' "$BASE/api/turn" || true
# 6. asking to stop is accepted
curl -fsS --max-time 5 -X POST -H 'Content-Type: application/json' \
--data '{}' "$BASE/api/cancel" > /dev/null || fail "POST /api/cancel"
# 6b. the write endpoints: config, undo, compact and image upload
curl -fsS --max-time 5 -X POST -H 'Content-Type: application/json' \
--data '{"approval":"ask","plan_mode":false}' "$BASE/api/config" |
grep -q '"ok":true' || fail "POST /api/config"
curl -fsS --max-time 5 "$BASE/api/state" | grep -q '"approval":"ask"' ||
fail "the config change is not in the state"
code="$(curl -s -o "$WORK/undo.txt" -w '%{http_code}' --max-time 5 \
-X POST "$BASE/api/undo")"
case "$code" in
200) grep -q . "$WORK/undo.txt" || fail "an undo without a report" ;;
409) ;; # nothing to undo in this run
*) fail "POST /api/undo returned $code" ;;
esac
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 30 \
-X POST "$BASE/api/compact")"
# The recorded stream is spent by now, so a model failure is a valid answer.
case "$code" in 200 | 500) ;; *) fail "POST /api/compact returned $code" ;; esac
if command -v base64 > /dev/null; then
# a real 1x1 PNG: the server sniffs the content, not the extension
# (kept with a .bin suffix so the find below only sees the stored copy)
echo 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFAAH/q842iQAAAABJRU5ErkJggg==' |
base64 -d > "$WORK/tiny.bin" 2> /dev/null || : > "$WORK/tiny.bin"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 -X POST \
--data-binary @"$WORK/tiny.bin" "$BASE/api/attach?name=e2e.png")"
[ "$code" = "200" ] || fail "POST /api/attach returned $code"
curl -fsS --max-time 5 "$BASE/api/state" |
grep -q '"pending_images":\["' || fail "the upload is not pending"
# the stored name is the content hash, so ask the disk for it
STORED="$(find "$WORK" -name '*.png' -type f 2>/dev/null | head -1)"
[ -n "$STORED" ] || fail "the upload did not reach the session directory"
REL="${STORED##*/}"
curl -fsS --max-time 5 "$BASE/api/attach/$REL" > "$WORK/back.png" ||
fail "GET /api/attach/$REL"
cmp -s "$WORK/tiny.bin" "$WORK/back.png" ||
fail "the image did not round trip byte for byte"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
"$BASE/api/attach/..%2f..%2fetc%2fpasswd")"
case "$code" in 400 | 404) ;; *) fail "a traversal in attach returned $code" ;; esac
curl -fsS --max-time 5 -X DELETE "$BASE/api/attach" | grep -q '"ok":true' ||
fail "DELETE /api/attach"
curl -fsS --max-time 5 "$BASE/api/state" |
grep -q '"pending_images":\[\]' || fail "the pending image was not cleared"
else
echo " (no base64 tool: the image upload check is skipped)"
fi
# 6c. a request with a foreign Host header is refused while no token is set:
# that is the DNS-rebinding guard.
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
-H 'Host: evil.example' "$BASE/api/state")"
[ "$code" = "401" ] || fail "a foreign Host header returned $code (want 401)"
if command -v python3 > /dev/null; then
python3 - "$WORK/session.json" <<'PY' || fail "the session JSON is not valid JSON / is empty"
import json, sys
d = json.load(open(sys.argv[1]))
assert isinstance(d.get("messages"), list) and d["messages"], d
PY
fi
# 7. the interactive approval path: a fresh server in ask mode must raise a
# permission event, and answering it must let the turn continue.
echo "== permission round trip (-A ask)"
stop_server "$SRV_PID" "the first server" # its replay is spent
rm -rf "$WORK/pw"; mkdir -p "$WORK/pw"
P_EVENTS="$WORK/permission-events.txt"
P_LOG="$WORK/permission-server.log" # its own log: the port parse must not
# pick up the first server's URL
CST_REPLAY_SSE="$REPLAY" CST_REPLAY_CHUNK=512 \
"$BIN" --web --web-port 0 --workspace "$WORK/pw" --new --api-key sk-e2e \
--protocol "$PROTO" --tools default --approval ask --confirm all \
> "$P_LOG" 2>&1 &
P_PID=$!
i=0
P_PORT=""
while [ $i -lt 50 ]; do
P_PORT="$(sed -n 's#^webui: http://127.0.0.1:\([0-9]*\)/.*#\1#p' "$P_LOG" | tail -1)"
[ -n "$P_PORT" ] && break
i=$((i + 1)); sleep 0.2
done
[ -n "$P_PORT" ] || fail "the ask-mode server did not start (see $P_LOG)"
P_BASE="http://127.0.0.1:$P_PORT"
curl -sN --max-time 60 "$P_BASE/api/events" > "$P_EVENTS" &
P_CURL=$!
sleep 0.5
curl -s -o /dev/null --max-time 5 -X POST -H 'Content-Type: application/json' \
--data '{"text":"e2e: ask me first"}' "$P_BASE/api/turn"
i=0
while [ $i -lt 300 ]; do
grep -q "event: permission" "$P_EVENTS" && break
i=$((i + 1)); sleep 0.2
done
if [ $i -ge 300 ]; then
kill "$P_PID" 2>/dev/null || true
kill "$P_CURL" 2>/dev/null || true
echo " (no permission question within 60 s - skipped; $(grep -c '^event: ' "$P_EVENTS") events seen)"
else
PID_REQ="$(sed -n 's/.*"id":\([0-9]*\),"tool".*/\1/p' "$P_EVENTS" | head -1)"
[ -n "$PID_REQ" ] || fail "a permission event without an id"
# Answer while the server is still waiting for it.
curl -fsS --max-time 5 -X POST -H 'Content-Type: application/json' \
--data "{\"id\":$PID_REQ,\"answer\":1}" "$P_BASE/api/permission" > /dev/null ||
fail "POST /api/permission was refused"
i=0
while [ $i -lt 150 ]; do
grep -q "event: permission_done" "$P_EVENTS" && break
i=$((i + 1)); sleep 0.2
done
grep -q "event: permission_done" "$P_EVENTS" || fail "the answer was not acknowledged"
stop_server "$P_PID" "the ask-mode server"
kill "$P_CURL" 2>/dev/null || true
echo " answered request $PID_REQ with 'allow once'"
fi
# 8. token auth, --web-dev, and a clean shutdown
echo "== token auth and --web-dev"
T_LOG="$WORK/token-server.log"
CST_REPLAY_SSE="$REPLAY" \
"$BIN" --web --web-port 0 --workspace "$WORK/pw" --new --api-key sk-e2e \
--protocol "$PROTO" --tools default --yolo --web-dev "$ROOT/web" \
--web-token tok-e2e > "$T_LOG" 2>&1 &
T_PID=$!
i=0
T_PORT=""
while [ $i -lt 50 ]; do
T_PORT="$(sed -n 's#^webui: http://127.0.0.1:\([0-9]*\)/.*#\1#p' "$T_LOG" | tail -1)"
[ -n "$T_PORT" ] && break
i=$((i + 1)); sleep 0.2
done
[ -n "$T_PORT" ] || fail "the token server did not start (see $T_LOG)"
T_BASE="http://127.0.0.1:$T_PORT"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "$T_BASE/api/state")"
[ "$code" = "401" ] || fail "a token-protected server answered $code without one"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
-H 'Authorization: Bearer tok-e2e' "$T_BASE/api/state")"
[ "$code" = "200" ] || fail "the Bearer token was rejected ($code)"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
"$T_BASE/api/state?token=tok-e2e")"
[ "$code" = "200" ] || fail "the query token was rejected ($code)"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
-H 'Cookie: a=b; cst_token=tok-e2e; z=y' "$T_BASE/api/state")"
[ "$code" = "200" ] || fail "the cookie token was rejected ($code)"
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 \
-H 'Authorization: Bearer tok-e2' "$T_BASE/api/state")"
[ "$code" = "401" ] || fail "a wrong token was accepted ($code)"
# --web-dev serves straight from the directory, so the bytes must match it
curl -fsS --max-time 5 -H 'Authorization: Bearer tok-e2e' "$T_BASE/app.js" \
> "$WORK/app.js" || fail "GET /app.js from --web-dev"
cmp -s "$WORK/app.js" "$ROOT/web/app.js" ||
fail "--web-dev did not serve web/app.js from disk"
# POST /api/quit must return and then take the server down on its own
curl -fsS --max-time 5 -X POST -H 'Authorization: Bearer tok-e2e' \
"$T_BASE/api/quit" | grep -q '"ok":true' || fail "POST /api/quit"
i=0
while [ $i -lt 50 ] && kill -0 "$T_PID" 2> /dev/null; do
i=$((i + 1)); sleep 0.2
done
kill -0 "$T_PID" 2> /dev/null && fail "the server ignored /api/quit"
wait "$T_PID" 2>/dev/null || true
# 9. ask_user waits for the answer, and the HTTP hardening checks (both
# need a scripted model: python3)
if command -v python3 > /dev/null; then
echo "== ask_user round trip"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_ask.py" > "$WORK/ask.txt" 2>&1 ||
{ cat "$WORK/ask.txt"; fail "the ask_user round trip"; }
echo "== HTTP hardening"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_http.py" > "$WORK/http.txt" 2>&1 ||
{ cat "$WORK/http.txt"; fail "the HTTP hardening checks"; }
echo "== settings and streaming"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_settings.py" > "$WORK/settings.txt" 2>&1 ||
{ cat "$WORK/settings.txt"; fail "the settings checks"; }
echo "== sessions, retry and export"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_sessions.py" > "$WORK/sessions.txt" 2>&1 ||
{ cat "$WORK/sessions.txt"; fail "the session checks"; }
echo "== diff, run settings and permission rules"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_workspace.py" > "$WORK/workspace.txt" 2>&1 ||
{ cat "$WORK/workspace.txt"; fail "the workspace checks"; }
echo "== /init, /skill, extensions and the thinking switch"
CST_AGENT="$BIN" python3 "$ROOT/scripts/e2e_webui_ext.py" > "$WORK/ext.txt" 2>&1 ||
{ cat "$WORK/ext.txt"; fail "the extension checks"; }
fi
echo "== OK: (events: $(grep -c '^event: ' "$EVENTS"))"
+164
View File
@@ -0,0 +1,164 @@
"""WebUI ask_user round trip against a fake OpenAI-compatible API.
python3 scripts/e2e_webui_ask.py (CST_AGENT= picks the binary)
Checks: the turn really waits for the answer, a page opened while the
question is open still receives it, and Last-Event-ID resumes the stream.
"""
import json, os, shutil, subprocess, sys, tempfile, threading, time, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
REQS = []
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def do_GET(self):
body = json.dumps({"data": [{"id": "deepseek-flash"}]}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
n = int(self.headers.get("Content-Length", 0))
REQS.append(json.loads(self.rfile.read(n)))
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
base = {"id": "x", "object": "chat.completion.chunk", "model": "deepseek-flash"}
def ev(o):
self.wfile.write(b"data: " + json.dumps({**base, **o}).encode() + b"\n\n")
self.wfile.flush()
if len(REQS) == 1:
args = json.dumps({"question": "Pick a colour?", "options": "red\nblue"})
ev({"choices": [{"index": 0, "delta": {"role": "assistant", "tool_calls": [
{"index": 0, "id": "call_1", "type": "function",
"function": {"name": "ask_user", "arguments": args}}]}}]})
ev({"choices": [{"index": 0, "delta": {}, "finish_reason": "tool_calls"}]})
else:
ev({"choices": [{"index": 0, "delta": {"role": "assistant", "content": "done"}}]})
ev({"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]})
self.wfile.write(b"data: [DONE]\n\n")
self.wfile.flush()
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
class Sse:
def __init__(self, base, last_id=None):
self.lines = []
req = urllib.request.Request(base + "/api/events")
if last_id is not None:
req.add_header("Last-Event-ID", str(last_id))
self.r = urllib.request.urlopen(req, timeout=60)
threading.Thread(target=self.pump, daemon=True).start()
def pump(self):
try:
for raw in self.r:
self.lines.append(raw.decode())
except Exception:
pass
def events(self, name):
out = []
for i, l in enumerate(self.lines):
if l.strip() == "event: " + name and i + 1 < len(self.lines):
out.append(json.loads(self.lines[i + 1][6:]))
return out
def ids(self):
return [int(l[4:]) for l in self.lines if l.startswith("id: ")]
def close(self):
try:
self.r.close()
except Exception:
pass
def wait(fn, t=15):
end = time.time() + t
while time.time() < end:
v = fn()
if v:
return v
time.sleep(0.1)
return None
def post(base, path, obj):
req = urllib.request.Request(base + path, data=json.dumps(obj).encode(),
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=5) as r:
return r.status
except urllib.error.HTTPError as e:
return e.code
api = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=api.serve_forever, daemon=True).start()
ws = tempfile.mkdtemp(prefix="cst-ask-")
log = open(ws + "/server.log", "w+")
p = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws, "--new",
"--api-key", "sk-x", "--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api.server_address[1],
"--protocol", "native-eos", "--approval", "auto"],
stdout=log, stderr=subprocess.STDOUT)
try:
def port():
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return l.split(":")[3].split("/")[0]
pt = wait(port)
base = "http://127.0.0.1:%s" % pt
a = Sse(base)
time.sleep(0.3)
check(post(base, "/api/turn", {"text": "ask me"}) == 202, "turn accepted")
ask = wait(lambda: a.events("ask_user"))
check(bool(ask), "ask_user event raised")
time.sleep(1.5)
check(len(REQS) == 1, "the turn waits for the answer (requests so far: %d)" % len(REQS))
# a page opened now must still see the open question
b = Sse(base)
check(bool(wait(lambda: b.events("ask_user"), 5)), "a late page gets the open question")
# reconnect with Last-Event-ID: nothing before that id is resent
last = a.ids()[-1]
c = Sse(base, last_id=last - 1)
got = wait(lambda: c.ids(), 5)
check(bool(got) and got[0] == last, "Last-Event-ID resumes after that id (%s, want %d)" % (got[:1] if got else got, last))
if ask:
check(post(base, "/api/ask_user", {"id": ask[0]["id"], "text": "blue"}) == 200, "answer accepted")
end = wait(lambda: a.events("turn_end"), 20)
check(bool(end) and end[0].get("rc") == 0, "turn ended rc=0")
tool_msgs = [m for m in (REQS[1]["messages"] if len(REQS) > 1 else []) if m.get("role") == "tool"]
check(bool(tool_msgs) and "blue" in json.dumps(tool_msgs), "the answer reached the model: %s" % json.dumps(tool_msgs)[:200])
for s in (a, b, c):
s.close()
finally:
p.terminate()
try:
p.wait(20)
except subprocess.TimeoutExpired:
p.kill()
check(False, "server hung on SIGTERM")
check(p.returncode is not None and p.returncode < 128 and p.returncode >= 0, "clean exit (%s)" % p.returncode)
log.close()
shutil.rmtree(ws, ignore_errors=True)
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+227
View File
@@ -0,0 +1,227 @@
"""WebUI /init, /skill, extensions and the thinking switch against a fake API.
python3 scripts/e2e_webui_ext.py (CST_AGENT= picks the binary)
Checks: show_thinking is validated, lands in the state and in settings.json;
/api/skill inlines the skill and 404s an unknown one; /api/init sends the
AGENTS.md prompt; /api/extensions lists skills and the built-in roles;
trust and reload work and write settings.json; the changing calls are a 409
while a turn runs, the listing is not. HOME points at a temp dir.
"""
import json, os, shutil, subprocess, sys, tempfile, threading, time, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
SEEN = [] # the full last user message of every request
class Api(BaseHTTPRequestHandler):
"""Answers "好的"; a prompt with "slow" takes 2 s."""
def log_message(self, *a):
pass
def do_GET(self):
body = json.dumps({"data": [{"id": "deepseek-flash"}]}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
req = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0))) or b"{}")
users = [m.get("content") for m in req.get("messages", []) if m.get("role") == "user"]
last = users[-1] if users and isinstance(users[-1], str) else ""
SEEN.append(last)
if "slow" in last:
time.sleep(2)
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
for o in ({"delta": {"role": "assistant", "content": "好的"}},
{"delta": {}, "finish_reason": "stop"}):
chunk = {"id": "x", "object": "chat.completion.chunk", "model": "deepseek-flash",
"choices": [dict(index=0, **o)]}
self.wfile.write(b"data: " + json.dumps(chunk, ensure_ascii=False).encode() + b"\n\n")
self.wfile.flush()
self.wfile.write(b"data: [DONE]\n\n")
self.wfile.flush()
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
def wait(fn, t=15):
end = time.time() + t
while time.time() < end:
v = fn()
if v:
return v
time.sleep(0.1)
return None
def call(base, path, obj=None, method=None):
data = None if obj is None else json.dumps(obj).encode()
req = urllib.request.Request(base + path, data=data, method=method or ("POST" if data else "GET"),
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=30) as r:
return r.status, json.loads(r.read() or b"{}")
except urllib.error.HTTPError as e:
try:
return e.code, json.loads(e.read() or b"{}")
except ValueError:
return e.code, {}
def skill(ws, name, desc, body):
d = os.path.join(ws, ".agent", "skills", name)
os.makedirs(d, exist_ok=True)
with open(os.path.join(d, "SKILL.md"), "w", encoding="utf-8") as f:
f.write("---\nname: %s\ndescription: %s\n---\n\n%s\n" % (name, desc, body))
api = ThreadingHTTPServer(("127.0.0.1", 0), Api)
threading.Thread(target=api.serve_forever, daemon=True).start()
ws = tempfile.mkdtemp(prefix="cst-ext-")
home = tempfile.mkdtemp(prefix="cst-home-")
skill(ws, "demo", "演示用的 skill", "DEMO-INSTRUCTIONS: 先说你好。")
env = dict(os.environ, HOME=home, XDG_CONFIG_HOME=home + "/.config",
XDG_DATA_HOME=home + "/.local/share", XDG_CACHE_HOME=home + "/.cache")
for k in ("DEEPSEEK_API_KEY", "OPENAI_API_KEY", "CST_API_KEY"):
env.pop(k, None)
log = open(ws + "/server.log", "w+")
proc = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws, "--new", "--api-key", "sk-x",
"--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api.server_address[1],
"--protocol", "native-eos", "--approval", "auto"],
stdout=log, stderr=subprocess.STDOUT, env=env)
def port():
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return l.split(":")[3].split("/")[0]
def settings():
try:
with open(home + "/.config/cst-agent/settings.json", encoding="utf-8") as f:
return json.load(f)
except OSError:
return {}
def busy(base):
return call(base, "/api/state")[1].get("status_busy")
def turn(base, path, body):
code, d = call(base, path, body)
if code == 202:
time.sleep(0.3)
wait(lambda: not busy(base), 20)
return code, d
try:
base = "http://127.0.0.1:%s" % wait(port)
# 1. the thinking switch
st = call(base, "/api/state")[1]
check(st.get("show_thinking") is False, "show_thinking starts off")
check(call(base, "/api/config", {"show_thinking": "yes"})[0] == 400, "show_thinking must be a bool")
code, _ = call(base, "/api/config", {"show_thinking": True})
check(code == 200 and call(base, "/api/state")[1].get("show_thinking") is True, "show_thinking on")
check(settings().get("show_thinking") is True, "show_thinking is saved in settings.json")
call(base, "/api/config", {"show_thinking": False})
check(settings() and not settings().get("show_thinking"), "and cleared again (%s)" % settings())
# 2. the extensions listing
code, ex = call(base, "/api/extensions")
check(code == 200 and ex.get("enabled") is True and ex.get("trusted") is False,
"extensions: enabled, not trusted (%s)" % code)
names = [s["name"] for s in ex.get("skills", [])]
check(names == ["demo"] and ex["skills"][0].get("desc") == "演示用的 skill"
and ex["skills"][0].get("source") == "workspace", "the workspace skill is listed (%s)" % ex.get("skills"))
roles = [r["name"] for r in ex.get("roles", [])]
check("explore" in roles and all(r.get("source") for r in ex["roles"]), "built-in roles are listed (%s)" % roles)
check(ex.get("mcp") == [] and ex.get("plugins") == [] and ex.get("runs") == [], "no MCP, plugins or runs")
check(ex.get("task_tool") is True and "1 skill" in ex.get("report", ""), "task tool on, report %r" % ex.get("report"))
# 3. /skill
code, d = turn(base, "/api/skill", {"name": "demo", "task": "做点事"})
check(code == 202 and "DEMO-INSTRUCTIONS" in d.get("text", "") and d["text"].endswith("做点事"),
"skill turn accepted (%s)" % code)
check(SEEN and "DEMO-INSTRUCTIONS" in SEEN[-1] and "做点事" in SEEN[-1], "the model saw the skill and the task")
code, d = turn(base, "/api/skill", {"name": "demo"})
check(code == 202 and SEEN[-1].endswith("Apply this skill now."), "a skill without a task")
check(call(base, "/api/skill", {"name": "nope"})[0] == 404, "unknown skill: 404")
check(call(base, "/api/skill", {"name": "../demo"})[0] == 404, "skill ../demo: 404")
check(call(base, "/api/skill", {})[0] == 400, "skill without a name: 400")
check(call(base, "/api/skill", {"name": "x" * 100})[0] == 400, "a long skill name: 400")
check(call(base, "/api/skill", {"name": "demo", "task": 5})[0] == 400, "a non-string task: 400")
# 4. /init
code, d = turn(base, "/api/init", {})
check(code == 202 and "AGENTS.md" in d.get("text", ""), "init accepted (%s)" % code)
check("AGENTS.md" in SEEN[-1], "the model saw the AGENTS.md prompt")
# 5. trust and reload
code, d = call(base, "/api/extensions/trust", {"on": True})
check(code == 200 and d.get("trusted") is True, "trust the workspace (%s %s)" % (code, d))
tw = settings().get("trusted_workspaces", [])
check(any(os.path.realpath(ws) == os.path.realpath(p) for p in tw), "settings.json lists it (%s)" % tw)
check(call(base, "/api/extensions")[1].get("trusted") is True, "the listing says trusted")
code, d = call(base, "/api/extensions/trust", {"on": False})
check(code == 200 and d.get("trusted") is False and not settings().get("trusted_workspaces"),
"untrust (%s)" % settings().get("trusted_workspaces"))
check(call(base, "/api/extensions/trust", {"on": "yes"})[0] == 400, "trust needs a bool")
skill(ws, "zeta", "第二个", "ZETA")
code, d = call(base, "/api/extensions/reload", {})
check(code == 200 and "2 skills" in d.get("report", ""), "reload sees the new skill (%s)" % d)
check([s["name"] for s in call(base, "/api/extensions")[1]["skills"]] == ["demo", "zeta"], "both listed")
# 6. while a turn runs
call(base, "/api/turn", {"text": "slow 慢一点"})
time.sleep(0.4)
check(call(base, "/api/extensions/reload", {})[0] == 409, "reload during a turn: 409")
check(call(base, "/api/extensions/trust", {"on": True})[0] == 409, "trust during a turn: 409")
check(call(base, "/api/skill", {"name": "demo"})[0] == 409, "skill during a turn: 409")
check(call(base, "/api/init", {})[0] == 409, "init during a turn: 409")
check(call(base, "/api/extensions")[0] == 200, "the listing works during a turn")
check(call(base, "/api/config", {"show_thinking": True})[0] == 200, "the thinking switch works during a turn")
wait(lambda: not busy(base), 20)
check(not settings().get("trusted_workspaces"), "the refused trust changed nothing")
# 7. another workspace brings its own skills
ws2 = tempfile.mkdtemp(prefix="cst-ext2-")
skill(ws2, "other", "另一个工作区的", "OTHER")
code, _ = call(base, "/api/workspace", {"path": ws2})
ex = call(base, "/api/extensions")[1]
check(code == 200 and [s["name"] for s in ex.get("skills", [])] == ["other"]
and ex.get("report") == "1 skill", "a workspace switch reloads the extensions (%s)" % ex.get("report"))
code, d = turn(base, "/api/skill", {"name": "other"})
check(code == 202 and "OTHER" in SEEN[-1], "and its skill runs")
shutil.rmtree(ws2, ignore_errors=True)
finally:
proc.terminate()
try:
proc.wait(20)
except subprocess.TimeoutExpired:
proc.kill()
check(False, "server hung on SIGTERM")
log.close()
shutil.rmtree(ws, ignore_errors=True)
shutil.rmtree(home, ignore_errors=True)
api.shutdown()
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+234
View File
@@ -0,0 +1,234 @@
"""WebUI HTTP hardening checks over raw sockets, against a scripted model.
python3 scripts/e2e_webui_http.py (CST_AGENT= picks the binary)
Cross-site requests, request parsing (Content-Length, NUL, long lines,
keep-alive), auth before the body, changes refused while a turn runs,
cancel waking a pending question, token mode and shutdown.
"""
import json, os, re, shutil, socket, subprocess, sys, tempfile, threading, time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
REQS = []
class Api(BaseHTTPRequestHandler):
"""First call: ask_user. Later calls: a plain answer."""
def log_message(self, *a):
pass
def do_GET(self):
body = json.dumps({"data": [{"id": "deepseek-flash"}]}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
n = int(self.headers.get("Content-Length", 0))
REQS.append(json.loads(self.rfile.read(n)))
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
base = {"id": "x", "object": "chat.completion.chunk", "model": "deepseek-flash"}
def ev(o):
self.wfile.write(b"data: " + json.dumps({**base, **o}).encode() + b"\n\n")
self.wfile.flush()
if len(REQS) == 1:
args = json.dumps({"question": "Pick?", "options": "a\nb"})
ev({"choices": [{"index": 0, "delta": {"role": "assistant", "tool_calls": [
{"index": 0, "id": "call_1", "type": "function",
"function": {"name": "ask_user", "arguments": args}}]}}]})
ev({"choices": [{"index": 0, "delta": {}, "finish_reason": "tool_calls"}]})
else:
ev({"choices": [{"index": 0, "delta": {"role": "assistant", "content": "ok"}}]})
ev({"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]})
self.wfile.write(b"data: [DONE]\n\n")
self.wfile.flush()
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
def raw(port, data, timeout=5, read_all=True):
"""Send bytes, return everything read until EOF/timeout."""
s = socket.create_connection(("127.0.0.1", port), timeout=timeout)
s.sendall(data)
out = b""
try:
while True:
b = s.recv(65536)
if not b:
break
out += b
if not read_all:
break
except socket.timeout:
pass
s.close()
return out.decode("latin1")
def status(resp):
# a pipelined response starts right after the previous body, not on a new line
return [int(m) for m in re.findall(r"HTTP/1\.1 (\d{3}) ", resp)]
def req(method, path, headers=None, body=b"", host=None):
h = {"Host": host or "127.0.0.1:%d" % PORT, "Connection": "close"}
h.update(headers or {})
if body or method == "POST":
h["Content-Length"] = str(len(body))
head = "%s %s HTTP/1.1\r\n" % (method, path) + "".join("%s: %s\r\n" % kv for kv in h.items()) + "\r\n"
return head.encode() + body
def start(ws, *extra):
log = open(os.path.join(ws, "server.log"), "w+")
p = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws, "--new",
"--api-key", "sk-x", "--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api.server_address[1],
"--protocol", "native-eos", "--approval", "auto", *extra],
stdout=log, stderr=subprocess.STDOUT)
end = time.time() + 10
while time.time() < end:
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return p, int(l.split(":")[3].split("/")[0]), log
time.sleep(0.1)
raise SystemExit("server did not start")
def stop(p, name):
p.terminate()
try:
p.wait(20)
except subprocess.TimeoutExpired:
p.kill()
check(False, name + " hung on SIGTERM")
return
check(0 <= p.returncode < 128, "%s exits cleanly (%s)" % (name, p.returncode))
api = ThreadingHTTPServer(("127.0.0.1", 0), Api)
threading.Thread(target=api.serve_forever, daemon=True).start()
root = tempfile.mkdtemp(prefix="cst-http-")
try:
ws = os.path.join(root, "a")
os.makedirs(ws)
p, PORT, log = start(ws)
me = "http://127.0.0.1:%d" % PORT
cfg = b'{"approval":"ask"}'
# 1. cross-site requests
r = raw(PORT, req("POST", "/api/config", {"Origin": "http://evil.example", "Content-Type": "text/plain"}, cfg))
check(status(r)[:1] == [401], "a cross-site POST is refused (%s)" % status(r))
r = raw(PORT, req("POST", "/api/config", {"Origin": "null"}, cfg))
check(status(r)[:1] == [401], "Origin: null is refused")
r = raw(PORT, req("POST", "/api/config", {"Origin": me, "Content-Type": "application/json"}, b'{"approval":"auto"}'))
check(status(r)[:1] == [200], "our own page may POST (%s)" % status(r))
r = raw(PORT, req("POST", "/api/config", {}, b'{"approval":"auto"}'))
check(status(r)[:1] == [200], "a client without Origin (curl) may POST")
r = raw(PORT, req("GET", "/api/attach/img-0000.png", host="evil.example"))
check(status(r)[:1] == [401], "session images need the Host check too (%s)" % status(r))
r = raw(PORT, req("GET", "/app.js"))
check(status(r)[:1] == [200], "assets are served")
# 2. request parsing
r = raw(PORT, req("GET", "/api/state%00x"))
check(status(r)[:1] == [400], "an encoded NUL in the path is a 400 (%s)" % status(r))
smuggle = b"GET /api/sessions HTTP/1.1\r\nHost: 127.0.0.1\r\n\r\n"
r = raw(PORT, ("POST /api/cancel HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nContent-Length: abc\r\n\r\n" % PORT).encode() + smuggle)
check(status(r) == [400], "Content-Length: abc is a 400, nothing smuggled (%s)" % status(r))
r = raw(PORT, ("POST /api/cancel HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nContent-Length: 0\r\nContent-Length: %d\r\n\r\n"
% (PORT, len(smuggle))).encode() + smuggle)
check(status(r) == [400], "two different Content-Lengths are a 400 (%s)" % status(r))
r = raw(PORT, ("GET /%s HTTP/1.1\r\nHost: x\r\n\r\n" % ("a" * 9000)).encode())
check(status(r)[:1] == [400], "an over-long request line is a 400 (%s)" % status(r))
r = raw(PORT, req("GET", "/api/state?x=http://a/b"))
check(status(r)[:1] == [200], "'://' in the query leaves the path alone (%s)" % status(r))
r = raw(PORT, req("GET", "/api/state", {"Connection": "Close"}), timeout=3)
check("Connection: close" in r, "Connection: Close (any case) closes")
ka = ("GET /api/state HTTP/1.1\r\nHost: 127.0.0.1:%d\r\n\r\n" % PORT).encode()
big = ("POST /api/turn HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nContent-Length: 5000000\r\n\r\n" % PORT).encode()
r = raw(PORT, ka + big)
check(status(r) == [200, 413], "a 413 after a keep-alive request still arrives (%s)" % status(r))
# 3. a pending question: changes are refused, approval may change, cancel wakes it
events = []
def sse():
s = socket.create_connection(("127.0.0.1", PORT))
s.sendall(("GET /api/events HTTP/1.1\r\nHost: 127.0.0.1:%d\r\n\r\n" % PORT).encode())
buf = b""
s.settimeout(60)
try:
while True:
b = s.recv(65536)
if not b:
break
buf += b
events[:] = [l[7:] for l in buf.decode("latin1").split("\n") if l.startswith("event: ")]
except Exception:
pass
threading.Thread(target=sse, daemon=True).start()
time.sleep(0.4)
r = raw(PORT, req("POST", "/api/turn", {"Content-Type": "application/json"}, b'{"text":"go"}'))
check(status(r)[:1] == [202], "turn accepted")
end = time.time() + 15
while time.time() < end and "ask_user" not in events:
time.sleep(0.1)
check("ask_user" in events, "the question is open")
for path, body in (("/api/sessions/new", b""), ("/api/workspace", json.dumps({"path": root}).encode()),
("/api/compact", b""), ("/api/undo", b""), ("/api/config", b'{"model":"x"}')):
r = raw(PORT, req("POST", path, {"Content-Type": "application/json"}, body))
check(status(r)[:1] == [409], "%s is refused mid-turn (%s)" % (path, status(r)))
r = raw(PORT, req("POST", "/api/config", {"Content-Type": "application/json"}, b'{"approval":"yolo"}'))
check(status(r)[:1] == [200], "approval may change mid-turn")
r = raw(PORT, req("POST", "/api/cancel", {"Content-Type": "application/json"}, b"{}"))
end = time.time() + 10
while time.time() < end and "turn_end" not in events:
time.sleep(0.1)
check("turn_end" in events, "cancel ends a turn parked on a question")
r = raw(PORT, req("POST", "/api/sessions/new", {"Content-Type": "application/json"}))
check(status(r)[:1] == [200], "after the turn, a new session is fine (%s)" % status(r))
stop(p, "the server")
# 4. token mode: the page loads, the API and images do not without it
ws2 = os.path.join(root, "b")
os.makedirs(ws2)
p, PORT, log = start(ws2, "--web-token", "tok-x")
r = raw(PORT, req("GET", "/app.js"))
check(status(r)[:1] == [200], "token mode: app.js loads without the token (%s)" % status(r))
r = raw(PORT, req("GET", "/api/state"))
check(status(r)[:1] == [401], "token mode: the API needs it")
r = raw(PORT, req("GET", "/api/attach/img-0000.png"))
check(status(r)[:1] == [401], "token mode: images need it")
r = raw(PORT, req("GET", "/api/state", {"Cookie": "cst_token=tok-x"}))
check(status(r)[:1] == [200], "token mode: the cookie works")
t0 = time.time()
r = raw(PORT, ("POST /api/attach HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nContent-Length: 33554431\r\n"
"Expect: 100-continue\r\n\r\n" % PORT).encode(), timeout=5)
check(status(r) == [401] and time.time() - t0 < 3,
"no token, no body read: 401 at once (%s, %.1fs)" % (status(r), time.time() - t0))
stop(p, "the token server")
# 5. option checks
rc = subprocess.run([BIN, "--web", "--web-port", "70000", "--workspace", ws2],
capture_output=True, timeout=10).returncode
check(rc != 0, "--web-port 70000 is refused")
rc = subprocess.run([BIN, "--web", "--web-token", "x" * 80, "--web-port", "0", "--workspace", ws2,
"--api-key", "k"], capture_output=True, timeout=10).returncode
check(rc != 0, "an over-long --web-token is refused")
finally:
shutil.rmtree(root, ignore_errors=True)
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+327
View File
@@ -0,0 +1,327 @@
"""WebUI session management against a fake OpenAI-compatible API.
python3 scripts/e2e_webui_sessions.py (CST_AGENT= picks the binary)
Checks: retry runs the last prompt again and edit-and-resend replaces it
(the model sees the new text, the session keeps one copy); retry is a 409
while a turn runs and when there is nothing to retry; export returns the
transcript as Markdown; rename, clear and delete work on the store, and
deleting the open session moves to a new one; history messages carry the
note / tool_result flags. HOME points at a temp dir.
"""
import json, os, shutil, subprocess, sys, tempfile, threading, time, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
SEEN = []
def last_prompt(msgs):
"""The prompts are one line; a note before one shares its user message."""
for m in reversed(msgs):
c = m.get("content")
if m.get("role") == "user" and isinstance(c, str):
return c.rstrip("\n").split("\n")[-1]
return ""
class Api(BaseHTTPRequestHandler):
"""Answers "回答:<last prompt>"; a prompt with "slow" takes 2 s."""
def log_message(self, *a):
pass
def do_GET(self):
body = json.dumps({"data": [{"id": "deepseek-flash"}]}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
req = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0))) or b"{}")
msgs = req.get("messages", [])
p = last_prompt(msgs)
tail = msgs[-1] if msgs else {}
tc = tail.get("content") if isinstance(tail.get("content"), str) else ""
answered = tail.get("role") == "tool" or tc.startswith("result ")
if not answered:
SEEN.append(p)
if "slow" in p:
time.sleep(2)
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
if answered:
chunks = ({"delta": {"role": "assistant", "content": "读完了"}},
{"delta": {}, "finish_reason": "stop"})
elif p.startswith("native"):
args = json.dumps({"path": "hello.txt"})
chunks = ({"delta": {"role": "assistant", "tool_calls": [
{"index": 0, "id": "c1", "type": "function",
"function": {"name": "read_file", "arguments": args}}]}},
{"delta": {}, "finish_reason": "tool_calls"})
elif p.startswith("ctl"):
chunks = ({"delta": {"role": "assistant",
"content": "我来读。\ncall t1 read_file\narg path 9\nhello.txtend\n"}},
{"delta": {}, "finish_reason": "stop"})
else:
chunks = ({"delta": {"role": "assistant", "content": "回答:" + p}},
{"delta": {}, "finish_reason": "stop"})
for o in chunks:
chunk = {"id": "x", "object": "chat.completion.chunk", "model": "deepseek-flash",
"choices": [dict(index=0, **o)]}
self.wfile.write(b"data: " + json.dumps(chunk, ensure_ascii=False).encode() + b"\n\n")
self.wfile.flush()
self.wfile.write(b"data: [DONE]\n\n")
self.wfile.flush()
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
def wait(fn, t=15):
end = time.time() + t
while time.time() < end:
v = fn()
if v:
return v
time.sleep(0.1)
return None
def call(base, path, obj=None, method=None, raw_reply=False):
data = None if obj is None else json.dumps(obj).encode()
req = urllib.request.Request(base + path, data=data, method=method or ("POST" if data else "GET"),
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=20) as r:
raw = r.read()
if raw_reply:
return r.status, r.headers.get("Content-Type", ""), raw
return r.status, json.loads(raw or b"{}")
except urllib.error.HTTPError as e:
raw = e.read()
if raw_reply:
return e.code, e.headers.get("Content-Type", ""), raw
try:
return e.code, json.loads(raw or b"{}")
except ValueError:
return e.code, {}
class Sse:
def __init__(self, base):
self.lines = []
self.r = urllib.request.urlopen(base + "/api/events", timeout=60)
threading.Thread(target=self.pump, daemon=True).start()
def pump(self):
try:
for raw in self.r:
self.lines.append(raw)
except Exception:
pass
def count(self, name):
return sum(1 for l in self.lines if l.strip() == b"event: " + name.encode())
def close(self):
try:
self.r.close()
except Exception:
pass
api = ThreadingHTTPServer(("127.0.0.1", 0), Api)
threading.Thread(target=api.serve_forever, daemon=True).start()
ws = tempfile.mkdtemp(prefix="cst-sess-")
home = tempfile.mkdtemp(prefix="cst-home-")
env = dict(os.environ, HOME=home, XDG_CONFIG_HOME=home + "/.config",
XDG_DATA_HOME=home + "/.local/share", XDG_CACHE_HOME=home + "/.cache")
for k in ("DEEPSEEK_API_KEY", "OPENAI_API_KEY", "CST_API_KEY"):
env.pop(k, None)
log = open(ws + "/server.log", "w+")
proc = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws, "--new", "--api-key", "sk-x",
"--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api.server_address[1],
"--protocol", "native-eos", "--approval", "auto"],
stdout=log, stderr=subprocess.STDOUT, env=env)
def port():
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return l.split(":")[3].split("/")[0]
def messages(base):
return call(base, "/api/session")[1].get("messages", [])
def prompts(base):
return [m["content"] for m in messages(base)
if m["role"] == "user" and not m.get("note") and not m.get("tool_result")]
def answers(base):
return [m["content"] for m in messages(base) if m["role"] == "assistant"]
def turn(base, sse, path, body, want=202):
n = sse.count("turn_end")
code, d = call(base, path, body)
if code == want == 202:
wait(lambda: sse.count("turn_end") > n, 20)
return code, d
try:
base = "http://127.0.0.1:%s" % wait(port)
sse = Sse(base)
time.sleep(0.3)
# 1. retry with nothing to retry, then a first turn
code, d = call(base, "/api/retry", {})
check(code == 409 and "nothing to retry" in d.get("error", ""), "retry on an empty session: 409 (%s)" % code)
turn(base, sse, "/api/turn", {"text": "第一句"})
check(prompts(base) == ["第一句"] and answers(base) == ["回答:第一句"], "first turn stored")
# 2. retry: same prompt again, one copy in the session
code, d = turn(base, sse, "/api/retry", {})
check(code == 202 and d.get("text") == "第一句", "retry accepted (%s %s)" % (code, d))
check(SEEN[-1] == "第一句", "the model saw the same prompt again")
check(prompts(base) == ["第一句"] and answers(base) == ["回答:第一句"],
"after retry: one prompt, one answer (%s)" % prompts(base))
# 3. edit and resend: the new text replaces the old
code, d = turn(base, sse, "/api/retry", {"text": "改过的问题"})
check(code == 202 and d.get("text") == "改过的问题", "edit-and-resend accepted")
check(SEEN[-1] == "改过的问题", "the model saw the edited prompt")
check(prompts(base) == ["改过的问题"] and answers(base) == ["回答:改过的问题"],
"after edit: %s / %s" % (prompts(base), answers(base)))
check(call(base, "/api/retry", {"text": ""})[0] == 400, "an empty edit is a 400")
check(call(base, "/api/retry", {"text": 5})[0] == 400, "a non-string edit is a 400")
# 4. a second turn, then retry only touches the last one
turn(base, sse, "/api/turn", {"text": "第二句"})
turn(base, sse, "/api/retry", {})
check(prompts(base) == ["改过的问题", "第二句"] and len(answers(base)) == 2,
"retry keeps earlier turns: %s" % prompts(base))
# 5. retry while a turn runs is a 409
n = sse.count("turn_end")
call(base, "/api/turn", {"text": "slow 请慢一点"})
code, d = call(base, "/api/retry", {})
check(code == 409, "retry during a turn: 409 (%s)" % code)
wait(lambda: sse.count("turn_end") > n, 20)
# 6. a note (plan mode changed mid-session) is flagged, and retry skips it
call(base, "/api/config", {"plan_mode": True})
turn(base, sse, "/api/turn", {"text": "计划一下"})
ms = messages(base)
notes = [m for m in ms if m.get("note")]
check(len(notes) >= 1 and all(m["role"] == "user" for m in notes),
"a mid-session note is flagged note:true (%d)" % len(notes))
turn(base, sse, "/api/retry", {})
check(SEEN[-1] == "计划一下" and prompts(base)[-1] == "计划一下", "retry after a note reruns the prompt")
call(base, "/api/config", {"plan_mode": False})
# 7. export
code, ctype, raw = call(base, "/api/export", raw_reply=True)
md = raw.decode("utf-8", "replace")
check(code == 200 and ctype.startswith("text/markdown"), "export: %s %s" % (code, ctype))
check(md.startswith("# ") and "## User" in md and "改过的问题" in md and "## Assistant" in md
and "回答:第二句" in md and "第一句" not in md, "export has the transcript")
# 8. rename and clear
st = call(base, "/api/state")[1]
cur = st.get("session")
code, _ = call(base, "/api/sessions/rename", {"title": "我的会话"})
check(code == 200, "rename the open session")
ss = call(base, "/api/sessions")[1]
check(any(s["id"] == cur and s["title"] == "我的会话" for s in ss.get("sessions", [])),
"the list shows the new title")
code, _, raw = call(base, "/api/export", raw_reply=True)
check(raw.decode("utf-8", "replace").startswith("# 我的会话"), "export uses the title")
long = "长" * 100
call(base, "/api/sessions/rename", {"id": cur, "title": long})
t = [s["title"] for s in call(base, "/api/sessions")[1]["sessions"] if s["id"] == cur][0]
check(0 < len(t.encode()) <= 120 and t == "长" * len(t), "a long title is cut on a character (%d bytes)"
% len(t.encode()))
call(base, "/api/sessions/rename", {"title": ""})
t = [s["title"] for s in call(base, "/api/sessions")[1]["sessions"] if s["id"] == cur][0]
check(t == "改过的问题", "an empty title falls back to the first prompt (%r)" % t)
check(call(base, "/api/sessions/rename", {"id": "nope", "title": "x"})[0] == 404, "rename unknown: 404")
check(call(base, "/api/sessions/rename", {"id": "../x", "title": "x"})[0] == 404, "rename ../x: 404")
check(call(base, "/api/sessions/rename", {})[0] == 400, "rename without a title: 400")
# 9. delete another session, then the open one
code, d = call(base, "/api/sessions/new", {})
second = d.get("id")
check(code == 200 and second and second != cur, "a second session")
code, d = call(base, "/api/sessions/delete", {"id": cur})
check(code == 200 and d.get("current") == second, "delete a closed session (%s %s)" % (code, d))
ids = [s["id"] for s in call(base, "/api/sessions")[1]["sessions"]]
check(cur not in ids and not os.path.exists(os.path.join(ws, ".agent", "sessions", cur + ".jsonl")),
"it is gone from the list and the disk")
n = sse.count("session")
code, d = call(base, "/api/sessions/delete", {"id": second})
check(code == 200 and d.get("current") and d["current"] != second, "delete the open session (%s)" % d)
check(call(base, "/api/state")[1].get("session") == d.get("current"), "the state moved to the new one")
check(wait(lambda: sse.count("session") > n, 5), "a session event was sent")
check(call(base, "/api/sessions/delete", {"id": "nope"})[0] == 404, "delete unknown: 404")
check(call(base, "/api/sessions/delete", {})[0] == 400, "delete without an id: 400")
# 10. history: calls and results come back apart, for native and ctl
with open(os.path.join(ws, "hello.txt"), "w", encoding="utf-8") as f:
f.write("你好世界\n")
turn(base, sse, "/api/turn", {"text": "native 读文件"})
ms = messages(base)
a = [m for m in ms if m["role"] == "assistant" and m.get("calls")]
r = [m for m in ms if m.get("tool_result")]
check(len(a) == 1 and a[0]["calls"][0].get("name") == "read_file"
and a[0]["calls"][0].get("args", {}).get("path") == "hello.txt" and a[0]["calls"][0].get("id") == "c1",
"native: the call is listed with its args (%s)" % (a[0].get("calls") if a else ms))
check(len(r) == 1 and r[0]["role"] == "tool" and r[0].get("tool_call_id") == "c1"
and "你好世界" in r[0].get("out", "") and not r[0]["out"].startswith("[tool ")
and r[0].get("exit") == 0 and r[0].get("truncated") is False,
"native: the result carries out/exit without the header (%s)" % r)
check(answers(base)[-1] == "读完了", "native: the turn finished")
check(call(base, "/api/settings", {"protocol": "ctl"})[0] == 200, "switch to the ctl protocol")
turn(base, sse, "/api/turn", {"text": "ctl 读文件"})
ms = messages(base)
a = [m for m in ms if m["role"] == "assistant" and m.get("calls") and m["calls"][0].get("id") == "t1"]
r = [m for m in ms if m.get("tool_result") and m.get("tool_call_id") == "t1"]
check(len(a) == 1 and a[0].get("text", "").strip() == "我来读。" and "call t1" in a[0]["content"]
and a[0]["calls"][0].get("args", {}).get("path") == "hello.txt"
and "call t1" not in a[0].get("html", ""),
"ctl: prose and calls are split (%s)" % (a[0] if a else ms[-4:]))
check(len(r) == 1 and r[0]["role"] == "user" and "你好世界" in r[0].get("out", "")
and not r[0]["out"].startswith("result ") and not r[0]["out"].endswith("end")
and r[0].get("exit") == 0,
"ctl: the result is flagged with out/exit (%s)" % r)
check(r and r[0]["content"] not in prompts(base), "ctl: the result is not a prompt")
call(base, "/api/settings", {"protocol": "native-eos"})
sse.close()
finally:
proc.terminate()
try:
proc.wait(20)
except subprocess.TimeoutExpired:
proc.kill()
check(False, "server hung on SIGTERM")
log.close()
shutil.rmtree(ws, ignore_errors=True)
shutil.rmtree(home, ignore_errors=True)
api.shutdown()
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+313
View File
@@ -0,0 +1,313 @@
"""WebUI settings and streaming against fake OpenAI-compatible APIs.
python3 scripts/e2e_webui_settings.py (CST_AGENT= picks the binary)
Checks: Chinese reasoning streams as whole characters in few events; the
provider list, the model list with capabilities, config validation and
persistence; a wrong key is rejected, a good one switches the provider
(requests then go to the new base URL) and is saved; and all of it
survives a restart. HOME and XDG_CONFIG_HOME point at a temp dir, so no
real credentials or settings are read or written.
"""
import json, os, shutil, stat, subprocess, sys, tempfile, threading, time, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
THINK = ["好的,我先看看工作区里", "有哪些文件,然后用中文", "回答这个问题。😀 结束"]
ANSWER = "这是中文回答。"
GOOD_KEYS = {"sk-good", "sk-good2"}
B_REQS = []
def sse_reply(h, model, think, answer):
h.send_response(200)
h.send_header("Content-Type", "text/event-stream")
h.end_headers()
base = {"id": "x", "object": "chat.completion.chunk", "model": model}
def ev(o):
h.wfile.write(b"data: " + json.dumps({**base, **o}, ensure_ascii=False).encode() + b"\n\n")
h.wfile.flush()
for part in think:
ev({"choices": [{"index": 0, "delta": {"role": "assistant", "reasoning_content": part}}]})
ev({"choices": [{"index": 0, "delta": {"content": answer}}]})
ev({"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]})
h.wfile.write(b"data: [DONE]\n\n")
h.wfile.flush()
def send_json(h, code, obj):
body = json.dumps(obj).encode()
h.send_response(code)
h.send_header("Content-Type", "application/json")
h.send_header("Content-Length", str(len(body)))
h.end_headers()
h.wfile.write(body)
class A(BaseHTTPRequestHandler):
"""The starting provider: any key, one model, Chinese reasoning."""
def log_message(self, *a):
pass
def do_GET(self):
send_json(self, 200, {"data": [{"id": "deepseek-flash"}]})
def do_POST(self):
self.rfile.read(int(self.headers.get("Content-Length", 0)))
sse_reply(self, "deepseek-flash", THINK, ANSWER)
class B(BaseHTTPRequestHandler):
"""The provider switched to: /models checks the key."""
def log_message(self, *a):
pass
def key_ok(self):
auth = self.headers.get("Authorization", "")
return auth.startswith("Bearer ") and auth[7:] in GOOD_KEYS
def do_GET(self):
if not self.key_ok():
return send_json(self, 401, {"error": {"message": "invalid api key"}})
send_json(self, 200, {"data": [{"id": "fake-vis"}, {"id": "fake-text"}]})
def do_POST(self):
body = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0))) or b"{}")
B_REQS.append({"auth": self.headers.get("Authorization", ""), "model": body.get("model")})
if not self.key_ok():
return send_json(self, 401, {"error": {"message": "invalid api key"}})
sse_reply(self, body.get("model") or "fake-vis", [], "B 的回答")
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
class Sse:
def __init__(self, base):
self.lines = []
self.r = urllib.request.urlopen(base + "/api/events", timeout=60)
threading.Thread(target=self.pump, daemon=True).start()
def pump(self):
try:
for raw in self.r:
self.lines.append(raw)
except Exception:
pass
def events(self, name):
out = []
for i, l in enumerate(self.lines):
if l.strip() == b"event: " + name.encode() and i + 1 < len(self.lines):
out.append(self.lines[i + 1][6:])
return out
def close(self):
try:
self.r.close()
except Exception:
pass
def wait(fn, t=15):
end = time.time() + t
while time.time() < end:
v = fn()
if v:
return v
time.sleep(0.1)
return None
def call(base, path, obj=None, method=None):
data = None if obj is None else json.dumps(obj).encode()
req = urllib.request.Request(base + path, data=data, method=method or ("POST" if data else "GET"),
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=20) as r:
raw = r.read()
return r.status, json.loads(raw or b"{}"), raw
except urllib.error.HTTPError as e:
raw = e.read()
try:
return e.code, json.loads(raw or b"{}"), raw
except ValueError:
return e.code, {}, raw
def serve(handler):
s = ThreadingHTTPServer(("127.0.0.1", 0), handler)
threading.Thread(target=s.serve_forever, daemon=True).start()
return s
api_a, api_b = serve(A), serve(B)
ws = tempfile.mkdtemp(prefix="cst-set-")
home = tempfile.mkdtemp(prefix="cst-home-")
cfgdir = home + "/.config/cst-agent"
os.makedirs(cfgdir)
with open(cfgdir + "/models.json", "w") as f:
json.dump({"providers": {"fakeb": {"api_base": "http://127.0.0.1:%d/v1" % api_b.server_address[1],
"key_env": "FAKEB_API_KEY", "default_model": ""}},
"models": [{"match": "fake-vis", "vision": True, "context": 200000},
{"match": "fake-text", "vision": False}]}, f)
env = dict(os.environ, HOME=home, XDG_CONFIG_HOME=home + "/.config",
XDG_DATA_HOME=home + "/.local/share", XDG_CACHE_HOME=home + "/.cache")
for k in ("DEEPSEEK_API_KEY", "FAKEB_API_KEY", "OPENAI_API_KEY", "CST_API_KEY"):
env.pop(k, None)
procs = []
def start(args, tag):
log = open("%s/%s.log" % (ws, tag), "w+")
p = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws] + args,
stdout=log, stderr=subprocess.STDOUT, env=env)
procs.append((p, log))
def port():
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return l.split(":")[3].split("/")[0]
pt = wait(port)
return p, "http://127.0.0.1:%s" % pt
def stop(p):
p.terminate()
try:
p.wait(20)
except subprocess.TimeoutExpired:
p.kill()
check(False, "server hung on SIGTERM")
check(p.returncode is not None and 0 <= p.returncode < 128, "clean exit (%s)" % p.returncode)
def leaked(raw):
return any(k.encode() in raw for k in GOOD_KEYS | {"sk-bad", "sk-x"})
try:
p, base = start(["--new", "--api-key", "sk-x", "--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api_a.server_address[1],
"--protocol", "native-eos", "--approval", "auto"], "first")
a = Sse(base)
time.sleep(0.3)
# 1. Chinese reasoning: whole characters, a handful of events
code, _, _ = call(base, "/api/turn", {"text": "用中文想一想"})
check(code == 202, "turn accepted")
end = wait(lambda: a.events("turn_end"), 20)
check(bool(end) and json.loads(end[0]).get("rc") == 0, "turn ended rc=0")
raw = a.events("think_delta")
ok = True
for r in raw:
try:
r.decode("utf-8")
except UnicodeDecodeError:
ok = False
check(ok, "every think_delta frame is valid UTF-8")
text = "".join(json.loads(r.decode("utf-8", "replace"))["text"] for r in raw)
want = "".join(THINK)
check(text == want and "\ufffd" not in text, "the thinking text round trips: %r" % text[:60])
check(0 < len(raw) <= 2 * len(THINK) + 2,
"think_delta events: %d for %d bytes" % (len(raw), len(want.encode())))
ans = "".join(json.loads(r)["text"] for r in a.events("assistant_delta"))
check(ans == ANSWER, "the answer round trips: %r" % ans)
# 2. providers and models
code, d, rawp = call(base, "/api/providers")
names = {x["name"]: x for x in d.get("providers", [])}
check(code == 200 and "deepseek" in names and "fakeb" in names, "provider list: %s" % sorted(names))
check(names.get("deepseek", {}).get("current") and names["deepseek"].get("has_key")
and names["deepseek"].get("key_from") == "session", "deepseek: current, key for this run")
check(names.get("fakeb", {}).get("has_key") is False and names["fakeb"].get("source") == "user",
"fakeb: no key yet, from models.json")
check(not leaked(rawp), "no key in /api/providers")
code, d, _ = call(base, "/api/models")
ids = [m["id"] for m in d.get("models", [])]
first = (d.get("models") or [{}])[0]
check(code == 200 and "deepseek-flash" in ids and first.get("api") is True
and "context" in first and "label" in first, "model list: %s" % ids[:5])
# 3. config: validation and persistence
check(call(base, "/api/config", {"effort": "bogus"})[0] == 400, "a bad effort is a 400")
check(call(base, "/api/config", {"provider": "fakeb"})[0] == 400, "provider via /api/config is a 400")
check(call(base, "/api/config", {"model": ""})[0] == 400, "an empty model is a 400")
code, d, _ = call(base, "/api/config", {"model": "deepseek-v4-pro", "effort": "high"})
check(code == 200 and d.get("ok"), "model + effort accepted")
code, s, _ = call(base, "/api/state")
check(s.get("model") == "deepseek-v4-pro" and s.get("effort") == "high", "the state has them")
check(s.get("effort_applies") is True and s.get("key_set") is True and "model_info" in s
and s.get("perm_timeout", 0) > 0 and "turn_usage" in s, "the state has the new fields")
st = open(cfgdir + "/settings.json").read() if os.path.exists(cfgdir + "/settings.json") else ""
check("deepseek-v4-pro" in st and "high" in st, "settings.json remembers model and effort")
# 4. switching provider: no key, a wrong key, a good key
code, d, _ = call(base, "/api/provider", {"name": "fakeb"})
check(code == 400 and d.get("need_key"), "fakeb without a key: need_key (%s)" % code)
code, d, rawp = call(base, "/api/provider", {"name": "fakeb", "key": "sk-bad"})
check(code == 401 and "拒绝" in d.get("error", ""), "a wrong key is rejected (%s %s)" % (code, d.get("error")))
check(not leaked(rawp), "no key in the rejection")
check(call(base, "/api/state")[1].get("provider") == "deepseek", "still on deepseek")
check(call(base, "/api/provider", {"name": "nope"})[0] == 404, "an unknown provider is a 404")
code, d, rawp = call(base, "/api/provider", {"name": "fakeb", "key": "sk-good"})
check(code == 200 and d.get("provider") == "fakeb" and d.get("model") == "fake-vis",
"switched to fakeb, first listed model picked (%s %s)" % (code, d))
check(not leaked(rawp), "no key in the switch reply")
creds = cfgdir + "/credentials"
cr = open(creds).read() if os.path.exists(creds) else ""
check("fakeb.api_key=sk-good" in cr, "the key was saved to credentials")
if os.name == "posix" and os.path.exists(creds):
check(stat.S_IMODE(os.stat(creds).st_mode) == 0o600, "credentials is 0600")
code, d, _ = call(base, "/api/models")
ms = {m["id"]: m for m in d.get("models", [])}
check("fake-vis" in ms and ms["fake-vis"].get("vision") is True and ms["fake-vis"].get("context") == 200000
and ms.get("fake-text", {}).get("vision") is False, "fakeb models with capabilities")
code, d, _ = call(base, "/api/config", {"effort": "low"})
check(code == 200 and d.get("warning"), "effort on a model without it: saved with a warning")
# 5. a turn now goes to the new base URL with the new key
n0 = len(B_REQS)
code, _, _ = call(base, "/api/turn", {"text": "hi B"})
check(code == 202, "turn on fakeb accepted")
wait(lambda: len(a.events("turn_end")) >= 2, 20)
check(len(B_REQS) > n0 and B_REQS[-1]["auth"] == "Bearer sk-good" and B_REQS[-1]["model"] == "fake-vis",
"the request reached fakeb: %s" % (B_REQS[-1:] or "none"))
# 6. replacing the key
code, d, _ = call(base, "/api/key", {"key": "sk-bad"})
check(code == 401, "a wrong replacement key is rejected (%s)" % code)
code, d, _ = call(base, "/api/key", {"key": "sk-good2"})
check(code == 200 and d.get("ok"), "the replacement key is accepted")
check("fakeb.api_key=sk-good2" in open(creds).read(), "the replacement key was saved")
check(call(base, "/api/key", {})[0] == 400, "an empty key body is a 400")
a.close()
stop(p)
# 7. the next start remembers provider, model and key
p, base = start(["--new"], "second")
code, s, raws = call(base, "/api/state")
check(s.get("provider") == "fakeb" and s.get("model") == "fake-vis" and s.get("key_set") is True
and s.get("key_from") == "saved", "restart: %s/%s key %s" % (s.get("provider"), s.get("model"), s.get("key_from")))
check(not leaked(raws), "no key in /api/state")
stop(p)
logs = b"".join(open(lg.name, "rb").read() for _, lg in procs)
check(not leaked(logs.replace(b"sk-x", b"")) , "no key in the server logs")
finally:
for p, log in procs:
if p.poll() is None:
p.kill()
log.close()
shutil.rmtree(ws, ignore_errors=True)
shutil.rmtree(home, ignore_errors=True)
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+206
View File
@@ -0,0 +1,206 @@
"""WebUI workspace views: git diff, run settings, saved permission rules.
python3 scripts/e2e_webui_workspace.py (CST_AGENT= picks the binary)
Checks: /api/diff lists git status and the working-tree diff (per file too),
an option-looking path is only ever a path, and a directory without git
says so; /api/settings reports protocol, tools, confirmations and the
sandbox, rejects a bad body without applying any of it, applies a good one,
and is a 409 while a turn runs; /api/permissions shows the saved rules and
DELETE clears them. HOME points at a temp dir.
"""
import json, os, shutil, subprocess, sys, tempfile, threading, time, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BIN = os.environ.get("CST_AGENT", os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "cst-agent"))
class Api(BaseHTTPRequestHandler):
"""A slow plain answer, so a turn is running for a while."""
def log_message(self, *a):
pass
def do_POST(self):
self.rfile.read(int(self.headers.get("Content-Length", 0)))
time.sleep(2)
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
for o in ({"delta": {"role": "assistant", "content": "好"}}, {"delta": {}, "finish_reason": "stop"}):
chunk = {"id": "x", "object": "chat.completion.chunk", "model": "m", "choices": [dict(index=0, **o)]}
self.wfile.write(b"data: " + json.dumps(chunk, ensure_ascii=False).encode() + b"\n\n")
self.wfile.write(b"data: [DONE]\n\n")
fails = 0
def check(ok, msg):
global fails
print(("PASS " if ok else "FAIL ") + msg)
fails += 0 if ok else 1
def wait(fn, t=15):
end = time.time() + t
while time.time() < end:
v = fn()
if v:
return v
time.sleep(0.1)
return None
def call(base, path, obj=None, method=None):
data = None if obj is None else json.dumps(obj).encode()
req = urllib.request.Request(base + path, data=data, method=method or ("POST" if data else "GET"),
headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=30) as r:
return r.status, json.loads(r.read() or b"{}")
except urllib.error.HTTPError as e:
raw = e.read()
try:
return e.code, json.loads(raw or b"{}")
except ValueError:
return e.code, {}
def git(*args):
return subprocess.run(["git", "-C", ws] + list(args), capture_output=True, text=True)
api = ThreadingHTTPServer(("127.0.0.1", 0), Api)
threading.Thread(target=api.serve_forever, daemon=True).start()
ws = tempfile.mkdtemp(prefix="cst-ws-")
plain = tempfile.mkdtemp(prefix="cst-nogit-")
home = tempfile.mkdtemp(prefix="cst-home-")
env = dict(os.environ, HOME=home, XDG_CONFIG_HOME=home + "/.config",
XDG_DATA_HOME=home + "/.local/share", XDG_CACHE_HOME=home + "/.cache",
GIT_CONFIG_NOSYSTEM="1")
for k in ("DEEPSEEK_API_KEY", "OPENAI_API_KEY", "CST_API_KEY"):
env.pop(k, None)
have_git = shutil.which("git") is not None
if have_git:
git("init", "-q")
git("config", "user.email", "e2e@example.com")
git("config", "user.name", "e2e")
open(ws + "/a.txt", "w").write("one\n")
git("add", "a.txt")
git("commit", "-q", "-m", "first")
open(ws + "/a.txt", "w").write("one\n第二行\n")
open(ws + "/new.txt", "w").write("new\n")
os.makedirs(ws + "/.agent", exist_ok=True)
json.dump({"tools": ["write_file"], "shell_prefixes": ["make"]}, open(ws + "/.agent/permissions.json", "w"))
log = open(home + "/server.log", "w+")
proc = subprocess.Popen([BIN, "--web", "--web-port", "0", "--workspace", ws, "--new", "--api-key", "sk-x",
"--provider", "deepseek", "--model", "deepseek-flash",
"--api-base", "http://127.0.0.1:%d/v1" % api.server_address[1],
"--protocol", "native-eos", "--approval", "auto"],
stdout=log, stderr=subprocess.STDOUT, env=env)
def port():
log.seek(0)
for l in log.read().splitlines():
if l.startswith("webui: http://127.0.0.1:"):
return l.split(":")[3].split("/")[0]
try:
base = "http://127.0.0.1:%s" % wait(port)
# 1. diff
if have_git:
code, d = call(base, "/api/diff")
files = {f["path"]: f["status"] for f in d.get("files", [])}
check(code == 200 and d.get("git") is True, "diff: git repo seen")
check(files.get("a.txt") == " M" and files.get("new.txt") == "??", "status lists both files: %s" % files)
check("+第二行" in d.get("diff", "") and "a.txt" in d.get("diff", ""), "the diff has the change")
code, d = call(base, "/api/diff?path=a.txt")
check(code == 200 and "+第二行" in d.get("diff", "") and d.get("path") == "a.txt", "diff of one file")
code, d = call(base, "/api/diff?path=new.txt")
check(code == 200 and d.get("diff") == "", "an untracked file has no diff")
evil = ws + "/pwned"
code, d = call(base, "/api/diff?path=" + urllib.request.quote("--output=" + evil))
check(code == 200 and not os.path.exists(evil), "an option-looking path stays a path")
else:
print("SKIP diff checks: no git")
# 2. settings: what is there
code, s = call(base, "/api/settings")
names = [t["name"] for t in s.get("tools", [])]
check(code == 200 and s.get("protocol") == "native-eos" and len(s.get("protocols", [])) == 5,
"settings: protocol %s of %s" % (s.get("protocol"), s.get("protocols")))
check("read_file" in names and all("perm" in t and "on" in t for t in s["tools"]), "tool list: %d tools" % len(names))
check(set(s.get("confirm", {})) == {"shell", "write", "net"}, "confirm flags present")
check(isinstance(s.get("sandbox", {}).get("kind"), str), "sandbox kind: %s" % s.get("sandbox"))
victim = next((t["name"] for t in s["tools"] if t["perm"] == "net"), names[-1])
# 3. a bad body changes nothing
for body, why in (({"protocol": "nope"}, "unknown protocol"), ({"max_steps": 0}, "max_steps 0"),
({"max_steps": "5"}, "max_steps as text"), ({"tool_profile": "huge"}, "unknown profile"),
({"tools": {"nope": False}}, "unknown tool"), ({"tools": {victim: "off"}}, "tool value not bool"),
({"confirm": {"disk": True}}, "unknown confirm key"),
({"max_steps": 9, "protocol": "nope"}, "mixed good and bad")):
check(call(base, "/api/settings", body)[0] == 400, "400 for %s" % why)
check(call(base, "/api/settings")[1].get("max_steps") == s.get("max_steps"), "nothing applied from a bad body")
# 4. a good body applies
code, d = call(base, "/api/settings", {"protocol": "ctl", "max_steps": 7, "tool_profile": "minimal",
"tools": {victim: False}, "confirm": {"shell": True, "net": False}})
check(code == 200 and d.get("ok"), "settings accepted (%s %s)" % (code, d))
code, s2 = call(base, "/api/settings")
t2 = {t["name"]: t for t in s2.get("tools", [])}
check(s2.get("protocol") == "ctl" and s2.get("max_steps") == 7 and s2.get("tool_profile") == "minimal",
"protocol/steps/profile applied")
check(t2.get(victim, {}).get("off") is True and t2[victim]["on"] is False, "%s switched off" % victim)
check(s2["confirm"]["shell"] is True and s2["confirm"]["net"] is False, "confirm flags applied")
st = call(base, "/api/state")[1]
check(st.get("protocol_tools") == "ctl" and st.get("max_steps") == 7 and st.get("tool_profile") == "minimal",
"the state shows them")
call(base, "/api/settings", {"protocol": "native-eos", "tool_profile": "default", "tools": {victim: True}})
t3 = {t["name"]: t for t in call(base, "/api/settings")[1]["tools"]}
check(t3[victim]["off"] is False and t3[victim]["on"] is True, "%s back on" % victim)
# 5. permissions
code, p = call(base, "/api/permissions")
check(code == 200 and p.get("tools") == ["write_file"] and p.get("shell_prefixes") == ["make"]
and "write_file" in p.get("rules", ""), "saved rules: %s %s" % (p.get("tools"), p.get("shell_prefixes")))
code, _ = call(base, "/api/permissions", method="DELETE")
check(code == 200, "DELETE /api/permissions")
p = call(base, "/api/permissions")[1]
check(p.get("tools") == [] and p.get("shell_prefixes") == [], "the rules are gone")
saved = json.load(open(ws + "/.agent/permissions.json"))
check(saved.get("tools") == [] and saved.get("shell_prefixes") == [], "and gone from the file")
# 6. while a turn runs: settings and the rule reset are 409, reads still work
code, _ = call(base, "/api/turn", {"text": "慢慢来"})
check(code == 202, "a slow turn started")
check(call(base, "/api/settings", {"max_steps": 3})[0] == 409, "settings during a turn: 409")
check(call(base, "/api/permissions", method="DELETE")[0] == 409, "rule reset during a turn: 409")
check(call(base, "/api/settings")[0] == 200 and call(base, "/api/permissions")[0] == 200,
"reads during a turn: 200")
if have_git:
check(call(base, "/api/diff")[0] == 200, "diff during a turn: 200")
wait(lambda: not call(base, "/api/state")[1].get("status_busy"), 20)
# 7. a directory without git
if have_git:
code, _ = call(base, "/api/workspace", {"path": plain})
code, d = call(base, "/api/diff")
check(code == 200 and d.get("git") is False and "diff" not in d, "no git repo: git=false")
finally:
proc.terminate()
try:
proc.wait(20)
except subprocess.TimeoutExpired:
proc.kill()
check(False, "server hung on SIGTERM")
log.close()
for d in (ws, plain, home):
shutil.rmtree(d, ignore_errors=True)
api.shutdown()
print("FAILED:", fails)
sys.exit(1 if fails else 0)
+40
View File
@@ -0,0 +1,40 @@
#!/bin/sh
# Embed web/ into .gen/web_assets.inc as C byte arrays (the same trick the
# Makefile uses for src/models_builtin.json). POSIX sh on purpose: the musl
# and busybox build environments have no bash. Usage:
# sh scripts/embed_web.sh [out.inc] [web-dir]
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
OUT="${1:-$ROOT/.gen/web_assets.inc}"
DIR="${2:-$ROOT/web}"
FILES="index.html app.js style.css favicon.svg"
mkdir -p "$(dirname "$OUT")"
{
echo "/* Generated by scripts/embed_web.sh from web/ - do not edit. */"
for f in $FILES; do
[ -f "$DIR/$f" ] || continue
var="wui_asset_$(echo "$f" | tr -c 'a-zA-Z0-9' '_')"
echo "static const unsigned char ${var}[] = {"
od -An -v -tx1 "$DIR/$f" | sed 's/\([0-9a-f][0-9a-f]\)/0x\1,/g'
echo "0x00};"
done
echo "static const wui_asset_t wui_assets[] = {"
for f in $FILES; do
[ -f "$DIR/$f" ] || continue
var="wui_asset_$(echo "$f" | tr -c 'a-zA-Z0-9' '_')"
case "$f" in
*.html) ctype="text/html; charset=utf-8" ;;
*.js) ctype="text/javascript; charset=utf-8" ;;
*.css) ctype="text/css; charset=utf-8" ;;
*.svg) ctype="image/svg+xml" ;;
*) ctype="application/octet-stream" ;;
esac
echo " {\"$f\", \"$ctype\", ${var}, sizeof(${var}) - 1},"
done
echo "};"
echo "static const int wui_asset_count = (int)(sizeof(wui_assets) / sizeof(wui_assets[0]));"
} > "$OUT.tmp"
mv "$OUT.tmp" "$OUT"
echo "embedded $(wc -c < "$OUT") bytes of C into $OUT"
+106
View File
@@ -0,0 +1,106 @@
# Install or update CST-Agent on Windows from a Gitea release.
#
# irm https://git.whcself.xyz/WHC/CST-agent/raw/branch/main/scripts/install_windows.ps1 | iex
# powershell -ExecutionPolicy Bypass -File install_windows.ps1 [-Version 0.6.0] [-Prefix DIR] [-Zip FILE] [-NoPath]
#
# Installs into %LOCALAPPDATA%\Programs\cst-agent (no admin rights needed),
# adds it to the user PATH and leaves cst-agent-update.cmd beside the exe,
# which reruns this script for that folder. Environment overrides:
# CST_VERSION, CST_PREFIX, CST_GITEA (server URL), CST_REPO (owner/name).
# Needs Windows 10 1903+ / Server 2022+ (UTF-8 process code page).
# Keep this file ASCII: Windows PowerShell 5.1 reads BOM-less scripts in the
# ANSI code page.
param(
[string]$Version = $env:CST_VERSION,
[string]$Prefix = $env:CST_PREFIX,
[string]$Zip = "",
[switch]$NoPath
)
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$Server = if ($env:CST_GITEA) { $env:CST_GITEA.TrimEnd("/") } else { "https://git.whcself.xyz" }
$Repo = if ($env:CST_REPO) { $env:CST_REPO } else { "WHC/CST-agent" }
if (-not $Prefix) { $Prefix = Join-Path $env:LOCALAPPDATA "Programs\cst-agent" }
$Prefix = [IO.Path]::GetFullPath($Prefix)
$build = [Environment]::OSVersion.Version.Build
if ($build -lt 18362) {
throw "CST-Agent needs Windows 10 1903 (build 18362) or newer; this is build $build."
}
$work = Join-Path ([IO.Path]::GetTempPath()) ("cst-agent-install-" + [Guid]::NewGuid().ToString("N"))
New-Item -ItemType Directory -Force $work | Out-Null
try {
if (-not $Zip) {
$api = "$Server/api/v1/repos/$Repo/releases"
$url = if (-not $Version -or $Version -eq "latest") { "$api/latest" } else { "$api/tags/v$($Version.TrimStart('v'))" }
Write-Host "==> $url"
try {
$rel = Invoke-RestMethod -Uri $url -UseBasicParsing
} catch {
throw "no release found at $url ($($_.Exception.Message))"
}
$asset = $rel.assets | Where-Object { $_.name -like "*-windows-x86_64.zip" } | Select-Object -First 1
if (-not $asset) { throw "release $($rel.tag_name) has no *-windows-x86_64.zip" }
$Zip = Join-Path $work $asset.name
Write-Host "==> download $($asset.name) ($($rel.tag_name))"
Invoke-WebRequest -Uri $asset.browser_download_url -OutFile $Zip -UseBasicParsing
$sum = $rel.assets | Where-Object { $_.name -eq "$($asset.name).sha256" } | Select-Object -First 1
if ($sum) {
$sumFile = Join-Path $work $sum.name
Invoke-WebRequest -Uri $sum.browser_download_url -OutFile $sumFile -UseBasicParsing
$want = ((Get-Content $sumFile -Raw).Trim() -split "\s+")[0].ToLower()
$got = (Get-FileHash -Algorithm SHA256 $Zip).Hash.ToLower()
if ($want -ne $got) { throw "sha256 mismatch for $($asset.name): expected $want, got $got" }
Write-Host "==> sha256 ok"
}
}
Expand-Archive -Path $Zip -DestinationPath $work -Force
$src = Join-Path $work "windows-x86_64"
if (-not (Test-Path (Join-Path $src "cst-agent.exe"))) { throw "$Zip has no windows-x86_64\cst-agent.exe" }
New-Item -ItemType Directory -Force $Prefix | Out-Null
# A running exe cannot be overwritten but can be renamed out of the way.
foreach ($name in @("cst-agent.exe", "busybox.exe")) {
$dst = Join-Path $Prefix $name
$old = "$dst.old"
if (Test-Path $old) { Remove-Item -Force $old -ErrorAction SilentlyContinue }
if (Test-Path $dst) {
try { Remove-Item -Force $dst } catch { Rename-Item -Force $dst "$name.old" }
}
}
Copy-Item -Force -Recurse (Join-Path $src "*") $Prefix
$update = Join-Path $Prefix "cst-agent-update.cmd"
if (-not (Test-Path $update)) {
Set-Content -Encoding Ascii -Path $update -Value @(
"@echo off",
"rem Update CST-Agent in this folder from the latest release.",
"powershell -NoProfile -ExecutionPolicy Bypass -File `"%~dp0install_windows.ps1`" -Prefix `"%~dp0.`" %*"
)
}
if (-not (Test-Path (Join-Path $Prefix "install_windows.ps1")) -and $PSCommandPath) {
Copy-Item -Force $PSCommandPath (Join-Path $Prefix "install_windows.ps1")
}
if (-not $NoPath) {
$userPath = [Environment]::GetEnvironmentVariable("Path", "User")
$parts = @($userPath -split ";" | Where-Object { $_ })
if (-not ($parts | Where-Object { $_.TrimEnd("\") -ieq $Prefix.TrimEnd("\") })) {
[Environment]::SetEnvironmentVariable("Path", (($parts + $Prefix) -join ";"), "User")
Write-Host "==> added $Prefix to the user PATH (open a new terminal to use it)"
}
if (-not (($env:Path -split ";") | Where-Object { $_.TrimEnd("\") -ieq $Prefix.TrimEnd("\") })) {
$env:Path = "$env:Path;$Prefix"
}
}
$ver = (& (Join-Path $Prefix "cst-agent.exe") --version) 2>&1 | Select-Object -First 1
Write-Host "==> installed $ver in $Prefix"
Write-Host "Run 'cst-agent' inside a project folder; update later with 'cst-agent-update'."
} finally {
Remove-Item -Recurse -Force $work -ErrorAction SilentlyContinue
}
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Static musl build of cst-agent for one architecture, packed as
# dist/cst-agent-VERSION-linux-ARCH.tar.gz (+ .sha256).
# scripts/package_linux.sh x86_64|x86|aarch64
# CST_LINUX_CHECK=1 also runs `make check` (only where the binaries can run).
# Builds in the source tree, so the tree is cleaned first.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
ARCH="${1:?usage: $0 x86_64|x86|aarch64}"
PREFIX="$ROOT/third_party/prefix-linux-$ARCH"
cd "$ROOT"
read -r LINUX_HOST bindir < <(bash scripts/ci_linux_toolchain.sh "$ARCH")
export PATH="$bindir:$PATH" LINUX_HOST
T="$LINUX_HOST-"
if [[ ! -f "$PREFIX/lib/libcurl.a" || ! -f "$PREFIX/lib/libncursesw.a" ]]; then
bash scripts/build_third_party_linux.sh "$ARCH"
fi
# Only the static prefix: no host .pc file may leak into the link.
export PKG_CONFIG_LIBDIR="$PREFIX/lib/pkgconfig"
unset PKG_CONFIG_PATH CC CFLAGS LDFLAGS
mk=(make CC="${T}gcc" AR="${T}ar" STATIC=1
CFLAGS="-std=c11 -Wall -Wextra -O2 -g -Werror")
make clean >/dev/null
make -C vendor/sashidown clean >/dev/null 2>&1 || true
"${mk[@]}" -j"$(nproc)" cst-agent
if "${T}readelf" -l cst-agent | grep -q 'Requesting program interpreter'; then
echo "cst-agent is dynamically linked" >&2
exit 1
fi
if [[ "${CST_LINUX_CHECK:-0}" == 1 ]]; then
"${mk[@]}" check
fi
ver="$(sed -n 's/^#define CST_VERSION_\(MAJOR\|MINOR\|PATCH\) \([0-9]*\)$/\2/p' include/cst_version.h | paste -sd.)"
name="cst-agent-$ver-linux-$ARCH"
rel="dist/$name"
rm -rf "$rel"
mkdir -p "$rel"
cp -f cst-agent "$rel/cst-agent"
"${T}strip" "$rel/cst-agent"
cp -f README.md "$rel/README.md"
echo "$ver" >"$rel/VERSION"
tgz="dist/$name.tar.gz"
tar -C dist -czf "$tgz" "$name"
(cd dist && sha256sum "$name.tar.gz" >"$name.tar.gz.sha256")
ls -la "$rel"
echo "$ROOT/$tgz"
+42 -7
View File
@@ -1,18 +1,53 @@
#!/usr/bin/env bash
# Assemble dist/windows-x86_64 from MinGW TUI build + busybox.
# Assemble dist/windows-x86_64/ and dist/cst-agent-VERSION-windows-x86_64.zip
# from the cross build (build-win-tui, else build-win; CST_WIN_BUILD=DIR
# picks another, e.g. build-msys2) and busybox-w32.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
REL="$ROOT/dist/windows-x86_64"
EXE="$ROOT/build-win-tui/cst-agent.exe"
[[ -f "$EXE" ]] || EXE="$ROOT/build-win/cst-agent.exe"
if [[ -n "${CST_WIN_BUILD:-}" ]]; then
EXE="$ROOT/$CST_WIN_BUILD/cst-agent.exe"
else
EXE="$ROOT/build-win-tui/cst-agent.exe"
[[ -f "$EXE" ]] || EXE="$ROOT/build-win/cst-agent.exe"
fi
BB="$ROOT/third_party/busybox/busybox.exe"
DLL="/usr/x86_64-w64-mingw32/bin/libwinpthread-1.dll"
[[ -f "$EXE" ]] || { echo "no cst-agent.exe; build first" >&2; exit 1; }
[[ -f "$BB" ]] || { echo "no busybox.exe; scripts/build_third_party_mingw.sh busybox-w32" >&2; exit 1; }
ver="$(sed -n 's/^#define CST_VERSION_\(MAJOR\|MINOR\|PATCH\) \([0-9]*\)$/\2/p' "$ROOT/include/cst_version.h" | paste -sd.)"
strip="${MINGW_HOST-x86_64-w64-mingw32}"
strip="${strip:+$strip-}strip"
command -v "$strip" >/dev/null || strip=strip # MSYS2: unprefixed
rm -rf "$REL"
mkdir -p "$REL"
cp -f "$EXE" "$REL/cst-agent.exe"
"$strip" "$REL/cst-agent.exe"
cp -f "$BB" "$REL/busybox.exe"
cp -f "$ROOT/third_party/busybox/LICENSE.busybox" "$REL/"
cp -f "$ROOT/docs/README-windows.md" "$REL/"
[[ -f "$DLL" ]] && cp -f "$DLL" "$REL/"
cp -f "$ROOT/docs/README-windows.md" "$REL/README.md"
cp -f "$ROOT/scripts/windows/cst-agent-update.cmd" "$ROOT/scripts/install_windows.ps1" "$REL/"
echo "$ver" >"$REL/VERSION"
zip="$ROOT/dist/cst-agent-$ver-windows-x86_64.zip"
rm -f "$zip"
(cd "$ROOT/dist" && zip -qr "$zip" windows-x86_64)
(cd "$ROOT/dist" && sha256sum "$(basename "$zip")" >"$(basename "$zip").sha256")
ls -la "$REL"
file "$REL/cst-agent.exe"
echo "$zip"
# NSIS installer from the same folder (CST_REQUIRE_NSIS=1: fail without makensis).
setup="$ROOT/dist/cst-agent-$ver-windows-x86_64-setup.exe"
if command -v makensis >/dev/null; then
rm -f "$setup"
makensis -V2 -WX -INPUTCHARSET UTF8 -DVERSION="$ver" -DSRCDIR="$REL" \
-DOUTFILE="$setup" "$ROOT/scripts/windows/cst-agent.nsi"
(cd "$ROOT/dist" && sha256sum "$(basename "$setup")" >"$(basename "$setup").sha256")
echo "$setup"
elif [[ "${CST_REQUIRE_NSIS:-0}" == 1 ]]; then
echo "makensis not found" >&2
exit 1
else
echo "makensis not found; skipping the installer" >&2
fi
@@ -0,0 +1,350 @@
--- a/ncurses/curses.priv.h
+++ b/ncurses/curses.priv.h
@@ -2294,7 +2294,10 @@
#if defined(_NC_WINDOWS_NATIVE) && !defined(_NC_MSC) && !USE_NAMED_PIPES
/*
* MinGW has wide-character functions, but they do not work correctly.
+ * Declare the originals first so a later <wchar.h> is not macro-expanded.
*/
+#include <stdlib.h>
+#include <wchar.h>
extern int __MINGW_NOTHROW _nc_wctomb(char *, wchar_t);
#define wctomb(s,wc) _nc_wctomb(s,wc)
#define wcrtomb(s,wc,n) _nc_wctomb(s,wc)
--- a/ncurses/tinfo/lib_win32con.c
+++ b/ncurses/tinfo/lib_win32con.c
@@ -653,6 +653,21 @@
return result;
}
+/* Whether handle_mouse() turns this record into an event: wheel, a button
+ * change (press or release), or a drag. */
+static bool
+mouse_wanted(const SCREEN *sp, const MOUSE_EVENT_RECORD *mer)
+{
+ int held = (int) (mer->dwButtonState & BUTTON_MASK);
+
+ if (mer->dwEventFlags & MOUSE_WHEELED)
+ return TRUE;
+ if (held != sp->_drv_mouse_new_buttons)
+ return TRUE;
+ return (mer->dwEventFlags & MOUSE_MOVED) && held
+ && (sp->_mouse_mask & REPORT_MOUSE_POSITION);
+}
+
#define AdjustY() (WINCONSOLE.buffered \
? 0 \
: (int) WINCONSOLE.SBI.srWindow.Top)
@@ -661,36 +676,47 @@
handle_mouse(SCREEN *sp, MOUSE_EVENT_RECORD mer)
{
MEVENT work;
- bool result = FALSE;
+ int old_buttons;
+ int new_buttons;
assert(sp);
- sp->_drv_mouse_old_buttons = sp->_drv_mouse_new_buttons;
- sp->_drv_mouse_new_buttons = mer.dwButtonState & BUTTON_MASK;
+ if (sp->_drv_mouse_tail >= FIFO_SIZE)
+ return FALSE;
- /*
- * We're only interested if the button is pressed or released.
- * FIXME: implement continuous event-tracking.
- */
- if (sp->_drv_mouse_new_buttons != sp->_drv_mouse_old_buttons) {
- memset(&work, 0, sizeof(work));
+ memset(&work, 0, sizeof(work));
+ work.x = mer.dwMousePosition.X;
+ work.y = mer.dwMousePosition.Y - AdjustY();
+
+ /* the wheel reports like xterm: button 4 up, button 5 down */
+ if (mer.dwEventFlags & MOUSE_WHEELED) {
+ work.bstate = ((SHORT) HIWORD(mer.dwButtonState) > 0)
+ ? BUTTON4_PRESSED
+ : BUTTON5_PRESSED;
+ sp->_drv_mouse_fifo[sp->_drv_mouse_tail++] = work;
+ return TRUE;
+ }
- if (sp->_drv_mouse_new_buttons) {
- work.bstate |= decode_mouse(sp, sp->_drv_mouse_new_buttons);
+ old_buttons = sp->_drv_mouse_new_buttons;
+ new_buttons = (int) (mer.dwButtonState & BUTTON_MASK);
+ sp->_drv_mouse_old_buttons = old_buttons;
+ sp->_drv_mouse_new_buttons = new_buttons;
+
+ if (new_buttons != old_buttons) {
+ if (new_buttons & ~old_buttons) {
+ work.bstate = decode_mouse(sp, new_buttons & ~old_buttons);
} else {
/* cf: BUTTON_PRESSED, BUTTON_RELEASED */
- work.bstate |= (decode_mouse(sp, sp->_drv_mouse_old_buttons)
- >> 1);
- result = TRUE;
+ work.bstate = decode_mouse(sp, old_buttons & ~new_buttons) >> 1;
}
-
- work.x = mer.dwMousePosition.X;
- work.y = mer.dwMousePosition.Y - AdjustY();
-
- sp->_drv_mouse_fifo[sp->_drv_mouse_tail] = work;
- sp->_drv_mouse_tail += 1;
+ } else if ((mer.dwEventFlags & MOUSE_MOVED) && new_buttons
+ && (sp->_mouse_mask & REPORT_MOUSE_POSITION)) {
+ work.bstate = REPORT_MOUSE_POSITION; /* drag */
+ } else {
+ return FALSE;
}
- return result;
+ sp->_drv_mouse_fifo[sp->_drv_mouse_tail++] = work;
+ return TRUE;
}
static int
@@ -813,6 +839,13 @@
returnCode(found);
}
+#if USE_WIDEC_SUPPORT
+/* UTF-8 bytes of the last non-ASCII key event not yet read */
+static int utf8_pend[4];
+static int utf8_len, utf8_pos;
+static unsigned utf8_high;
+#endif
+
NCURSES_EXPORT(int)
_nc_console_twait(
const SCREEN *sp,
@@ -845,6 +878,14 @@
TR(TRACE_IEVENT, ("start twait: hdl=%p, %d milliseconds, mode: %d",
hdl, milliseconds, mode));
+#if USE_WIDEC_SUPPORT
+ if ((mode & TW_INPUT) && utf8_pos < utf8_len) {
+ if (timeleft)
+ *timeleft = milliseconds;
+ return TW_INPUT;
+ }
+#endif
+
if (milliseconds < 0)
milliseconds = NC_INFINITY;
@@ -940,9 +981,7 @@
continue;
case MOUSE_EVENT:
T(("twait:event MOUSE_EVENT"));
- if (decode_mouse(sp,
- (inp_rec.Event.MouseEvent.dwButtonState
- & BUTTON_MASK)) == 0) {
+ if (!mouse_wanted(sp, &inp_rec.Event.MouseEvent)) {
CONSUME();
} else if (mode & TW_MOUSE) {
code = TW_MOUSE;
@@ -1024,6 +1063,16 @@
T((T_CALLED("lib_win32con::_nc_console_read(%p)"), sp));
+#if USE_WIDEC_SUPPORT
+ /* wget_wch() decodes the input as a byte stream in the locale's
+ * multibyte encoding (UTF-8), so non-ASCII UTF-16 key events are
+ * handed over one UTF-8 byte per call. */
+ if (utf8_pos < utf8_len) {
+ *buf = utf8_pend[utf8_pos++];
+ returnCode(1);
+ }
+#endif
+
while ((b = read_keycode(hdl, &inp_rec, 1, &nRead))) {
if (b && nRead > 0) {
if (rc < 0)
@@ -1034,6 +1083,36 @@
continue;
*buf = (int) inp_rec.Event.KeyEventChar;
vk = inp_rec.Event.KeyEvent.wVirtualKeyCode;
+#if USE_WIDEC_SUPPORT
+ if (*buf >= 0x80) {
+ unsigned cp = (unsigned) *buf;
+ if (cp >= 0xD800 && cp < 0xDC00) {
+ utf8_high = cp;
+ continue;
+ }
+ if (cp >= 0xDC00 && cp < 0xE000) {
+ if (!utf8_high)
+ continue;
+ cp = 0x10000 + ((utf8_high - 0xD800) << 10) + (cp - 0xDC00);
+ }
+ utf8_high = 0;
+ utf8_len = 0;
+ if (cp < 0x800) {
+ utf8_pend[utf8_len++] = (int) (0xC0 | (cp >> 6));
+ } else if (cp < 0x10000) {
+ utf8_pend[utf8_len++] = (int) (0xE0 | (cp >> 12));
+ utf8_pend[utf8_len++] = (int) (0x80 | ((cp >> 6) & 0x3F));
+ } else {
+ utf8_pend[utf8_len++] = (int) (0xF0 | (cp >> 18));
+ utf8_pend[utf8_len++] = (int) (0x80 | ((cp >> 12) & 0x3F));
+ utf8_pend[utf8_len++] = (int) (0x80 | ((cp >> 6) & 0x3F));
+ }
+ utf8_pend[utf8_len++] = (int) (0x80 | (cp & 0x3F));
+ *buf = utf8_pend[0];
+ utf8_pos = 1;
+ break;
+ }
+#endif
/*
* There are 24 virtual function-keys, and typically
* 12 function-keys on a keyboard. Use the shift-modifier
--- a/ncurses/widechar/widechars.c
+++ b/ncurses/widechar/widechars.c
@@ -97,20 +97,10 @@
wchar_t temp;
if (s != NULL && n != 0) {
+ /* _nc_mbtowc() returns the bytes used, not a character count */
count = _nc_mbtowc(&temp, s, n);
- if (count == 1) {
- int check = WideCharToMultiByte(CP_UTF8,
- 0,
- &temp,
- 1,
- NULL,
- 0, /* compute length only */
- NULL,
- NULL);
- TR(TRACE_BITS, ("\tcheck ->%d\n", check));
- if (check > 0 && (size_t) check <= n) {
- result = check;
- }
+ if (count > 0 && (size_t) count <= n) {
+ result = count;
}
} else {
result = 0;
--- a/ncurses/win32con/win32_driver.c
+++ b/ncurses/win32con/win32_driver.c
@@ -87,8 +87,13 @@
if (ch & A_BOLD)
res |= FOREGROUND_INTENSITY;
- if (ch & A_DIM)
- res |= BACKGROUND_INTENSITY;
+ /* dim lowers the foreground: bright -> normal, light gray -> dark gray */
+ if ((ch & A_DIM) && !(ch & A_BOLD)) {
+ if (res & FOREGROUND_INTENSITY)
+ res &= ~FOREGROUND_INTENSITY;
+ else if ((res & 7) == 7)
+ res = (res & ~7) | FOREGROUND_INTENSITY;
+ }
return res;
}
@@ -159,7 +164,6 @@
int y, int x, cchar_t *str, int limit)
{
int actual = 0;
- MakeArray(ci, CHAR_INFO, limit);
COORD loc, siz;
SMALL_RECT rec;
int i;
@@ -169,39 +173,71 @@
AssertTCB();
SetSP();
- for (i = actual = 0; i < limit; i++) {
- ch = str[i];
- if (isWidecExt(ch))
- continue;
- ci[actual].CharInfoChar = CharOf(ch);
- ci[actual].Attributes = MapAttr(WINCONSOLE.SBI.wAttributes,
- AttrOf(ch));
- if (AttrOf(ch) & A_ALTCHARSET) {
- if (_nc_wacs) {
- int which = CharOf(ch);
- if (which > 0
- && which < ACS_LEN
- && CharOf(_nc_wacs[which]) != 0) {
- ci[actual].CharInfoChar = CharOf(_nc_wacs[which]);
+ /*
+ * The console stores a double-width character as two cells flagged
+ * leading/trailing byte. Never start or end a write inside one, and
+ * emit both halves so the columns after it stay put.
+ */
+ if (limit > 0 && x > 0 && isWidecExt(str[0])) {
+ --str;
+ --x;
+ ++limit;
+ }
+ if (limit > 0 && x + limit < screen_columns(sp) && isWidecExt(str[limit]))
+ ++limit;
+
+ {
+ MakeArray(ci, CHAR_INFO, limit);
+
+ for (i = actual = 0; i < limit; i++) {
+ ch = str[i];
+ if (isWidecExt(ch)) {
+ if (actual > 0
+ && (ci[actual - 1].Attributes & COMMON_LVB_LEADING_BYTE)) {
+ ci[actual] = ci[actual - 1];
+ ci[actual].Attributes = (WORD)
+ ((ci[actual].Attributes & ~COMMON_LVB_LEADING_BYTE)
+ | COMMON_LVB_TRAILING_BYTE);
} else {
ci[actual].CharInfoChar = ' ';
+ ci[actual].Attributes = MapAttr(WINCONSOLE.SBI.wAttributes,
+ AttrOf(ch));
}
+ ++actual;
+ continue;
}
+ ci[actual].CharInfoChar = CharOf(ch);
+ ci[actual].Attributes = MapAttr(WINCONSOLE.SBI.wAttributes,
+ AttrOf(ch));
+ if (AttrOf(ch) & A_ALTCHARSET) {
+ if (_nc_wacs) {
+ int which = CharOf(ch);
+ if (which > 0
+ && which < ACS_LEN
+ && CharOf(_nc_wacs[which]) != 0) {
+ ci[actual].CharInfoChar = CharOf(_nc_wacs[which]);
+ } else {
+ ci[actual].CharInfoChar = ' ';
+ }
+ }
+ }
+ if (i + 1 < limit && isWidecExt(str[i + 1]))
+ ci[actual].Attributes |= COMMON_LVB_LEADING_BYTE;
+ ++actual;
}
- ++actual;
- }
- loc.X = (SHORT) 0;
- loc.Y = (SHORT) 0;
- siz.X = (SHORT) actual;
- siz.Y = 1;
-
- rec.Left = (SHORT) x;
- rec.Top = (SHORT) (y + AdjustY());
- rec.Right = (SHORT) (x + limit - 1);
- rec.Bottom = rec.Top;
+ loc.X = (SHORT) 0;
+ loc.Y = (SHORT) 0;
+ siz.X = (SHORT) actual;
+ siz.Y = 1;
+
+ rec.Left = (SHORT) x;
+ rec.Top = (SHORT) (y + AdjustY());
+ rec.Right = (SHORT) (x + actual - 1);
+ rec.Bottom = rec.Top;
- return write_screen(WINCONSOLE.hdl, ci, siz, loc, &rec);
+ return write_screen(WINCONSOLE.hdl, ci, siz, loc, &rec);
+ }
}
#define con_write(tcb, y, x, str, n) con_write16(tcb, y, x, str, n)
#else
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env bash
# Upload release files to the Gitea release vVERSION, creating the release
# (and its tag at HEAD) when missing.
# GITEA_TOKEN=... scripts/release.sh FILE...
# Each FILE's FILE.sha256 is uploaded too when it exists; assets of the same
# name are replaced. Several CI jobs may call this at once: the first creates
# the release, the others find it. The token is only read from the
# environment and handed to curl on stdin: it never appears in argv, on disk
# or in the output.
#
# CI: set GITEA_TOKEN (or GITHUB_TOKEN), CST_GITEA, CST_REPO. The remote-branch
# check is skipped under GITEA_ACTIONS / GITHUB_ACTIONS / CI.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
SERVER="${CST_GITEA:-${GITHUB_SERVER_URL:-https://git.whcself.xyz}}"
REPO="${CST_REPO:-${GITHUB_REPOSITORY:-WHC/CST-agent}}"
API="$SERVER/api/v1/repos/$REPO"
TOKEN="${GITEA_TOKEN:-${GITHUB_TOKEN:-}}"
[[ -n "$TOKEN" ]] || {
echo "set GITEA_TOKEN (a Gitea token with write:repository)" >&2
exit 1
}
[[ $# -gt 0 ]] || { echo "usage: $0 FILE..." >&2; exit 1; }
command -v python3 >/dev/null || { echo "missing python3" >&2; exit 1; }
files=()
for f in "$@"; do
[[ -f "$f" ]] || { echo "missing $f" >&2; exit 1; }
f="$(cd "$(dirname "$f")" && pwd)/$(basename "$f")"
files+=("$f")
[[ "$f" != *.sha256 && -f "$f.sha256" ]] && files+=("$f.sha256")
done
cd "$ROOT"
ver="$(sed -n 's/^#define CST_VERSION_\(MAJOR\|MINOR\|PATCH\) \([0-9]*\)$/\2/p' include/cst_version.h | paste -sd.)"
tag="v$ver"
# If this job was triggered by a version tag, it must match cst_version.h.
if [[ "${GITHUB_REF:-}" == refs/tags/v* ]]; then
ref_tag="${GITHUB_REF#refs/tags/}"
if [[ "$ref_tag" != "$tag" ]]; then
echo "pushed tag $ref_tag != cst_version.h $tag; bump include/cst_version.h" >&2
exit 1
fi
fi
api() { # METHOD PATH [curl args...]
local m="$1" p="$2"
shift 2
printf 'Authorization: token %s\n' "$TOKEN" |
curl -fsS -X "$m" -H @- "$API$p" "$@"
}
json() { # CODE [ARGS...]: d = the JSON on stdin, ARGS in sys.argv[1:]
local code="$1"
shift
python3 -c "import json,sys; d=json.load(sys.stdin); $code" "$@"
}
BODY="Linux (static, musl, any distro): cst-agent-$ver-linux-{x86_64,x86,aarch64}.tar.gz -- extract and run ./cst-agent, or copy it onto PATH.
Windows x86_64: cst-agent-$ver-windows-x86_64-setup.exe installs per user (no admin rights) and adds PATH; cst-agent-$ver-windows-x86_64.zip is the portable build (install_windows.ps1 inside).
Every file has a .sha256 next to it."
if rel="$(api GET "/releases/tags/$tag" 2>/dev/null)"; then
echo "==> release $tag exists"
else
head="$(git rev-parse HEAD)"
if [[ -z "${CI:-}${GITEA_ACTIONS:-}${GITHUB_ACTIONS:-}" ]]; then
if ! git branch -r --contains "$head" 2>/dev/null | grep -q .; then
echo "HEAD $head is not on any remote branch; push it first" >&2
exit 1
fi
fi
echo "==> creating release $tag at $head"
body="$(python3 -c 'import json,sys; print(json.dumps({"tag_name": sys.argv[1], "target_commitish": sys.argv[2], "name": "CST-Agent " + sys.argv[1], "body": sys.argv[3]}))' "$tag" "$head" "$BODY")"
if ! rel="$(api POST /releases -H 'Content-Type: application/json' -d "$body" 2>/dev/null)"; then
# Another job created it first (409).
rel=""
for _ in 1 2 3 4 5 6; do
sleep 5
rel="$(api GET "/releases/tags/$tag" 2>/dev/null)" && break
rel=""
done
[[ -n "$rel" ]] || { echo "could not create or find release $tag" >&2; exit 1; }
echo "==> release $tag created by another job"
fi
fi
id="$(json 'print(d["id"])' <<<"$rel")"
for f in "${files[@]}"; do
name="$(basename "$f")"
old="$(json "print(' '.join(str(a['id']) for a in d.get('assets') or [] if a['name'] == sys.argv[1]))" "$name" <<<"$rel")"
for aid in $old; do
echo "==> replacing $name"
api DELETE "/releases/$id/assets/$aid" >/dev/null
done
echo "==> upload $name"
api POST "/releases/$id/assets?name=$name" -F "attachment=@$f" >/dev/null
done
echo "$SERVER/$REPO/releases/tag/$tag"
-27
View File
@@ -1,27 +0,0 @@
@echo off
REM Run ON Windows after copying dist/windows-x86_64 (or a full git checkout + MSYS2 build).
REM This script only documents env + launch; prefer bootstrap_windows.sh on Linux for cross-build.
setlocal
if "%OPENAI_API_KEY%"=="" if "%CST_API_KEY%"=="" (
echo Set OPENAI_API_KEY or CST_API_KEY first. See docs\env_openai.md
echo Example:
echo set OPENAI_API_KEY=sk-...
echo set OPENAI_BASE_URL=https://api.openai.com/v1
echo set OPENAI_MODEL=gpt-4.1-mini
exit /b 1
)
if not exist cst-agent.exe (
echo cst-agent.exe not found in %CD%
echo Copy dist\windows-x86_64\* here, or build with MSYS2/MinGW.
exit /b 1
)
if not exist busybox.exe (
echo WARNING: busybox.exe missing — run_shell may fail. Place busybox.exe beside cst-agent.exe
)
echo Launching CST-Agent...
cst-agent.exe --workspace .
endlocal
+44
View File
@@ -0,0 +1,44 @@
import os
import sys, time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from harness import Term
import fakeapi
from keys import input_text, styled, check, FAILS, base
for term in ("xterm-256color", "screen-256color"):
t = Term(args=("-b", base, "-k", "sk-test"), term=term)
t.send("some text")
t.send("\x1b", 0.4)
check(term + " lone esc clears input", input_text(t) == [], t)
t.send("alpha beta")
t.send("\x1bb") # alt+b word left
t.send("X")
check(term + " alt+b word left", "alpha Xbeta" in input_text(t), t)
t.send("\x1b\r") # alt+enter newline
t.send("Y")
check(term + " alt+enter newline", input_text(t)[-2:] == ["alpha X", "Ybeta"], t)
t.send("\x03", 0.2)
# picker with CSI form
t.send("/model\r", 0.8)
s0 = styled(t)
t.arrow("down", "csi")
s1 = styled(t)
t.arrow("up", "csi")
s2 = styled(t)
check(term + " picker csi down", s0 != s1, t)
check(term + " picker csi up", s0 == s2, t)
t.send("\x1b", 0.4)
# home/end variants
t.send("middle")
for seq in ("\x1b[H", "\x1bOH", "\x1b[1~", "\x1b[7~"):
t.send("\x1b[F" + seq)
t.send("<")
t.send("\x08" if False else "")
check(term + " home variants", input_text(t)[-1:] == ["<<<<middle"], t)
for seq in ("\x1b[F", "\x1bOF", "\x1b[4~", "\x1b[8~"):
t.send("\x1b[H" + seq)
t.send(">")
check(term + " end variants", input_text(t)[-1:] == ["<<<<middle>>>>"], t)
t.close()
print("FAILED:", len(FAILS))
sys.exit(min(len(FAILS), 100))
+43
View File
@@ -0,0 +1,43 @@
import json, sys, threading, time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
LOG = []
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def do_GET(self):
body = json.dumps({"data": [{"id": "deepseek-flash"}, {"id": "deepseek-v4-pro"}]}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
n = int(self.headers.get("Content-Length", 0))
raw = self.rfile.read(n)
try:
LOG.append(json.loads(raw))
except Exception:
LOG.append({"_raw": raw[:200].decode("latin1")})
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
def ev(o):
self.wfile.write(b"data: " + json.dumps(o).encode() + b"\n\n")
self.wfile.flush()
base = {"id": "x", "object": "chat.completion.chunk", "model": "deepseek-flash"}
ev({**base, "choices": [{"index": 0, "delta": {"role": "assistant", "content": "reply-%d" % len(LOG)}}]})
ev({**base, "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12}})
self.wfile.write(b"data: [DONE]\n\n")
self.wfile.flush()
def start():
s = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=s.serve_forever, daemon=True).start()
return s, "http://127.0.0.1:%d/v1" % s.server_address[1]
+99
View File
@@ -0,0 +1,99 @@
"""Drive the cst-agent TUI in a pty and read the screen back (pyte).
Run from the repo root after `make`:
nix-shell -p python3Packages.pyte --run "python3 scripts/tui_keys/keys.py"
nix-shell -p python3Packages.pyte --run "python3 scripts/tui_keys/esc.py"
(or `pip install pyte`). A local fake OpenAI-compatible server answers the
model calls, so no API key is needed. Exit status is the failure count.
"""
import os, pty, sys, time, select, fcntl, termios, struct, signal, shutil, tempfile
import pyte
BIN = os.environ.get("CST_BIN") or os.path.join(
os.path.dirname(os.path.abspath(__file__)), "..", "..", "cst-agent")
class Term:
def __init__(self, cols=120, rows=36, kitty=False, env=None, args=(), home=None,
ws=None, term="xterm-256color"):
self.cols, self.rows, self.kitty = cols, rows, kitty
self.screen = pyte.Screen(cols, rows)
self.stream = pyte.ByteStream(self.screen)
self.home = home or tempfile.mkdtemp(prefix="cstk_home_")
self.ws = ws or tempfile.mkdtemp(prefix="cstk_ws_")
e = dict(os.environ)
e.update({"TERM": term, "HOME": self.home,
"XDG_CONFIG_HOME": os.path.join(self.home, ".config"),
"LANG": "C.UTF-8", "LC_ALL": "C.UTF-8",
"CST_PROVIDER": "deepseek", "DEEPSEEK_API_KEY": "sk-dummy-test",
"CST_METRICS": "0"})
e.pop("NO_COLOR", None)
if env:
e.update(env)
self.raw = b""
pid, fd = pty.fork()
if pid == 0:
os.chdir(self.ws)
os.execve(BIN, [BIN, "-N", *args], e)
self.pid, self.fd = pid, fd
fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack("HHHH", rows, cols, 0, 0))
self.pump(1.5)
def pump(self, t=0.25):
end = time.time() + t
while True:
left = end - time.time()
if left <= 0:
break
r, _, _ = select.select([self.fd], [], [], left)
if not r:
continue
try:
d = os.read(self.fd, 65536)
except OSError:
break
if not d:
break
self.raw += d
if self.kitty and b"\x1b[?u" in d:
os.write(self.fd, b"\x1b[?0u")
self.stream.feed(d)
def send(self, b, t=0.15):
if isinstance(b, str):
b = b.encode()
os.write(self.fd, b)
self.pump(t)
def appcursor(self):
return (1 << 5) in self.screen.mode # pyte DECCKM
def arrow(self, d, form=None, mod=None, t=0.15):
c = {"up": "A", "down": "B", "right": "C", "left": "D"}[d]
if mod:
seq = "\x1b[1;%d%s" % (mod, c)
else:
f = form or ("ss3" if self.appcursor() else "csi")
seq = ("\x1bO" if f == "ss3" else "\x1b[") + c
self.send(seq, t)
def lines(self):
return [l.rstrip() for l in self.screen.display]
def find(self, s):
return [(i, l) for i, l in enumerate(self.lines()) if s in l]
def cursor(self):
return self.screen.cursor.y, self.screen.cursor.x
def dump(self, title=""):
print("=" * 20, title, "cursor", self.cursor())
for i, l in enumerate(self.lines()):
print("%2d|%s" % (i, l))
def close(self):
try:
os.kill(self.pid, signal.SIGTERM)
os.waitpid(self.pid, 0)
except Exception:
pass
+273
View File
@@ -0,0 +1,273 @@
import os
import sys, time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from harness import Term
import fakeapi
srv, base = fakeapi.start()
FAILS = []
VERBOSE = "-v" in sys.argv
def check(name, cond, t=None):
print(("PASS " if cond else "FAIL ") + name)
if not cond:
FAILS.append(name)
if t:
t.dump(name)
def input_text(t):
"""Input box rows: the bar rows right above the mode line."""
L = t.lines()
mode = [i for i, l in enumerate(L) if l.lstrip().startswith("┃ ") and
("Build ·" in l or "Plan ·" in l)]
if not mode:
return []
rows = []
i = mode[-1] - 1
while i >= 0:
s = L[i].lstrip()
if not s.startswith("┃"):
break
body = s[2:].split(" ")[0].rstrip() if s.startswith("┃ ") else ""
if body == "" and rows:
break
if body:
rows.append(body)
i -= 1
rows.reverse()
if rows and (rows[0].startswith("Ask anything") or rows[0].startswith("Enter queues")):
return []
return rows
def styled(t):
"""Screen text plus the reverse/bg attributes, to see highlight moves."""
out = []
for y in range(t.rows):
row = t.screen.buffer[y]
out.append("".join("%s%s%s" % (row[x].data, int(row[x].reverse), row[x].bg[:3])
for x in range(t.cols)))
return out
def clear(t):
if [r for r in input_text(t) if r]:
t.send("\x03", 0.2) # ctrl+c clears a non-empty input (twice on empty quits)
def new(kitty=False, **kw):
t = Term(kitty=kitty, args=("-b", base, "-k", "sk-test", "-m", "deepseek-flash"), **kw)
return t
def basic(t, form, tag):
clear(t)
t.send("hello world")
for _ in range(5):
t.arrow("left", form)
t.send("X")
check(f"{tag} left x5 insert", "hello Xworld" in input_text(t), t)
t.arrow("right", form); t.arrow("right", form)
t.send("Y")
check(f"{tag} right x2 insert", "hello XwoYrld" in input_text(t), t)
# boundaries
for _ in range(30):
t.arrow("left", form, t=0.02)
t.pump(0.2)
t.send("<")
check(f"{tag} left past start", "<hello XwoYrld" in input_text(t), t)
for _ in range(30):
t.arrow("right", form, t=0.02)
t.pump(0.2)
t.send(">")
check(f"{tag} right past end", "<hello XwoYrld>" in input_text(t), t)
clear(t)
def cjk(t):
clear(t)
t.send("你好世界")
y, x_end = t.cursor()
t.arrow("left"); t.arrow("left")
y2, x2 = t.cursor()
check("cjk cursor moves 2 cells per char", x_end - x2 == 4, t)
t.send("A")
check("cjk insert between wide chars", "你好A世界" in input_text(t), t)
t.arrow("right")
t.send("B")
check("cjk right over wide char", "你好A世B界" in input_text(t), t)
clear(t)
t.send("a😀b")
t.arrow("left"); t.arrow("left")
t.send("Z")
check("emoji left", "aZ😀b" in input_text(t), t)
clear(t)
def multiline(t):
clear(t)
t.send("line one is long")
t.send("\x0e") # ctrl+n newline
t.send("l2")
t.send("\x0e")
t.send("third line here")
rows = input_text(t)
check("multiline 3 rows", rows[-3:] == ["line one is long", "l2", "third line here"], t)
for _ in range(4):
t.arrow("left") # cursor before "here"
t.arrow("up") # col 11 -> clamp to end of "l2"
t.send("U")
rows = input_text(t)
check("up clamps column to short line", rows[-2] == "l2U", t)
t.arrow("up") # col 3 on line one
t.send("V")
rows = input_text(t)
check("up keeps column", rows[-3] == "linVe one is long", t)
t.arrow("down"); t.arrow("down")
t.send("W")
rows = input_text(t)
check("down twice to last line", "W" in rows[-1], t)
clear(t)
def history(t):
clear(t)
t.send("first prompt\r", 1.5)
t.send("second prompt\r", 1.5)
t.arrow("up")
check("hist up -> newest", input_text(t)[-1:] == ["second prompt"], t)
t.arrow("up")
check("hist up -> older", input_text(t)[-1:] == ["first prompt"], t)
t.arrow("up")
check("hist up at oldest stays", input_text(t)[-1:] == ["first prompt"], t)
t.arrow("down")
check("hist down -> newer", input_text(t)[-1:] == ["second prompt"], t)
t.arrow("down")
check("hist down past newest -> empty", input_text(t) in ([], [""]), t)
# draft survives a trip through history
t.send("my draft")
t.arrow("up")
t.arrow("down")
check("draft restored after up/down", input_text(t)[-1:] == ["my draft"], t)
clear(t)
def wrapped(t):
clear(t)
long = " ".join("w%02d" % i for i in range(40)) # ~160 chars, wraps
t.send(long, 0.4)
nrows = len(input_text(t))
t.arrow("up")
txt = "".join(input_text(t))
check("up on wrapped line keeps the text", "w39" in txt and "w00" in txt, t)
t.send("Q")
rows = input_text(t)
check("up on wrapped line moves one visual row", nrows >= 2 and "Q" in "".join(rows[:-1]), t)
clear(t)
def completion(t):
clear(t)
t.send("/", 0.3)
before = styled(t)
t.arrow("down"); t.arrow("down")
mid = styled(t)
t.arrow("up")
after = styled(t)
check("completion popup reacts to down", before != mid, t)
check("completion up differs from down-down", mid != after, t)
check("completion arrows do not touch input", input_text(t)[-1:] == ["/"], t)
t.send("\x1b", 0.3)
clear(t)
def modifiers(t):
clear(t)
t.send("alpha beta gamma")
t.arrow("left", mod=5) # ctrl+left -> word left
t.send("X")
check("ctrl+left word", "alpha beta Xgamma" in input_text(t), t)
t.arrow("left", mod=3) # alt+left
t.arrow("left", mod=3)
t.send("Y")
check("alt+left word", "alpha Ybeta Xgamma" in input_text(t), t)
t.arrow("right", mod=5)
t.send("Z")
check("ctrl+right word", "alpha YbetaZ Xgamma" in input_text(t), t)
t.arrow("left", mod=2) # shift+left: should at least not insert junk
t.send("S")
txt = "".join(input_text(t))
check("shift+left no junk", "[" not in txt and ";" not in txt and "1;2" not in txt, t)
clear(t)
def split_and_burst(t, form):
clear(t)
t.send("abcdef")
ss = "\x1bO" if form == "ss3" else "\x1b["
# escape sequence split across writes (slow link)
import os
os.write(t.fd, ss[:1].encode()); time.sleep(0.008)
os.write(t.fd, (ss[1:] + "D").encode()); t.pump(0.2)
t.send("1")
check(f"split {form} seq", "abcde1f" in input_text(t), t)
# key-repeat burst in one write
t.send((ss + "D") * 4, 0.3)
t.send("2")
check(f"burst {form} x4", "ab2cde1f" in input_text(t), t)
clear(t)
def picker(t):
clear(t)
t.send("/model\r", 0.8)
s0 = styled(t)
t.arrow("down")
s1 = styled(t)
t.arrow("up")
s2 = styled(t)
check("picker down changes selection", s0 != s1, t)
check("picker up returns", s0 == s2, t)
t.send("\x1b", 0.4)
clear(t)
def run(kitty, tag):
print("#####", tag)
t = new(kitty=kitty)
try:
basic(t, None, tag + " native")
basic(t, "csi" if t.appcursor() else "ss3", tag + " other-form")
cjk(t)
multiline(t)
modifiers(t)
split_and_burst(t, "ss3")
split_and_burst(t, "csi")
completion(t)
wrapped(t)
history(t)
picker(t)
finally:
t.close()
if __name__ == "__main__":
run(False, "xterm")
run(True, "kitty")
for term in ("screen-256color", "tmux-256color", "linux", "vt220", "xterm"):
print("#####", term)
t = Term(args=("-b", base, "-k", "sk-test"), term=term)
try:
basic(t, "ss3", term + " ss3")
basic(t, "csi", term + " csi")
modifiers(t)
multiline(t)
cjk(t)
finally:
t.close()
print("FAILED:", len(FAILS))
for f in FAILS:
print(" -", f)
sys.exit(min(len(FAILS), 100))
+65
View File
@@ -0,0 +1,65 @@
"""TUI /provider and /key against a fake API whose /models checks the key.
nix-shell -p python3Packages.pyte --run "python3 scripts/tui_keys/provider.py"
A wrong key is asked for again, a good one is saved and switches the
provider; /key replaces it. Same code path as the WebUI (provider_switch.c).
"""
import json, os, sys, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from harness import Term
from keys import check, base
GOOD = {"sk-good", "sk-good2"}
class B(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def do_GET(self):
auth = self.headers.get("Authorization", "")
ok = auth.startswith("Bearer ") and auth[7:] in GOOD
body = json.dumps({"data": [{"id": "fake-one"}]} if ok else {"error": {"message": "bad key"}}).encode()
self.send_response(200 if ok else 401)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
srv = ThreadingHTTPServer(("127.0.0.1", 0), B)
threading.Thread(target=srv.serve_forever, daemon=True).start()
t = Term(args=("-b", base, "-k", "sk-test"))
cfg = os.path.join(t.home, ".config", "cst-agent")
os.makedirs(cfg, exist_ok=True)
t.close()
with open(os.path.join(cfg, "models.json"), "w") as f:
json.dump({"providers": {"fakeb": {"api_base": "http://127.0.0.1:%d/v1" % srv.server_address[1],
"key_env": "FAKEB_API_KEY", "default_model": ""}}}, f)
t = Term(args=("-b", base, "-k", "sk-test"), home=t.home, env={"FAKEB_API_KEY": ""})
creds = os.path.join(cfg, "credentials")
def screen():
return "\n".join(t.lines())
t.send("/provider fakeb\r", 0.8)
check("a key prompt for fakeb", "API key for fakeb" in screen(), t)
t.send("sk-bad\r", 1.5)
check("a wrong key is asked for again", "rejected that key" in screen(), t)
check("the wrong key is not saved", not os.path.exists(creds) or "sk-bad" not in open(creds).read(), t)
t.send("sk-good\r", 2.0)
check("switched to fakeb", "provider → fakeb" in screen(), t)
check("the good key is saved", os.path.exists(creds) and "fakeb.api_key=sk-good" in open(creds).read(), t)
check("no key on screen", "sk-good" not in screen() and "sk-bad" not in screen(), t)
t.send("/key\r", 0.8)
t.send("sk-good2\r", 2.0)
check("/key replaces it", "fakeb.api_key=sk-good2" in open(creds).read(), t)
t.close()
from keys import FAILS
print("FAILED:", len(FAILS))
sys.exit(min(len(FAILS), 100))
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Run the cross-built Windows tests inside the test VMs (scripts/winvm/vm.sh).
#
# scripts/win_test.sh [-b BUILD_DIR] [-v "en zh"] [-u "Administrator tester 测试"] [-f FILTER]
#
# BUILD_DIR (default build-win) must contain test_*.exe from the MinGW build;
# busybox.exe is taken from third_party/busybox/. VMs that are not running are
# started and stopped again afterwards. Exit status is the number of failed
# variant/user runs.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
VM="$ROOT/scripts/winvm/vm.sh"
BUILD="$ROOT/build-win"
VARIANTS="en zh"
USERS_LIST="Administrator tester 测试"
FILTER="*"
while getopts "b:v:u:f:" o; do
case "$o" in
b) BUILD="$(cd "$OPTARG" && pwd)" ;;
v) VARIANTS="$OPTARG" ;;
u) USERS_LIST="$OPTARG" ;;
f) FILTER="$OPTARG" ;;
*) sed -n '2,8p' "$0"; exit 2 ;;
esac
done
shopt -s nullglob
exes=("$BUILD"/test_*.exe)
[[ ${#exes[@]} -gt 0 ]] || { echo "no test_*.exe in $BUILD" >&2; exit 1; }
stage="$(mktemp -d)"
trap 'rm -rf "$stage"' EXIT
mkdir -p "$stage/bundle/bin" "$stage/bundle/data/tests"
cp "${exes[@]}" "$stage/bundle/bin/"
for f in "$BUILD"/*.dll "$ROOT/third_party/busybox/busybox.exe"; do
[[ -f "$f" ]] && cp "$f" "$stage/bundle/bin/"
done
cp -r "$ROOT/tests/fixtures" "$stage/bundle/data/tests/"
cp "$ROOT/scripts/winvm/run_tests.ps1" "$stage/bundle/"
tar -C "$stage" -cf "$stage/bundle.tar" bundle
total_fail=0
summary=()
for v in $VARIANTS; do
started=0
if [[ "$("$VM" status "$v")" != running ]]; then
"$VM" start "$v"
started=1
fi
"$VM" ssh "$v" 'Remove-Item -Recurse -Force C:\cst-test\bundle -ErrorAction SilentlyContinue; New-Item -ItemType Directory -Force C:\cst-test | Out-Null' >/dev/null
"$VM" put "$v" "$stage/bundle.tar" "C:/cst-test/bundle.tar"
"$VM" ssh "$v" 'tar -xf C:\cst-test\bundle.tar -C C:\cst-test; Remove-Item C:\cst-test\bundle.tar' >/dev/null
for u in $USERS_LIST; do
echo "===== $v / $u"
rc=0
"$VM" ps "$v" "$u" "$stage/bundle/run_tests.ps1" -Bundle 'C:\cst-test\bundle' -Filter "'$FILTER'" || rc=$?
summary+=("$v/$u: $([[ $rc == 0 ]] && echo ok || echo FAIL)")
[[ $rc == 0 ]] || total_fail=$((total_fail + 1))
done
[[ $started == 1 ]] && "$VM" stop "$v"
done
echo
printf '%s\n' "${summary[@]}"
exit "$total_fail"
+4
View File
@@ -0,0 +1,4 @@
@echo off
rem Update CST-Agent in this folder from the latest release.
rem cst-agent-update [-Version 0.6.0]
powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0install_windows.ps1" -Prefix "%~dp0." %*
+151
View File
@@ -0,0 +1,151 @@
; CST-Agent per-user installer (no admin rights).
; makensis -INPUTCHARSET UTF8 -DVERSION=0.8.5 -DSRCDIR=dist/windows-x86_64 -DOUTFILE=setup.exe cst-agent.nsi
; Installs the same files as the zip into %LOCALAPPDATA%\Programs\cst-agent,
; adds that folder to the user PATH and registers an uninstaller under HKCU.
Unicode true
SetCompressor /SOLID lzma
RequestExecutionLevel user
!ifndef VERSION
!error "pass -DVERSION=x.y.z"
!endif
!ifndef SRCDIR
!error "pass -DSRCDIR=<dist/windows-x86_64>"
!endif
!ifndef OUTFILE
!define OUTFILE "cst-agent-${VERSION}-windows-x86_64-setup.exe"
!endif
!define APP "CST-Agent"
!define UNINST_KEY "Software\Microsoft\Windows\CurrentVersion\Uninstall\cst-agent"
!include "MUI2.nsh"
!include "LogicLib.nsh"
!include "x64.nsh"
Name "${APP} ${VERSION}"
OutFile "${OUTFILE}"
InstallDir "$LOCALAPPDATA\Programs\cst-agent"
InstallDirRegKey HKCU "${UNINST_KEY}" "InstallLocation"
BrandingText "${APP} ${VERSION}"
VIProductVersion "${VERSION}.0"
VIAddVersionKey /LANG=1033 "ProductName" "${APP}"
VIAddVersionKey /LANG=1033 "ProductVersion" "${VERSION}"
VIAddVersionKey /LANG=1033 "FileVersion" "${VERSION}"
VIAddVersionKey /LANG=1033 "FileDescription" "${APP} installer"
VIAddVersionKey /LANG=1033 "LegalCopyright" "WHC"
!define MUI_ABORTWARNING
!define MUI_FINISHPAGE_RUN
!define MUI_FINISHPAGE_RUN_TEXT "$(RunWebUI)"
!define MUI_FINISHPAGE_RUN_FUNCTION LaunchWebUI
!insertmacro MUI_PAGE_DIRECTORY
!insertmacro MUI_PAGE_INSTFILES
!insertmacro MUI_PAGE_FINISH
!insertmacro MUI_UNPAGE_CONFIRM
!insertmacro MUI_UNPAGE_INSTFILES
!insertmacro MUI_LANGUAGE "English"
!insertmacro MUI_LANGUAGE "SimpChinese"
LangString NeedX64 ${LANG_ENGLISH} "${APP} needs 64-bit Windows."
LangString NeedX64 ${LANG_SIMPCHINESE} "${APP} 需要 64 位 Windows。"
LangString NeedBuild ${LANG_ENGLISH} "${APP} needs Windows 10 1903 (build 18362) or newer; this is build $0."
LangString NeedBuild ${LANG_SIMPCHINESE} "${APP} 需要 Windows 10 1903(内部版本 18362)或更新版本,当前为 $0。"
LangString RunWebUI ${LANG_ENGLISH} "Open the ${APP} WebUI"
LangString RunWebUI ${LANG_SIMPCHINESE} "打开 ${APP} WebUI"
LangString PathFail ${LANG_ENGLISH} "Could not add $INSTDIR to the user PATH; add it by hand."
LangString PathFail ${LANG_SIMPCHINESE} "未能把 $INSTDIR 加入用户 PATH,请手动添加。"
; PowerShell edits the user PATH: NSIS strings are capped at 1024 chars, and
; .NET broadcasts the change so new terminals see it. The folder is passed in
; the environment, so quotes in it cannot break the command.
!macro EditUserPath ACTION
System::Call 'Kernel32::SetEnvironmentVariable(t "CST_INSTDIR", t "$INSTDIR")'
nsExec::ExecToLog `powershell -NoProfile -ExecutionPolicy Bypass -Command "$$d = $$env:CST_INSTDIR.TrimEnd('\'); $$p = [Environment]::GetEnvironmentVariable('Path', 'User'); $$parts = @([string]$$p -split ';' | Where-Object { $$_ -and $$_.TrimEnd('\') -ine $$d }); if ('${ACTION}' -eq 'add') { $$parts += $$d }; [Environment]::SetEnvironmentVariable('Path', ($$parts -join ';'), 'User')"`
Pop $0
!macroend
; A running exe cannot be overwritten but can be renamed out of the way.
!macro MoveAside NAME
Delete "$INSTDIR\${NAME}.old"
ClearErrors
Delete "$INSTDIR\${NAME}"
${If} ${Errors}
Rename "$INSTDIR\${NAME}" "$INSTDIR\${NAME}.old"
${EndIf}
!macroend
Function .onInit
${IfNot} ${RunningX64}
MessageBox MB_ICONSTOP "$(NeedX64)"
Abort
${EndIf}
SetRegView 64
ReadRegStr $0 HKLM "SOFTWARE\Microsoft\Windows NT\CurrentVersion" "CurrentBuildNumber"
${If} $0 < 18362
MessageBox MB_ICONSTOP "$(NeedBuild)"
Abort
${EndIf}
FunctionEnd
Function LaunchWebUI
SetOutPath "$PROFILE"
Exec '"$INSTDIR\cst-agent.exe" --web --web-open'
FunctionEnd
Section "Install"
SetShellVarContext current
SetOutPath "$INSTDIR"
!insertmacro MoveAside "cst-agent.exe"
!insertmacro MoveAside "busybox.exe"
File "${SRCDIR}/cst-agent.exe"
File "${SRCDIR}/busybox.exe"
File "${SRCDIR}/LICENSE.busybox"
File "${SRCDIR}/README.md"
File "${SRCDIR}/cst-agent-update.cmd"
File "${SRCDIR}/install_windows.ps1"
File "${SRCDIR}/VERSION"
WriteUninstaller "$INSTDIR\uninstall.exe"
!insertmacro EditUserPath "add"
${If} $0 != 0
DetailPrint "$(PathFail)"
${EndIf}
; The WebUI's workspace is the shortcut's start folder.
SetOutPath "$PROFILE"
CreateShortCut "$SMPROGRAMS\${APP} WebUI.lnk" "$INSTDIR\cst-agent.exe" "--web --web-open" "$INSTDIR\cst-agent.exe" 0
SetOutPath "$INSTDIR"
WriteRegStr HKCU "${UNINST_KEY}" "DisplayName" "${APP}"
WriteRegStr HKCU "${UNINST_KEY}" "DisplayVersion" "${VERSION}"
WriteRegStr HKCU "${UNINST_KEY}" "Publisher" "WHC"
WriteRegStr HKCU "${UNINST_KEY}" "DisplayIcon" "$INSTDIR\cst-agent.exe"
WriteRegStr HKCU "${UNINST_KEY}" "InstallLocation" "$INSTDIR"
WriteRegStr HKCU "${UNINST_KEY}" "UninstallString" '"$INSTDIR\uninstall.exe"'
WriteRegStr HKCU "${UNINST_KEY}" "QuietUninstallString" '"$INSTDIR\uninstall.exe" /S'
WriteRegStr HKCU "${UNINST_KEY}" "URLInfoAbout" "https://git.whcself.xyz/WHC/CST-agent"
WriteRegDWORD HKCU "${UNINST_KEY}" "NoModify" 1
WriteRegDWORD HKCU "${UNINST_KEY}" "NoRepair" 1
SectionEnd
Section "Uninstall"
SetShellVarContext current
!insertmacro EditUserPath "remove"
Delete "$SMPROGRAMS\${APP} WebUI.lnk"
Delete "$INSTDIR\cst-agent.exe"
Delete "$INSTDIR\cst-agent.exe.old"
Delete "$INSTDIR\busybox.exe"
Delete "$INSTDIR\busybox.exe.old"
Delete "$INSTDIR\LICENSE.busybox"
Delete "$INSTDIR\README.md"
Delete "$INSTDIR\cst-agent-update.cmd"
Delete "$INSTDIR\install_windows.ps1"
Delete "$INSTDIR\VERSION"
Delete "$INSTDIR\uninstall.exe"
RMDir "$INSTDIR"
DeleteRegKey HKCU "${UNINST_KEY}"
SectionEnd
+98
View File
@@ -0,0 +1,98 @@
<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State">
<settings pass="windowsPE">
<component name="Microsoft-Windows-International-Core-WinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
<SetupUILanguage>
<UILanguage>en-US</UILanguage>
</SetupUILanguage>
<InputLocale>en-US</InputLocale>
<SystemLocale>en-US</SystemLocale>
<UILanguage>en-US</UILanguage>
<UserLocale>en-US</UserLocale>
</component>
<component name="Microsoft-Windows-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
<DiskConfiguration>
<Disk wcm:action="add">
<DiskID>0</DiskID>
<WillWipeDisk>true</WillWipeDisk>
<CreatePartitions>
<CreatePartition wcm:action="add">
<Order>1</Order>
<Type>Primary</Type>
<Extend>true</Extend>
</CreatePartition>
</CreatePartitions>
<ModifyPartitions>
<ModifyPartition wcm:action="add">
<Order>1</Order>
<PartitionID>1</PartitionID>
<Active>true</Active>
<Format>NTFS</Format>
<Label>Windows</Label>
<Letter>C</Letter>
</ModifyPartition>
</ModifyPartitions>
</Disk>
</DiskConfiguration>
<ImageInstall>
<OSImage>
<InstallFrom>
<MetaData wcm:action="add">
<Key>/IMAGE/NAME</Key>
<Value>@IMAGE_NAME@</Value>
</MetaData>
</InstallFrom>
<InstallTo>
<DiskID>0</DiskID>
<PartitionID>1</PartitionID>
</InstallTo>
</OSImage>
</ImageInstall>
<UserData>
<AcceptEula>true</AcceptEula>
<FullName>CST</FullName>
<Organization>CST</Organization>
</UserData>
</component>
</settings>
<settings pass="specialize">
<component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
<ComputerName>cst-win</ComputerName>
<TimeZone>China Standard Time</TimeZone>
</component>
</settings>
<settings pass="oobeSystem">
<component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
<OOBE>
<HideEULAPage>true</HideEULAPage>
<HideOnlineAccountScreens>true</HideOnlineAccountScreens>
<HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
<ProtectYourPC>3</ProtectYourPC>
<NetworkLocation>Work</NetworkLocation>
</OOBE>
<UserAccounts>
<AdministratorPassword>
<Value>@ADMIN_PASSWORD@</Value>
<PlainText>true</PlainText>
</AdministratorPassword>
</UserAccounts>
<AutoLogon>
<Enabled>true</Enabled>
<Username>Administrator</Username>
<Password>
<Value>@ADMIN_PASSWORD@</Value>
<PlainText>true</PlainText>
</Password>
<LogonCount>1</LogonCount>
</AutoLogon>
<FirstLogonCommands>
<SynchronousCommand wcm:action="add">
<Order>1</Order>
<CommandLine>powershell -NoProfile -ExecutionPolicy Bypass -Command "foreach ($d in Get-PSDrive -PSProvider FileSystem) { $p = Join-Path $d.Root 'setup.ps1'; if (Test-Path $p) { &amp; $p; break } }"</CommandLine>
<Description>Enable OpenSSH</Description>
<RequiresUserInput>false</RequiresUserInput>
</SynchronousCommand>
</FirstLogonCommands>
</component>
</settings>
</unattend>
+28
View File
@@ -0,0 +1,28 @@
# Idempotent VM setup after install: test users, sshd access, console defaults.
# Users come from C:\Windows\Temp\cst-users.txt ("name<TAB>password", UTF-8), deleted afterwards.
$ErrorActionPreference = 'Stop'
$list = 'C:\Windows\Temp\cst-users.txt'
if (Test-Path $list) {
foreach ($line in Get-Content -Path $list -Encoding UTF8) {
$name, $pw = $line -split "`t", 2
if (-not $name) { continue }
$sec = ConvertTo-SecureString $pw -AsPlainText -Force
if (Get-LocalUser -Name $name -ErrorAction SilentlyContinue) {
Set-LocalUser -Name $name -Password $sec
} else {
New-LocalUser -Name $name -Password $sec -PasswordNeverExpires -AccountNeverExpires | Out-Null
}
foreach ($g in 'Users', 'OpenSSH Users') {
Add-LocalGroupMember -Group $g -Member $name -ErrorAction SilentlyContinue
}
"user ok: $name"
}
Remove-Item -Force $list
}
# the built-in rule only covers the Private profile; QEMU user networking is Public
Set-NetFirewallRule -Name 'OpenSSH-Server-In-TCP' -Profile Any -Enabled True
Set-SConfig -AutoLaunch $false
New-Item -ItemType Directory -Force C:\cst-test | Out-Null
"provision ok"
+51
View File
@@ -0,0 +1,51 @@
# Run cross-built test_*.exe inside the VM as the current user.
# run_tests.ps1 BUNDLE [FILTER]
# BUNDLE holds bin\ (exes, busybox.exe) and data\ (repo-relative fixtures).
# Each run copies data\ into a per-user work dir and uses it as the cwd.
param([Parameter(Mandatory = $true)][string]$Bundle, [string]$Filter = '*')
$ErrorActionPreference = 'Stop'
[Console]::OutputEncoding = [Text.UTF8Encoding]::new($false)
$OutputEncoding = [Console]::OutputEncoding
$work = Join-Path $env:TEMP ('cst-test-' + [guid]::NewGuid().ToString('N').Substring(0, 8))
New-Item -ItemType Directory -Force $work | Out-Null
Copy-Item -Recurse -Force (Join-Path $Bundle 'data\*') $work
$env:PATH = (Join-Path $Bundle 'bin') + ';' + $env:PATH
$acp = [Text.Encoding]::Default.CodePage
"== user=$env:USERNAME acp=$acp work=$work"
$fail = 0
$pass = 0
foreach ($exe in Get-ChildItem (Join-Path $Bundle 'bin') -Filter "test_$Filter.exe" | Sort-Object Name) {
$name = $exe.BaseName
$out = Join-Path $work "$name.out"
$err = Join-Path $work "$name.err"
$sw = [Diagnostics.Stopwatch]::StartNew()
$p = Start-Process -FilePath $exe.FullName -WorkingDirectory $work -NoNewWindow -PassThru `
-RedirectStandardOutput $out -RedirectStandardError $err
$null = $p.Handle
if (-not $p.WaitForExit(300000)) {
& taskkill.exe /T /F /PID $p.Id | Out-Null
$code = 'timeout'
} else {
$p.WaitForExit()
$code = $p.ExitCode
}
$secs = [math]::Round($sw.Elapsed.TotalSeconds, 1)
if ($code -eq 0) {
$pass++
"PASS $name (${secs}s)"
} else {
$fail++
"FAIL $name (exit $code, ${secs}s)"
foreach ($f in $out, $err) {
if (Test-Path $f) {
Get-Content -Path $f -Encoding UTF8 -Tail 25 | ForEach-Object { " $_" }
}
}
}
}
"== $pass passed, $fail failed"
Set-Location $env:TEMP
Remove-Item -Recurse -Force $work -ErrorAction SilentlyContinue
exit $fail
+28
View File
@@ -0,0 +1,28 @@
Start-Transcript -Path C:\setup.log -Append
$ErrorActionPreference = 'Continue'
if (-not (Get-Service sshd -ErrorAction SilentlyContinue)) {
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
}
Set-Service -Name sshd -StartupType Automatic
Start-Service sshd
$keys = 'C:\ProgramData\ssh\administrators_authorized_keys'
Get-Content (Join-Path $PSScriptRoot 'vm_key.pub') | Set-Content -Path $keys -Encoding ascii
icacls $keys /inheritance:r /grant 'Administrators:F' /grant 'SYSTEM:F'
New-ItemProperty -Path HKLM:\SOFTWARE\OpenSSH -Name DefaultShell -PropertyType String -Force `
-Value 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe'
if (Get-NetFirewallRule -Name 'OpenSSH-Server-In-TCP' -ErrorAction SilentlyContinue) {
# the built-in rule only covers the Private profile; QEMU user networking is Public
Set-NetFirewallRule -Name 'OpenSSH-Server-In-TCP' -Profile Any -Enabled True
} else {
New-NetFirewallRule -Name 'OpenSSH-Server-In-TCP' -DisplayName 'OpenSSH Server (sshd)' `
-Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -Profile Any
}
powercfg /change monitor-timeout-ac 0
powercfg /change standby-timeout-ac 0
New-Item -Path C:\setup.done -ItemType File -Force
Stop-Transcript
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""Type text into the VM console via QMP.
type.py QMP_SOCKET TEXT (\\n = Enter)
type.py QMP_SOCKET @FILE the file's first line, then Enter (secrets stay
off the command line)
type.py QMP_SOCKET --cad Ctrl+Alt+Del
"""
import json
import os
import socket
import sys
import time
SHIFTED = {
'!': '1', '@': '2', '#': '3', '$': '4', '%': '5', '^': '6', '&': '7',
'*': '8', '(': '9', ')': '0', '_': 'minus', '+': 'equal', '{': 'bracket_left',
'}': 'bracket_right', '|': 'backslash', ':': 'semicolon', '"': 'apostrophe',
'<': 'comma', '>': 'dot', '?': 'slash', '~': 'grave_accent',
}
PLAIN = {
' ': 'spc', '\n': 'ret', '\t': 'tab', '-': 'minus', '=': 'equal',
'[': 'bracket_left', ']': 'bracket_right', '\\': 'backslash', ';': 'semicolon',
"'": 'apostrophe', ',': 'comma', '.': 'dot', '/': 'slash', '`': 'grave_accent',
}
def keys_for(ch):
if ch.isascii() and ch.isalpha():
return (['shift'] if ch.isupper() else []) + [ch.lower()]
if ch.isdigit():
return [ch]
if ch in PLAIN:
return [PLAIN[ch]]
if ch in SHIFTED:
return ['shift', SHIFTED[ch]]
raise SystemExit(f'unsupported char {ch!r}')
def main():
arg = sys.argv[2]
if arg == '--cad':
combos = [['ctrl', 'alt', 'delete']]
else:
if arg.startswith('@'):
with open(arg[1:], encoding='utf-8') as fp:
text = fp.readline().rstrip('\r\n') + '\n'
else:
text = arg.encode().decode('unicode_escape')
combos = [keys_for(ch) for ch in text]
s = socket.socket(socket.AF_UNIX)
s.connect(sys.argv[1])
f = s.makefile('rw')
f.readline()
f.write(json.dumps({'execute': 'qmp_capabilities'}) + '\n')
f.flush()
f.readline()
for combo in combos:
keys = [{'type': 'qcode', 'data': k} for k in combo]
f.write(json.dumps({'execute': 'send-key', 'arguments': {'keys': keys}}) + '\n')
f.flush()
f.readline()
time.sleep(0.03)
main()
+288
View File
@@ -0,0 +1,288 @@
#!/usr/bin/env bash
# Headless Windows Server 2025 Core test VMs (QEMU/KVM).
#
# vm.sh create download the evaluation ISO, install base.qcow2
# vm.sh variants en.qcow2 / zh.qcow2 overlays on top of base.qcow2
# (zh: zh-CN system locale, ANSI code page 936)
# vm.sh start|stop|status|wait VARIANT
# vm.sh ssh VARIANT [USER] [command...] USER: Administrator (default), tester, 测试
# vm.sh put VARIANT SRC... DEST scp as Administrator
# vm.sh ps VARIANT [USER] FILE.ps1 [args...] run a script (uploaded with a UTF-8 BOM)
# vm.sh shot VARIANT [out.png] console screenshot
# vm.sh type VARIANT TEXT type into the console (\n = Enter)
# vm.sh login VARIANT unlock the console as Administrator
#
# State (ISO, disks, keys, passwords) lives in $CST_WINVM_DIR, default ~/winvm.
# Keys and passwords are created there with mode 0600 and never printed.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
D="${CST_WINVM_DIR:-$HOME/winvm}"
ISO_URL="https://aka.ms/WinServ2025iso-enus"
IMAGE_NAME="Windows Server 2025 SERVERSTANDARDCORE"
USERS=(tester 测试)
if ! command -v qemu-system-x86_64 >/dev/null || ! command -v sshpass >/dev/null ||
! command -v socat >/dev/null || ! command -v xorriso >/dev/null; then
if command -v nix-shell >/dev/null && [[ -z "${CST_WINVM_NIX:-}" ]]; then
export CST_WINVM_NIX=1
exec nix-shell -p qemu sshpass socat xorriso imagemagick python3 \
--run "$(printf '%q ' "$0" "$@")"
fi
echo "need qemu-system-x86_64, sshpass, socat, xorriso" >&2
exit 1
fi
die() { echo "vm.sh: $*" >&2; exit 1; }
port_of() {
case "$1" in
base) echo 2221 ;;
en) echo 2222 ;;
zh) echo 2223 ;;
*) die "unknown variant '$1' (base, en, zh)" ;;
esac
}
pw_file() {
case "$1" in
Administrator) echo "$D/admin-password" ;;
tester) echo "$D/tester-password" ;;
测试) echo "$D/ceshi-password" ;;
*) die "unknown user '$1'" ;;
esac
}
new_password() { # prefix -> file, mixed classes for Windows complexity rules
local f="$2"
[[ -s "$f" ]] && return 0
(umask 077; printf '%s%s9a' "$1" "$(head -c 256 /dev/urandom | tr -dc 'A-Za-z0-9' | head -c 16)" >"$f")
}
running() {
local pid
pid="$(cat "$D/$1.pid" 2>/dev/null)" || return 1
[[ -n "$pid" ]] && kill -0 "$pid" 2>/dev/null
}
ssh_opts() {
echo -p "$(port_of "$1")" -o UserKnownHostsFile="$D/known_hosts" \
-o StrictHostKeyChecking=accept-new -o LogLevel=ERROR -o ConnectTimeout=10
}
vm_ssh() { # variant user command...
local v="$1" u="$2"
shift 2
# shellcheck disable=SC2046
if [[ "$u" == Administrator ]]; then
ssh -i "$D/vm_key" -o BatchMode=yes $(ssh_opts "$v") "Administrator@127.0.0.1" "$@"
else
SSHPASS="$(cat "$(pw_file "$u")")" sshpass -e ssh -o PubkeyAuthentication=no \
-o PreferredAuthentications=password $(ssh_opts "$v") "$u@127.0.0.1" "$@"
fi
}
vm_put() { # variant src... dest
local v="$1"
shift
local dest="${!#}"
local srcs=("${@:1:$#-1}")
# shellcheck disable=SC2046
scp -q -r -i "$D/vm_key" -o BatchMode=yes $(ssh_opts "$v" | sed 's/^-p/-P/') \
"${srcs[@]}" "Administrator@127.0.0.1:$dest"
}
qemu_start() { # variant disk [extra args...]
local v="$1" disk="$2"
shift 2
running "$v" && return 0
[[ -f "$disk" ]] || die "missing $disk"
qemu-system-x86_64 \
-name "cst-win-$v" -machine q35,accel=kvm \
-cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
-smp 4 -m 6G -rtc base=localtime \
-device ahci,id=ahci \
-drive file="$disk",if=none,id=d0,cache=writeback,discard=unmap \
-device ide-hd,drive=d0,bus=ahci.0 \
-netdev user,id=n0,hostfwd=tcp:127.0.0.1:"$(port_of "$v")"-:22 \
-device e1000e,netdev=n0 \
-device qemu-xhci,id=xhci -device usb-tablet,bus=xhci.0 \
-display none -qmp unix:"$D/$v.qmp",server,nowait \
-pidfile "$D/$v.pid" -daemonize "$@"
}
disk_of() {
case "$1" in
base) echo "$D/base.qcow2" ;;
*) echo "$D/$1.qcow2" ;;
esac
}
vm_wait() { # variant [timeout_s]
local v="$1" t="${2:-900}" i
for ((i = 0; i < t; i += 5)); do
if vm_ssh "$v" Administrator 'Test-Path C:\setup.done' 2>/dev/null | grep -q True; then
return 0
fi
running "$v" || die "$v is not running"
sleep 5
done
die "$v: no SSH after ${t}s"
}
vm_stop() {
local v="$1" i
running "$v" || return 0
vm_ssh "$v" Administrator 'Stop-Computer -Force' >/dev/null 2>&1 || true
for ((i = 0; i < 120; i++)); do
running "$v" || { rm -f "$D/$v.pid" "$D/$v.qmp"; return 0; }
sleep 1
done
printf '%s\n' '{"execute":"qmp_capabilities"}' '{"execute":"quit"}' |
socat - UNIX-CONNECT:"$D/$v.qmp" >/dev/null || true
rm -f "$D/$v.pid" "$D/$v.qmp"
}
with_bom() { # file -> temp copy with a UTF-8 BOM (Windows PowerShell 5.1 reads ANSI otherwise)
local out
out="$(mktemp --suffix=.ps1)"
printf '\xef\xbb\xbf' >"$out"
sed '1s/^\xef\xbb\xbf//' "$1" >>"$out"
echo "$out"
}
vm_ps() { # variant user file args...
local v="$1" u="$2" f="$3"
shift 3
local tmp name
tmp="$(with_bom "$f")"
name="cst-$(basename "$f" .ps1)-$$.ps1"
vm_put "$v" "$tmp" "C:/Windows/Temp/$name"
rm -f "$tmp"
vm_ssh "$v" Administrator "icacls C:\\Windows\\Temp\\$name /grant 'Users:RX' | Out-Null" >/dev/null
local rc=0
vm_ssh "$v" "$u" "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\Windows\\Temp\\$name $*" || rc=$?
vm_ssh "$v" Administrator "Remove-Item -Force C:\\Windows\\Temp\\$name" >/dev/null 2>&1 || true
return $rc
}
provision() { # variant: users, sshd, console; idempotent
local v="$1" u tmp
tmp="$(mktemp)"
chmod 600 "$tmp"
for u in "${USERS[@]}"; do
new_password "Tst!" "$(pw_file "$u")"
printf '%s\t%s\n' "$u" "$(cat "$(pw_file "$u")")" >>"$tmp"
done
vm_put "$v" "$tmp" "C:/Windows/Temp/cst-users.txt"
rm -f "$tmp"
vm_ps "$v" Administrator "$HERE/provision.ps1"
}
cmd_create() {
mkdir -p "$D"
chmod 700 "$D"
[[ -f "$D/base.qcow2" ]] && die "$D/base.qcow2 exists"
if [[ ! -s "$D/server2025.iso" ]]; then
echo "==> downloading the Windows Server 2025 evaluation ISO (~8 GB)"
curl -fL -C - -o "$D/server2025.iso" "$ISO_URL"
fi
[[ -f "$D/vm_key" ]] || ssh-keygen -q -t ed25519 -N "" -C cst-winvm -f "$D/vm_key"
new_password "Cst!" "$D/admin-password"
local u
u="$(mktemp -d)"
chmod 700 "$u"
cp "$HERE/setup.ps1" "$D/vm_key.pub" "$u/"
(umask 077
sed -e "s|@IMAGE_NAME@|$IMAGE_NAME|" -e "s|@ADMIN_PASSWORD@|$(cat "$D/admin-password")|g" \
"$HERE/autounattend.xml.in" >"$u/autounattend.xml"
xorriso -as mkisofs -quiet -J -r -V UNATTEND -o "$D/unattend.iso" "$u/" 2>/dev/null)
rm -rf "$u"
qemu-img create -q -f qcow2 "$D/base.qcow2" 60G
echo "==> unattended install (20-30 minutes)"
qemu_start base "$D/base.qcow2" \
-drive file="$D/server2025.iso",media=cdrom,if=none,id=cd0,readonly=on \
-device ide-cd,drive=cd0,bus=ahci.1 \
-drive file="$D/unattend.iso",media=cdrom,if=none,id=cd1,readonly=on \
-device ide-cd,drive=cd1,bus=ahci.2 \
-boot order=c,once=d
vm_wait base 3600
provision base
vm_stop base
echo "OK: $D/base.qcow2; next: vm.sh variants"
}
cmd_variants() {
[[ -f "$D/base.qcow2" ]] || die "no base.qcow2; run vm.sh create"
running base && die "stop the base VM first"
local v
for v in en zh; do
running "$v" && vm_stop "$v"
rm -f "$D/$v.qcow2"
qemu-img create -q -f qcow2 -b "$D/base.qcow2" -F qcow2 "$D/$v.qcow2"
done
qemu_start zh "$D/zh.qcow2"
vm_wait zh
vm_ssh zh Administrator 'Set-WinSystemLocale -SystemLocale zh-CN; Set-Culture zh-CN; Set-WinHomeLocation -GeoId 45; Restart-Computer -Force' || true
sleep 20
vm_wait zh
echo "zh ANSI code page: $(vm_ssh zh Administrator '[Text.Encoding]::Default.CodePage')"
vm_stop zh
echo "OK: en.qcow2 zh.qcow2"
}
cmd="${1:-}"
shift || true
case "$cmd" in
create) cmd_create ;;
variants) cmd_variants ;;
provision) provision "${1:?variant}" ;;
start)
qemu_start "${1:?variant}" "$(disk_of "$1")"
vm_wait "$1"
;;
stop) vm_stop "${1:?variant}" ;;
status) running "${1:?variant}" && echo running || echo stopped ;;
wait) vm_wait "${1:?variant}" "${2:-900}" ;;
ssh)
v="${1:?variant}"
shift
u=Administrator
if [[ $# -gt 0 ]] && [[ "$1" == Administrator || "$1" == tester || "$1" == 测试 ]]; then
u="$1"
shift
fi
vm_ssh "$v" "$u" "$@"
;;
put) vm_put "$@" ;;
ps)
v="${1:?variant}"
shift
u=Administrator
if [[ "$1" == Administrator || "$1" == tester || "$1" == 测试 ]]; then
u="$1"
shift
fi
vm_ps "$v" "$u" "$@"
;;
shot)
v="${1:?variant}"
out="${2:-$D/$v.png}"
printf '%s\n' '{"execute":"qmp_capabilities"}' \
'{"execute":"screendump","arguments":{"filename":"'"$D/$v.ppm"'"}}' |
socat - UNIX-CONNECT:"$D/$v.qmp" >/dev/null
sleep 1
magick "$D/$v.ppm" -resize '1024x1024>' "$out"
rm -f "$D/$v.ppm"
echo "$out"
;;
type) python3 "$HERE/type.py" "$D/${1:?variant}.qmp" "${2:?text}" ;;
login)
python3 "$HERE/type.py" "$D/${1:?variant}.qmp" --cad
sleep 3
python3 "$HERE/type.py" "$D/$1.qmp" "@$D/admin-password"
;;
*) sed -n '2,15p' "$0" | sed 's/^# \{0,1\}//'; exit 2 ;;
esac
+25
View File
@@ -0,0 +1,25 @@
# Windows x86_64 cross toolchain (mingw-w64, UCRT runtime, static winpthreads).
# nix-shell shell-win.nix --run 'scripts/bootstrap_windows.sh'
{ pkgs ? import <nixpkgs> { } }:
let
win = pkgs.pkgsCross.ucrt64;
in
win.mkShell {
nativeBuildInputs = with pkgs; [
cmake
ninja
gnumake
git
perl
python3
pkg-config
zip
curl
nsis
];
buildInputs = [ win.windows.pthreads ];
# mkShell exports CC/AR for the target; the build scripts call the prefixed tools directly.
shellHook = ''
export MINGW_HOST=x86_64-w64-mingw32
'';
}
+1135 -252
View File
File diff suppressed because it is too large Load Diff
+26 -23
View File
@@ -6,7 +6,6 @@
#include "cst_agent.h"
#include "os_compat.h"
#include <dirent.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
@@ -45,7 +44,7 @@ int cst_agent_store_ensure(const char *workspace) {
char gi[OS_PATH_MAX];
if (os_path_join(gi, sizeof(gi), base, ".gitignore") == 0 &&
OS_ACCESS(gi, OS_F_OK) != 0) {
FILE *fp = fopen(gi, "w");
FILE *fp = fopen(gi, "wb");
if (fp) {
fputs("*\n", fp);
fclose(fp);
@@ -58,7 +57,8 @@ static void make_id(char *out, size_t n) {
time_t now = time(NULL);
struct tm tm;
os_localtime(&now, &tm);
unsigned r = (unsigned)(os_getpid() ^ (unsigned)now);
static unsigned seq;
unsigned r = (unsigned)(os_getpid() ^ (unsigned)now) + 0x9e37u * seq++;
snprintf(out, n, "%04d%02d%02d-%02d%02d%02d-%04x", tm.tm_year + 1900,
tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec,
r & 0xffff);
@@ -97,7 +97,7 @@ int cst_agent_store_set_current(const char *workspace, const char *id) {
return -1;
if (os_path_join(path, sizeof(path), dir, "CURRENT") != 0)
return -1;
FILE *fp = fopen(path, "w");
FILE *fp = fopen(path, "wb");
if (!fp)
return -1;
fprintf(fp, "%s\n", id);
@@ -113,7 +113,7 @@ int cst_agent_store_get_current(const char *workspace, char *id_out,
return -1;
if (os_path_join(path, sizeof(path), dir, "CURRENT") != 0)
return -1;
FILE *fp = fopen(path, "r");
FILE *fp = fopen(path, "rb");
if (!fp)
return -1;
if (!fgets(id_out, (int)id_sz, fp)) {
@@ -159,7 +159,7 @@ int cst_agent_store_list(const char *workspace, char ids[][128], int max_ids) {
char dir[OS_PATH_MAX];
if (cst_agent_store_dir(workspace, dir, sizeof(dir)) != 0)
return 0;
DIR *d = opendir(dir);
os_dir_t *d = os_dir_open(dir);
if (!d)
return 0;
typedef struct {
@@ -168,23 +168,23 @@ int cst_agent_store_list(const char *workspace, char ids[][128], int max_ids) {
} ent_t;
ent_t *ents = calloc((size_t)max_ids * 4, sizeof(ent_t));
int n = 0, cap = max_ids * 4;
struct dirent *de;
while ((de = readdir(d)) != NULL) {
size_t len = strlen(de->d_name);
if (len < 7 || strcmp(de->d_name + len - 6, ".jsonl") != 0)
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL) {
size_t len = strlen(de->name);
if (len < 7 || strcmp(de->name + len - 6, ".jsonl") != 0)
continue;
if (n >= cap)
break;
snprintf(ents[n].id, sizeof(ents[n].id), "%.*s", (int)(len - 6),
de->d_name);
de->name);
char full[OS_PATH_MAX];
if (os_path_join(full, sizeof(full), dir, de->d_name) != 0)
if (os_path_join(full, sizeof(full), dir, de->name) != 0)
continue;
struct stat st;
ents[n].mtime = (stat(full, &st) == 0) ? st.st_mtime : 0;
n++;
}
closedir(d);
os_dir_close(d);
for (int i = 0; i < n; i++) {
for (int j = i + 1; j < n; j++) {
if (ents[j].mtime > ents[i].mtime) {
@@ -218,7 +218,7 @@ int cst_agent_store_title_set(const char *workspace, const char *id,
return -1;
if (!title || !title[0])
return remove(p) == 0 || errno == ENOENT ? 0 : -1;
FILE *fp = fopen(p, "w");
FILE *fp = fopen(p, "wb");
if (!fp)
return -1;
for (const char *c = title; *c; c++)
@@ -255,7 +255,7 @@ int cst_agent_store_info(const char *workspace, const char *id,
info->mtime = (long long)st.st_mtime;
char tp[OS_PATH_MAX];
if (side_path(workspace, id, ".title", tp, sizeof(tp)) == 0) {
FILE *tf = fopen(tp, "r");
FILE *tf = fopen(tp, "rb");
if (tf) {
if (fgets(info->title, sizeof(info->title), tf)) {
size_t n = strlen(info->title);
@@ -265,7 +265,7 @@ int cst_agent_store_info(const char *workspace, const char *id,
fclose(tf);
}
}
FILE *fp = fopen(p, "r");
FILE *fp = fopen(p, "rb");
if (!fp)
return 0;
size_t cap = 1 << 16;
@@ -289,7 +289,10 @@ int cst_agent_store_info(const char *workspace, const char *id,
cJSON *o = cJSON_Parse(line);
const cJSON *r = cJSON_GetObjectItemCaseSensitive(o, "role");
const cJSON *c = cJSON_GetObjectItemCaseSensitive(o, "content");
if (cJSON_IsString(r) && strcmp(r->valuestring, "user") == 0 &&
const cJSON *k = cJSON_GetObjectItemCaseSensitive(
cJSON_GetObjectItemCaseSensitive(o, "meta"), "kind");
int note = cJSON_IsString(k) && strcmp(k->valuestring, "note") == 0;
if (cJSON_IsString(r) && strcmp(r->valuestring, "user") == 0 && !note &&
cJSON_IsString(c) && c->valuestring[0]) {
clip_line(info->first_user, sizeof(info->first_user), c->valuestring);
have_first = 1;
@@ -311,16 +314,16 @@ int cst_agent_store_delete(const char *workspace, const char *id) {
remove(p);
/* sub-agent transcripts live in <id>/ */
if (side_path(workspace, id, "", p, sizeof(p)) == 0) {
DIR *d = opendir(p);
os_dir_t *d = os_dir_open(p);
if (d) {
struct dirent *de;
while ((de = readdir(d)) != NULL) {
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL) {
char f[OS_PATH_MAX];
if (strncmp(de->d_name, "sub-", 4) == 0 &&
os_path_join(f, sizeof(f), p, de->d_name) == 0)
if (strncmp(de->name, "sub-", 4) == 0 &&
os_path_join(f, sizeof(f), p, de->name) == 0)
remove(f);
}
closedir(d);
os_dir_close(d);
rmdir(p);
}
}
+751
View File
@@ -0,0 +1,751 @@
#ifndef _DEFAULT_SOURCE
#define _DEFAULT_SOURCE
#endif
#include "cJSON.h"
#include "cst_attach.h"
#include "cst_pdf.h"
#include "cst_util.h"
#include "os_compat.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* --- base64 -------------------------------------------------------------- */
static const char B64[] =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
char *cst_base64_encode(const void *data, size_t n) {
const unsigned char *p = data;
size_t olen = ((n + 2) / 3) * 4;
char *out = malloc(olen + 1);
if (!out)
return NULL;
size_t o = 0;
for (size_t i = 0; i < n; i += 3) {
unsigned v = (unsigned)p[i] << 16;
if (i + 1 < n)
v |= (unsigned)p[i + 1] << 8;
if (i + 2 < n)
v |= (unsigned)p[i + 2];
out[o++] = B64[(v >> 18) & 63];
out[o++] = B64[(v >> 12) & 63];
out[o++] = i + 1 < n ? B64[(v >> 6) & 63] : '=';
out[o++] = i + 2 < n ? B64[v & 63] : '=';
}
out[o] = 0;
return out;
}
static int b64_val(unsigned char c) {
if (c >= 'A' && c <= 'Z')
return c - 'A';
if (c >= 'a' && c <= 'z')
return c - 'a' + 26;
if (c >= '0' && c <= '9')
return c - '0' + 52;
if (c == '+' || c == '-')
return 62;
if (c == '/' || c == '_')
return 63;
return -1;
}
char *cst_base64_decode(const char *s, size_t n, size_t *out_len) {
if (out_len)
*out_len = 0;
if (!s)
return NULL;
char *out = malloc(n / 4 * 3 + 4);
if (!out)
return NULL;
size_t o = 0;
unsigned acc = 0;
int bits = 0;
for (size_t i = 0; i < n; i++) {
unsigned char c = (unsigned char)s[i];
if (c == '=')
break;
if (c == ' ' || c == '\t' || c == '\r' || c == '\n')
continue;
int v = b64_val(c);
if (v < 0) {
free(out);
return NULL;
}
acc = (acc << 6) | (unsigned)v;
bits += 6;
if (bits >= 8) {
bits -= 8;
out[o++] = (char)((acc >> bits) & 0xFF);
}
}
if (out_len)
*out_len = o;
return out;
}
/* --- format sniffing ----------------------------------------------------- */
static int be32(const unsigned char *p) {
return (p[0] << 24) | (p[1] << 16) | (p[2] << 8) | p[3];
}
static int le32(const unsigned char *p) {
return (int)((unsigned)p[0] | ((unsigned)p[1] << 8) |
((unsigned)p[2] << 16) | ((unsigned)p[3] << 24));
}
static int le16(const unsigned char *p) {
return (int)((unsigned)p[0] | ((unsigned)p[1] << 8));
}
static const char *mime_of_ext(const char *mime) {
if (strcmp(mime, "application/pdf") == 0 ||
strcmp(mime, "text/plain") == 0)
return "txt"; /* a document is stored as the text it was reduced to */
if (strcmp(mime, "image/png") == 0)
return "png";
if (strcmp(mime, "image/jpeg") == 0)
return "jpg";
if (strcmp(mime, "image/gif") == 0)
return "gif";
if (strcmp(mime, "image/webp") == 0)
return "webp";
if (strcmp(mime, "image/bmp") == 0)
return "bmp";
return "bin";
}
/* A JPEG SOF marker carries the frame size; skip it when scanning. */
static int jpeg_sof(int marker) {
switch (marker) {
case 0xC0:
case 0xC1:
case 0xC2:
case 0xC3:
case 0xC5:
case 0xC6:
case 0xC7:
case 0xC9:
case 0xCA:
case 0xCB:
case 0xCD:
case 0xCE:
case 0xCF:
return 1;
default:
return 0;
}
}
int cst_attach_sniff(const void *data, size_t n, char *mime, size_t mime_sz) {
const unsigned char *p = data;
const char *m = NULL;
if (n >= 8 && memcmp(p, "\x89PNG\r\n\x1a\n", 8) == 0)
m = "image/png";
else if (n >= 3 && p[0] == 0xFF && p[1] == 0xD8 && p[2] == 0xFF)
m = "image/jpeg";
else if (n >= 6 && memcmp(p, "GIF87a", 6) == 0)
m = "image/gif";
else if (n >= 6 && memcmp(p, "GIF89a", 6) == 0)
m = "image/gif";
else if (n >= 12 && memcmp(p, "RIFF", 4) == 0 && memcmp(p + 8, "WEBP", 4) == 0)
m = "image/webp";
else if (n >= 2 && p[0] == 'B' && p[1] == 'M')
m = "image/bmp";
if (!m)
return -1;
snprintf(mime, mime_sz, "%s", m);
return 0;
}
/* PNG / GIF / BMP are direct; JPEG needs a marker walk; WebP has three
* container flavours. */
int cst_attach_dimensions(const void *data, size_t n, int *w, int *h) {
const unsigned char *p = data;
*w = *h = 0;
char mime[32];
if (cst_attach_sniff(data, n, mime, sizeof(mime)) != 0)
return -1;
if (strcmp(mime, "image/png") == 0) {
/* 8 signature + 4 length + "IHDR" + w + h */
if (n < 24 || memcmp(p + 12, "IHDR", 4) != 0)
return -1;
*w = be32(p + 16);
*h = be32(p + 20);
return 0;
}
if (strcmp(mime, "image/gif") == 0) {
if (n < 10)
return -1;
*w = le16(p + 6);
*h = le16(p + 8);
return 0;
}
if (strcmp(mime, "image/bmp") == 0) {
if (n < 26)
return -1;
int wi = le32(p + 18);
int hi = le32(p + 22);
if (wi < 0)
wi = -wi; /* bottom-up rows are stored with a negative height */
if (hi < 0)
hi = -hi;
*w = wi;
*h = hi;
return 0;
}
if (strcmp(mime, "image/webp") == 0) {
if (n < 30)
return -1;
const unsigned char *c = p + 12;
if (memcmp(c, "VP8 ", 4) == 0) {
/* 4 tag + 3 start code, then 16-bit width / height (14 bits each) */
if (n < 30 || !(c[7] == 0x9D && c[8] == 0x01 && c[9] == 0x2A))
return -1;
*w = le16(c + 10) & 0x3FFF;
*h = le16(c + 12) & 0x3FFF;
return 0;
}
if (memcmp(c, "VP8L", 4) == 0) {
if (n < 25 || c[8] != 0x2F)
return -1;
unsigned b = (unsigned)c[9] | ((unsigned)c[10] << 8) |
((unsigned)c[11] << 16) | ((unsigned)c[12] << 24);
*w = (int)(b & 0x3FFF) + 1;
*h = (int)((b >> 14) & 0x3FFF) + 1;
return 0;
}
if (memcmp(c, "VP8X", 4) == 0) {
if (n < 30)
return -1;
*w = (int)((unsigned)c[12] | ((unsigned)c[13] << 8) |
((unsigned)c[14] << 16)) + 1;
*h = (int)((unsigned)c[15] | ((unsigned)c[16] << 8) |
((unsigned)c[17] << 16)) + 1;
return 0;
}
return -1;
}
if (strcmp(mime, "image/jpeg") == 0) {
size_t i = 2;
while (i + 3 < n) {
if (p[i] != 0xFF)
return -1; /* lost sync */
unsigned marker = p[i + 1];
i += 2;
if (marker == 0xD8 || (marker >= 0xD0 && marker <= 0xD7))
continue; /* standalone markers: no payload */
if (marker == 0xD9 || marker == 0x01)
continue;
if (i + 1 >= n)
return -1;
size_t seg = (size_t)((p[i] << 8) | p[i + 1]);
if (seg < 2 || i + seg > n)
return -1;
if (jpeg_sof((int)marker) && seg >= 7) {
*h = (p[i + 3] << 8) | p[i + 4];
*w = (p[i + 5] << 8) | p[i + 6];
return 0;
}
i += seg;
}
}
return -1;
}
/* --- storage ------------------------------------------------------------- */
/* <session dir>/attachments/<session stem> — beside the JSONL so moving the
* workspace keeps the two together. */
int cst_attach_dir(const char *session_path, char *out, size_t out_sz) {
if (!session_path || !session_path[0])
return -1;
char dir[OS_PATH_MAX];
snprintf(dir, sizeof(dir), "%s", session_path);
char *slash = strrchr(dir, '/');
#ifdef _WIN32
char *bslash = strrchr(dir, '\\');
if (bslash && (!slash || bslash > slash))
slash = bslash;
#endif
char leaf[256];
const char *base = slash ? slash + 1 : dir;
size_t bl = strlen(base);
if (bl >= sizeof(leaf))
bl = sizeof(leaf) - 1;
memcpy(leaf, base, bl);
leaf[bl] = 0;
if (slash)
*(slash + 1) = 0;
else {
dir[0] = '.';
dir[1] = 0;
}
char *dot = strstr(leaf, ".jsonl");
if (dot)
*dot = 0;
char rel[320];
snprintf(rel, sizeof(rel), "attachments/%s", leaf[0] ? leaf : "session");
char full[OS_PATH_MAX];
if (os_path_join(full, sizeof(full), dir, rel) != 0)
return -1;
if (os_mkdir_p(full, 0700) != 0)
return -1;
snprintf(out, out_sz, "%s", full);
return 0;
}
/* Stored names are "<id>.<ext>" made by store(); anything else in a JSONL
* (hand-edited or crafted) could point outside the attachment directory. */
int cst_attach_rel_ok(const char *rel) {
if (!rel || !rel[0] || rel[0] == '.')
return 0;
for (const char *p = rel; *p; p++) {
char c = *p;
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '-' || c == '_' || c == '.'))
return 0;
}
return strstr(rel, "..") == NULL;
}
int cst_attach_full_path(const char *session_path, const cst_attachment_t *a,
char *out, size_t out_sz) {
if (!a || !cst_attach_rel_ok(a->rel))
return -1;
char dir[OS_PATH_MAX];
if (cst_attach_dir(session_path, dir, sizeof(dir)) != 0)
return -1;
return os_path_join(out, out_sz, dir, a->rel);
}
static char *read_whole(const char *path, size_t *out_len, size_t max) {
FILE *f = fopen(path, "rb");
if (!f)
return NULL;
cst_buf_t b = {0};
char chunk[16384];
size_t n;
while ((n = fread(chunk, 1, sizeof(chunk), f)) > 0) {
if (b.len + n > max || cst_buf_append(&b, chunk, n) != 0) {
cst_buf_free(&b);
fclose(f);
return NULL;
}
}
fclose(f);
*out_len = b.len;
return cst_buf_steal(&b);
}
/* FNV over the bytes, the name and the clock: short, stable, unique enough
* inside one session. */
static void make_id(const char *name, const char *data, size_t n, char *out,
size_t out_sz) {
uint64_t h = cst_fnv1a64(CST_FNV_INIT, data, n > 4096 ? 4096 : n);
h = cst_fnv1a64(h, name, strlen(name));
h = cst_fnv1a64(h, &n, sizeof(n));
double now = os_now_ms();
h = cst_fnv1a64(h, &now, sizeof(now));
h = cst_fnv1a64(h, &(int){os_getpid()}, sizeof(int));
snprintf(out, out_sz, "img-%08x", (unsigned)(h & 0xFFFFFFFFu));
}
/* n bytes, NUL-terminated: a text payload is handed around as a C string. */
static char *memdup_z(const char *data, size_t n) {
char *copy = malloc(n + 1);
if (!copy)
return NULL;
memcpy(copy, data, n);
copy[n] = 0;
return copy;
}
/* What a picked file is, plus the payload to store for it: an image stays as
* it is, a PDF is reduced to its text layer (the mime still says PDF, the
* stored file holds text), anything else must be UTF-8 text. The caller frees
* the result; *out_len is its length. */
static char *prepare(const char *data, size_t n, char *mime, size_t mime_sz,
int *pages, size_t *out_len, char *err, size_t err_sz) {
*pages = 0;
*out_len = n;
if (cst_attach_sniff(data, n, mime, mime_sz) == 0) {
if (n > CST_ATTACH_MAX_BYTES) {
snprintf(err, err_sz, "image is %.1f MB; the limit is %d MB",
(double)n / (1024 * 1024), (int)(CST_ATTACH_MAX_BYTES >> 20));
return NULL;
}
char *copy = memdup_z(data, n);
if (!copy)
snprintf(err, err_sz, "out of memory");
return copy;
}
if (n >= 4 && memcmp(data, "%PDF", 4) == 0) {
int pg = 0;
char perr[256];
char *text = cst_pdf_text(data, n, &pg, perr, sizeof(perr));
if (!text) {
snprintf(err, err_sz, "cannot read the PDF: %.200s", perr);
return NULL;
}
snprintf(mime, mime_sz, "application/pdf");
*pages = pg;
*out_len = strlen(text);
return text;
}
if (n > CST_ATTACH_TEXT_MAX) {
snprintf(err, err_sz, "text is %.1f MB; the limit is %d MB",
(double)n / (1024 * 1024), (int)(CST_ATTACH_TEXT_MAX >> 20));
return NULL;
}
if (memchr(data, 0, n) || !cst_utf8_valid(data, n)) {
snprintf(err, err_sz,
"not an image (PNG, JPEG, GIF, WebP or BMP), a PDF or UTF-8 "
"text; the content decides, not the extension");
return NULL;
}
snprintf(mime, mime_sz, "text/plain");
char *copy = memdup_z(data, n);
if (!copy)
snprintf(err, err_sz, "out of memory");
return copy;
}
/* Write one payload whose kind is already known. */
static cst_attachment_t *store(const char *session_path, const char *name,
const char *data, size_t n, const char *mime,
int pages, const char *origin, char *err,
size_t err_sz) {
if (!name || !name[0])
name = "image";
char dir[OS_PATH_MAX];
if (cst_attach_dir(session_path, dir, sizeof(dir)) != 0) {
snprintf(err, err_sz, "cannot create the attachment directory");
return NULL;
}
cst_attachment_t *a = calloc(1, sizeof(*a));
if (!a) {
snprintf(err, err_sz, "out of memory");
return NULL;
}
make_id(name, data, n, a->id, sizeof(a->id));
snprintf(a->rel, sizeof(a->rel), "%s.%s", a->id, mime_of_ext(mime));
snprintf(a->name, sizeof(a->name), "%s", name && name[0] ? name : "image");
snprintf(a->mime, sizeof(a->mime), "%s", mime);
snprintf(a->origin, sizeof(a->origin), "%s", origin ? origin : "user");
snprintf(a->detail, sizeof(a->detail), "auto");
a->bytes = (long long)n;
a->pages = pages;
if (cst_attach_is_image(a))
cst_attach_dimensions(data, n, &a->width, &a->height);
char full[OS_PATH_MAX];
if (cst_attach_full_path(session_path, a, full, sizeof(full)) != 0) {
snprintf(err, err_sz, "attachment path is too long");
free(a);
return NULL;
}
FILE *f = fopen(full, "wb");
if (!f) {
snprintf(err, err_sz, "cannot write %.200s", full);
free(a);
return NULL;
}
size_t w = fwrite(data, 1, n, f);
if (fclose(f) != 0 || w != n) {
snprintf(err, err_sz, "cannot write %.200s", full);
free(a);
return NULL;
}
return a;
}
cst_attachment_t *cst_attach_import_bytes(const char *session_path,
const char *name, const char *data,
size_t n, char *err, size_t err_sz) {
char dummy[8];
if (!err || !err_sz) {
err = dummy;
err_sz = sizeof(dummy);
}
err[0] = 0;
if (!session_path || !session_path[0]) {
snprintf(err, err_sz, "no session to store the file in");
return NULL;
}
if (!data || !n) {
snprintf(err, err_sz, "the file is empty");
return NULL;
}
char mime[32];
if (cst_attach_sniff(data, n, mime, sizeof(mime)) != 0) {
snprintf(err, err_sz,
"not a supported image (PNG, JPEG, GIF, WebP or BMP; "
"checked by content, not by extension)");
return NULL;
}
if (n > CST_ATTACH_MAX_BYTES) {
snprintf(err, err_sz, "image is %.1f MB; the limit is %d MB",
(double)n / (1024 * 1024), (int)(CST_ATTACH_MAX_BYTES >> 20));
return NULL;
}
return store(session_path, name, data, n, mime, 0, "user", err, err_sz);
}
cst_attachment_t *cst_attach_import_file(const char *session_path,
const char *src_path, char *err,
size_t err_sz) {
char dummy[8];
if (!err || !err_sz) {
err = dummy;
err_sz = sizeof(dummy);
}
err[0] = 0;
size_t n = 0;
char *data = read_whole(src_path, &n, CST_ATTACH_MAX_BYTES);
if (!data) {
snprintf(err, err_sz, "cannot read %.300s (or it is larger than %d MB)",
src_path, (int)(CST_ATTACH_MAX_BYTES >> 20));
return NULL;
}
const char *leaf = src_path;
for (const char *p = src_path; *p; p++)
if (*p == '/' || *p == '\\')
leaf = p + 1;
char mime[32];
int pages = 0;
size_t pn = 0;
char *payload =
prepare(data, n, mime, sizeof(mime), &pages, &pn, err, err_sz);
free(data);
if (!payload)
return NULL;
cst_attachment_t *a =
store(session_path, leaf, payload, pn, mime, pages, "user", err, err_sz);
free(payload);
return a;
}
/* Images are opaque and travel as image parts; everything else is text the
* store holds and the request inlines. */
int cst_attach_is_image(const cst_attachment_t *a) {
return a && strncmp(a->mime, "image/", 6) == 0;
}
int cst_attach_is_text(const cst_attachment_t *a) {
return a && !cst_attach_is_image(a);
}
char *cst_attach_read(const char *session_path, const cst_attachment_t *a,
size_t *out_len, char *err, size_t err_sz) {
char dummy[8];
if (!err || !err_sz) {
err = dummy;
err_sz = sizeof(dummy);
}
err[0] = 0;
char full[OS_PATH_MAX];
if (cst_attach_full_path(session_path, a, full, sizeof(full)) != 0) {
snprintf(err, err_sz, "attachment has no stored file");
return NULL;
}
char *data = read_whole(full, out_len, CST_ATTACH_MAX_BYTES);
if (!data) {
snprintf(err, err_sz,
"%.200s is missing; it was deleted from the session store",
a->name);
return NULL;
}
if (!cst_attach_is_image(a)) /* a document is text, nothing to verify */
return data;
char mime[32];
if (cst_attach_sniff(data, *out_len, mime, sizeof(mime)) != 0) {
snprintf(err, err_sz, "%.200s in the session store is no longer an image",
a->name);
free(data);
return NULL;
}
return data;
}
char *cst_attach_data_url(const char *session_path, const cst_attachment_t *a) {
if (!cst_attach_is_image(a)) /* a document travels as text, not as a URL */
return NULL;
char err[256];
size_t n = 0;
char *data = cst_attach_read(session_path, a, &n, err, sizeof(err));
if (!data)
return NULL;
char mime[32];
cst_attach_sniff(data, n, mime, sizeof(mime)); /* the bytes decide */
char *b64 = cst_base64_encode(data, n);
free(data);
if (!b64)
return NULL;
cst_buf_t b = {0};
cst_buf_printf(&b, "data:%s;base64,", mime);
cst_buf_puts(&b, b64);
free(b64);
return cst_buf_steal(&b);
}
void cst_attach_describe(const cst_attachment_t *a, char *out, size_t out_sz) {
if (!a) {
snprintf(out, out_sz, "(none)");
return;
}
const char *kind = a->mime;
char dim[32] = "";
if (cst_attach_is_image(a)) {
if (strncmp(a->mime, "image/", 6) == 0)
kind = a->mime + 6;
if (a->width > 0 && a->height > 0)
snprintf(dim, sizeof(dim), "%dx%d ", a->width, a->height);
} else if (strcmp(a->mime, "application/pdf") == 0) {
kind = "pdf";
if (a->pages > 0)
snprintf(dim, sizeof(dim), "%d page%s ", a->pages,
a->pages == 1 ? "" : "s");
} else if (strcmp(a->mime, "text/plain") == 0 || !a->mime[0]) {
kind = "text";
}
out[0] = 0;
if (a->bytes >= 1024 * 1024)
snprintf(out, out_sz, "%s%s %.1f MB", dim, kind,
(double)a->bytes / (1024 * 1024));
else
snprintf(out, out_sz, "%s%s %.0f KB", dim, kind,
(double)a->bytes / 1024);
}
/* --- lifetime ------------------------------------------------------------ */
void cst_attach_free(cst_attachment_t *a) { free(a); }
/* Entries always live in one array (a 1-element array for a single import),
* so freeing the block releases them all. */
void cst_attachments_free(cst_attachment_t *list, int n) {
(void)n;
free(list);
}
cst_attachment_t *cst_attachments_dup(const cst_attachment_t *list, int n) {
if (!list || n <= 0)
return NULL;
cst_attachment_t *out = calloc((size_t)n, sizeof(*out));
if (!out)
return NULL;
for (int i = 0; i < n; i++)
out[i] = list[i];
return out;
}
int cst_attachments_push(cst_attachment_t **list, int *n, int *cap,
const cst_attachment_t *src) {
if (!list || !n || !cap || !src)
return -1;
if (*n == *cap) {
int nc = *cap ? *cap * 2 : 4;
cst_attachment_t *nv = realloc(*list, (size_t)nc * sizeof(**list));
if (!nv)
return -1;
*list = nv;
*cap = nc;
}
(*list)[(*n)++] = *src;
return 0;
}
/* --- JSONL --------------------------------------------------------------- */
char *cst_attachments_to_json(const cst_attachment_t *list, int n) {
if (!list || n <= 0)
return NULL;
cJSON *arr = cJSON_CreateArray();
if (!arr)
return NULL;
for (int i = 0; i < n; i++) {
const cst_attachment_t *a = &list[i];
cJSON *o = cJSON_CreateObject();
cJSON_AddStringToObject(o, "id", a->id);
cJSON_AddStringToObject(o, "name", a->name);
cJSON_AddStringToObject(o, "mime", a->mime);
cJSON_AddStringToObject(o, "rel", a->rel);
cJSON_AddNumberToObject(o, "bytes", (double)a->bytes);
if (a->width > 0 && a->height > 0) {
cJSON_AddNumberToObject(o, "width", a->width);
cJSON_AddNumberToObject(o, "height", a->height);
}
if (a->pages > 0)
cJSON_AddNumberToObject(o, "pages", a->pages);
if (a->detail[0] && strcmp(a->detail, "auto") != 0)
cJSON_AddStringToObject(o, "detail", a->detail);
if (a->origin[0])
cJSON_AddStringToObject(o, "origin", a->origin);
cJSON_AddItemToArray(arr, o);
}
char *s = cJSON_PrintUnformatted(arr);
cJSON_Delete(arr);
return s;
}
cst_attachment_t *cst_attachments_from_json(const void *array_json, int *n_out) {
if (n_out)
*n_out = 0;
const cJSON *arr = array_json;
if (!cJSON_IsArray(arr))
return NULL;
int n = cJSON_GetArraySize(arr);
if (n <= 0)
return NULL;
cst_attachment_t *list = calloc((size_t)n, sizeof(*list));
if (!list)
return NULL;
int k = 0;
const cJSON *it = NULL;
cJSON_ArrayForEach(it, arr) {
if (!cJSON_IsObject(it))
continue;
const cJSON *jid = cJSON_GetObjectItemCaseSensitive(it, "id");
const cJSON *jrel = cJSON_GetObjectItemCaseSensitive(it, "rel");
if (!cJSON_IsString(jid) || !cJSON_IsString(jrel) ||
!cst_attach_rel_ok(jrel->valuestring))
continue;
cst_attachment_t *a = &list[k];
snprintf(a->id, sizeof(a->id), "%s", jid->valuestring);
snprintf(a->rel, sizeof(a->rel), "%s", jrel->valuestring);
const cJSON *v = cJSON_GetObjectItemCaseSensitive(it, "name");
snprintf(a->name, sizeof(a->name), "%s",
cJSON_IsString(v) ? v->valuestring : a->id);
v = cJSON_GetObjectItemCaseSensitive(it, "mime");
snprintf(a->mime, sizeof(a->mime), "%s",
cJSON_IsString(v) ? v->valuestring : "image/*");
v = cJSON_GetObjectItemCaseSensitive(it, "bytes");
a->bytes = cJSON_IsNumber(v) ? (long long)v->valuedouble : 0;
v = cJSON_GetObjectItemCaseSensitive(it, "width");
a->width = cJSON_IsNumber(v) ? v->valueint : 0;
v = cJSON_GetObjectItemCaseSensitive(it, "height");
a->height = cJSON_IsNumber(v) ? v->valueint : 0;
v = cJSON_GetObjectItemCaseSensitive(it, "pages");
a->pages = cJSON_IsNumber(v) ? v->valueint : 0;
v = cJSON_GetObjectItemCaseSensitive(it, "detail");
snprintf(a->detail, sizeof(a->detail), "%s",
cJSON_IsString(v) ? v->valuestring : "auto");
v = cJSON_GetObjectItemCaseSensitive(it, "origin");
snprintf(a->origin, sizeof(a->origin), "%s",
cJSON_IsString(v) ? v->valuestring : "");
k++;
}
if (k == 0) {
free(list);
return NULL;
}
if (n_out)
*n_out = k;
return list;
}
+651
View File
@@ -0,0 +1,651 @@
#include "cst_clipboard.h"
#include "cst_attach.h"
#include "os_compat.h"
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#include <windows.h>
#else
#include <unistd.h>
#endif
static void set_err(char *err, size_t err_sz, const char *msg) {
if (err && err_sz)
snprintf(err, err_sz, "%s", msg);
}
static void put_le32(unsigned char *p, unsigned long v) {
p[0] = (unsigned char)(v & 0xff);
p[1] = (unsigned char)((v >> 8) & 0xff);
p[2] = (unsigned char)((v >> 16) & 0xff);
p[3] = (unsigned char)((v >> 24) & 0xff);
}
static void put_be32(unsigned char *p, unsigned long v) {
p[0] = (unsigned char)((v >> 24) & 0xff);
p[1] = (unsigned char)((v >> 16) & 0xff);
p[2] = (unsigned char)((v >> 8) & 0xff);
p[3] = (unsigned char)(v & 0xff);
}
static unsigned long get_le32(const unsigned char *p) {
return (unsigned long)p[0] | (unsigned long)p[1] << 8 |
(unsigned long)p[2] << 16 | (unsigned long)p[3] << 24;
}
static unsigned long crc32_update(unsigned long crc, const unsigned char *p,
size_t n) {
static unsigned long table[256];
static int ready;
if (!ready) {
for (unsigned long i = 0; i < 256; i++) {
unsigned long c = i;
for (int k = 0; k < 8; k++)
c = c & 1 ? 0xedb88320UL ^ (c >> 1) : c >> 1;
table[i] = c;
}
ready = 1;
}
crc ^= 0xffffffffUL;
for (size_t i = 0; i < n; i++)
crc = table[(crc ^ p[i]) & 0xff] ^ (crc >> 8);
return crc ^ 0xffffffffUL;
}
static unsigned char *png_chunk(unsigned char *o, const char *type,
const unsigned char *data, size_t n) {
put_be32(o, (unsigned long)n);
memcpy(o + 4, type, 4);
if (n)
memcpy(o + 8, data, n);
put_be32(o + 8 + n, crc32_update(0, o + 4, n + 4));
return o + 12 + n;
}
/* RGB rows (filter byte 0 each) in stored deflate blocks: no compressor in
* the tree, and clipboard screenshots stay well under the attach cap. */
static char *png_from_raw(const unsigned char *raw, size_t raw_n, long w,
long h, size_t *n) {
size_t blocks = raw_n / 65535 + 1;
size_t z_n = 2 + raw_n + blocks * 5 + 4;
size_t total = 8 + (12 + 13) + (12 + z_n) + 12;
unsigned char *out = malloc(total);
unsigned char *z = malloc(z_n);
if (!out || !z) {
free(out);
free(z);
return NULL;
}
unsigned char *q = z;
*q++ = 0x78;
*q++ = 0x01;
unsigned long a = 1, b = 0;
size_t left = raw_n, pos = 0;
do {
size_t len = left > 65535 ? 65535 : left;
*q++ = left == len ? 1 : 0;
q[0] = (unsigned char)(len & 0xff);
q[1] = (unsigned char)(len >> 8);
q[2] = (unsigned char)(~len & 0xff);
q[3] = (unsigned char)((~len >> 8) & 0xff);
q += 4;
memcpy(q, raw + pos, len);
for (size_t i = 0; i < len; i++) {
a = (a + raw[pos + i]) % 65521;
b = (b + a) % 65521;
}
q += len;
pos += len;
left -= len;
} while (left);
put_be32(q, b << 16 | a);
q += 4;
unsigned char ihdr[13];
put_be32(ihdr, (unsigned long)w);
put_be32(ihdr + 4, (unsigned long)h);
ihdr[8] = 8; /* bit depth */
ihdr[9] = 2; /* RGB */
ihdr[10] = ihdr[11] = ihdr[12] = 0;
unsigned char *o = out;
memcpy(o, "\x89PNG\r\n\x1a\n", 8);
o = png_chunk(o + 8, "IHDR", ihdr, sizeof(ihdr));
o = png_chunk(o, "IDAT", z, (size_t)(q - z));
o = png_chunk(o, "IEND", NULL, 0);
free(z);
*n = (size_t)(o - out);
return (char *)out;
}
/* CF_DIB is a BMP without its 14-byte file header; the pixel offset depends
* on the header size, BI_BITFIELDS masks and the color table. */
char *cst_dib_to_image(const void *dib_v, size_t sz, size_t *n) {
const unsigned char *dib = dib_v;
if (!dib || sz < 40)
return NULL;
unsigned long hsize = get_le32(dib);
long w = (long)(int)get_le32(dib + 4);
long hh = (long)(int)get_le32(dib + 8);
unsigned bits = (unsigned)(dib[14] | dib[15] << 8);
unsigned long comp = get_le32(dib + 16);
unsigned long used = get_le32(dib + 32);
if (hsize < 40 || hsize > sz)
return NULL;
size_t off = hsize;
if (hsize == 40 && comp == 3 /* BI_BITFIELDS */)
off += 12;
size_t colors = used;
if (!colors && bits <= 8)
colors = (size_t)1 << bits;
if (colors > 256)
return NULL;
off += colors * 4;
if (off > sz)
return NULL;
long h = hh < 0 ? -hh : hh;
if ((bits == 24 || bits == 32) && (comp == 0 || comp == 3) && w > 0 &&
h > 0 && w <= 32768 && h <= 32768) {
size_t stride = ((size_t)w * bits / 8 + 3) & ~(size_t)3;
size_t raw_n = (size_t)h * (1 + (size_t)w * 3);
if (stride * (size_t)h <= sz - off && raw_n < CST_ATTACH_MAX_BYTES) {
unsigned char *raw = malloc(raw_n);
if (!raw)
return NULL;
size_t bpp = bits / 8;
for (long y = 0; y < h; y++) {
long src_y = hh > 0 ? h - 1 - y : y; /* positive height = bottom-up */
const unsigned char *s = dib + off + (size_t)src_y * stride;
unsigned char *d = raw + (size_t)y * (1 + (size_t)w * 3);
*d++ = 0;
for (long x = 0; x < w; x++, s += bpp) {
*d++ = s[2];
*d++ = s[1];
*d++ = s[0];
}
}
char *png = png_from_raw(raw, raw_n, w, h, n);
free(raw);
return png;
}
}
char *out = malloc(14 + sz);
if (!out)
return NULL;
unsigned char *hd = (unsigned char *)out;
hd[0] = 'B';
hd[1] = 'M';
put_le32(hd + 2, (unsigned long)(14 + sz));
put_le32(hd + 6, 0);
put_le32(hd + 10, (unsigned long)(14 + off));
memcpy(out + 14, dib, sz);
*n = 14 + sz;
return out;
}
#ifdef _WIN32
int cst_clipboard_image(char **data, size_t *n, char *err, size_t err_sz) {
*data = NULL;
*n = 0;
UINT png = RegisterClipboardFormatA("PNG");
if (!OpenClipboard(NULL)) {
set_err(err, err_sz, "could not open the clipboard");
return -1;
}
HANDLE h = NULL;
int is_png = 0;
if (png && IsClipboardFormatAvailable(png)) {
h = GetClipboardData(png);
is_png = 1;
} else if (IsClipboardFormatAvailable(CF_DIB)) {
h = GetClipboardData(CF_DIB);
}
const unsigned char *p = h ? GlobalLock(h) : NULL;
if (!p) {
CloseClipboard();
set_err(err, err_sz, "no image on the clipboard");
return -1;
}
size_t sz = (size_t)GlobalSize(h);
char *out = NULL;
if (sz > CST_ATTACH_MAX_BYTES) {
set_err(err, err_sz, "clipboard image is too large");
} else if (is_png) {
out = malloc(sz ? sz : 1);
if (out) {
memcpy(out, p, sz);
*n = sz;
}
} else {
out = cst_dib_to_image(p, sz, n);
if (!out)
set_err(err, err_sz, "clipboard bitmap is malformed");
}
GlobalUnlock(h);
CloseClipboard();
if (!out)
return -1;
*data = out;
return 0;
}
#else
/* Runs cmd (stderr already redirected by the caller); 127 = not installed. */
static int run_bytes(const char *cmd, char **data, size_t *n) {
os_run_result_t r;
memset(&r, 0, sizeof(r));
if (os_run_shell_c(NULL, cmd, CST_ATTACH_MAX_BYTES + 1, &r) != 0) {
os_run_result_free(&r);
return -1;
}
if (r.exit_code != 0 || r.out_len == 0) {
int code = r.exit_code == 127 ? 127 : -1;
os_run_result_free(&r);
return code;
}
*data = r.out;
*n = r.out_len;
return 0;
}
static int type_ok(const char *s, size_t n) {
if (n < 7 || n > 63 || strncmp(s, "image/", 6) != 0)
return 0;
for (size_t i = 0; i < n; i++)
if (!isalnum((unsigned char)s[i]) && !strchr("/.+-", s[i]))
return 0;
return 1;
}
/* Best image MIME type from a newline-separated list: PNG first. */
static int pick_type(const char *list, size_t len, char *out, size_t out_sz) {
out[0] = 0;
size_t i = 0;
while (i < len) {
size_t j = i;
while (j < len && list[j] != '\n' && list[j] != '\r')
j++;
if (type_ok(list + i, j - i)) {
int is_png = j - i == 9 && strncmp(list + i, "image/png", 9) == 0;
if (is_png || !out[0])
snprintf(out, out_sz, "%.*s", (int)(j - i), list + i);
if (is_png)
break;
}
i = j + 1;
}
return out[0] ? 0 : -1;
}
static int via_lister(const char *list_cmd, const char *get_fmt,
const char *tool, char **data, size_t *n, char *err,
size_t err_sz) {
char *types = NULL;
size_t tn = 0;
int rc = run_bytes(list_cmd, &types, &tn);
if (rc == 127) {
char msg[96];
snprintf(msg, sizeof(msg), "reading the clipboard needs %s", tool);
set_err(err, err_sz, msg);
return -1;
}
char type[64];
int ok = rc == 0 && pick_type(types, tn, type, sizeof(type)) == 0;
free(types);
if (!ok) {
set_err(err, err_sz, "no image on the clipboard");
return -1;
}
char cmd[256];
snprintf(cmd, sizeof(cmd), get_fmt, type);
if (run_bytes(cmd, data, n) != 0) {
set_err(err, err_sz, "could not read the clipboard image");
return -1;
}
return 0;
}
#ifdef __APPLE__
static int mac_image(char **data, size_t *n, char *err, size_t err_sz) {
int rc = run_bytes("pngpaste - 2>/dev/null", data, n);
if (rc == 0)
return 0;
char path[1024];
int fd = os_mkstemp(path, sizeof(path), "cst-clip");
if (fd < 0) {
set_err(err, err_sz, "could not create a temp file");
return -1;
}
close(fd);
char cmd[1600];
snprintf(cmd, sizeof(cmd),
"osascript -e 'set f to open for access (POSIX file \"%s\") with "
"write permission' -e 'set eof f to 0' -e 'write (the clipboard as "
"\xc2\xabclass PNGf\xc2\xbb) to f' -e 'close access f' "
">/dev/null 2>&1",
path);
os_run_result_t r;
memset(&r, 0, sizeof(r));
int ran = os_run_shell_c(NULL, cmd, 4096, &r);
int code = r.exit_code;
os_run_result_free(&r);
char *buf = NULL;
size_t len = 0;
if (ran == 0 && code == 0) {
FILE *f = fopen(path, "rb");
if (f) {
char chunk[65536];
size_t got;
while ((got = fread(chunk, 1, sizeof(chunk), f)) > 0 &&
len <= CST_ATTACH_MAX_BYTES) {
char *nb = realloc(buf, len + got);
if (!nb)
break;
buf = nb;
memcpy(buf + len, chunk, got);
len += got;
}
fclose(f);
}
}
remove(path);
if (!buf || !len) {
free(buf);
set_err(err, err_sz, "no image on the clipboard");
return -1;
}
*data = buf;
*n = len;
return 0;
}
#endif
int cst_clipboard_image(char **data, size_t *n, char *err, size_t err_sz) {
*data = NULL;
*n = 0;
#ifdef __APPLE__
return mac_image(data, n, err, err_sz);
#else
const char *wl = getenv("WAYLAND_DISPLAY");
if (wl && *wl)
return via_lister("wl-paste --list-types 2>/dev/null",
"wl-paste --no-newline --type '%s' 2>/dev/null",
"wl-clipboard (wl-paste)", data, n, err, err_sz);
const char *x = getenv("DISPLAY");
if (x && *x)
return via_lister("xclip -selection clipboard -t TARGETS -o 2>/dev/null",
"xclip -selection clipboard -t '%s' -o 2>/dev/null",
"xclip", data, n, err, err_sz);
set_err(err, err_sz, "no clipboard here (no Wayland or X11 display)");
return -1;
#endif
}
#endif
/* --- text clipboard ------------------------------------------------------ */
#ifdef _WIN32
int cst_clipboard_text(char **data, size_t *n, char *err, size_t err_sz) {
*data = NULL;
*n = 0;
if (!IsClipboardFormatAvailable(CF_UNICODETEXT) || !OpenClipboard(NULL)) {
set_err(err, err_sz, "no text on the clipboard");
return -1;
}
HANDLE h = GetClipboardData(CF_UNICODETEXT);
const wchar_t *w = h ? GlobalLock(h) : NULL;
if (!w) {
if (h)
GlobalUnlock(h);
CloseClipboard();
set_err(err, err_sz, "no text on the clipboard");
return -1;
}
char *utf8 = os_wide_to_utf8(w);
size_t len = utf8 ? strlen(utf8) : 0;
GlobalUnlock(h);
CloseClipboard();
if (!utf8 || !len) {
free(utf8);
set_err(err, err_sz, "no text on the clipboard");
return -1;
}
if (len > CST_CLIPBOARD_TEXT_MAX) {
free(utf8);
set_err(err, err_sz, "clipboard text is too large");
return -1;
}
*data = utf8;
*n = len;
return 0;
}
int cst_clipboard_set_text(const char *text, size_t n, char *err,
size_t err_sz) {
if (!text || os_clipboard_set(text, n) != 0) {
set_err(err, err_sz, "could not put text on the clipboard");
return -1;
}
return 0;
}
#else
/* stdout of a clipboard reader, capped and NUL-terminated. */
static int clip_read(const char *cmd, const char *tool, char **data, size_t *n,
char *err, size_t err_sz) {
os_run_result_t r;
memset(&r, 0, sizeof(r));
if (os_run_shell_c(NULL, cmd, CST_CLIPBOARD_TEXT_MAX + 1, &r) != 0) {
os_run_result_free(&r);
set_err(err, err_sz, "could not read the clipboard");
return -1;
}
if (r.exit_code != 0 || r.out_len == 0) {
int missing = r.exit_code == 127;
os_run_result_free(&r);
if (missing) {
char msg[96];
snprintf(msg, sizeof(msg), "reading the clipboard needs %s", tool);
set_err(err, err_sz, msg);
} else {
set_err(err, err_sz, "no text on the clipboard");
}
return -1;
}
if (r.out_len > CST_CLIPBOARD_TEXT_MAX) {
os_run_result_free(&r);
set_err(err, err_sz, "clipboard text is too large");
return -1;
}
char *copy = malloc(r.out_len + 1);
if (!copy) {
os_run_result_free(&r);
set_err(err, err_sz, "out of memory");
return -1;
}
memcpy(copy, r.out, r.out_len);
copy[r.out_len] = 0;
os_run_result_free(&r);
*data = copy;
*n = r.out_len;
return 0;
}
int cst_clipboard_text(char **data, size_t *n, char *err, size_t err_sz) {
*data = NULL;
*n = 0;
#ifdef __APPLE__
return clip_read("pbpaste 2>/dev/null", "pbpaste", data, n, err, err_sz);
#else
const char *wl = getenv("WAYLAND_DISPLAY");
if (wl && *wl)
return clip_read("wl-paste --type text/plain --no-newline 2>/dev/null",
"wl-clipboard (wl-paste)", data, n, err, err_sz);
const char *x = getenv("DISPLAY");
if (x && *x)
return clip_read("xclip -selection clipboard -o 2>/dev/null", "xclip",
data, n, err, err_sz);
set_err(err, err_sz, "no clipboard here (no Wayland or X11 display)");
return -1;
#endif
}
int cst_clipboard_set_text(const char *text, size_t n, char *err,
size_t err_sz) {
if (!text)
return -1;
if (n > CST_CLIPBOARD_TEXT_MAX) {
set_err(err, err_sz, "text is too large for the clipboard");
return -1;
}
const char *fmt = NULL, *tool = NULL;
#ifdef __APPLE__
fmt = "pbcopy < '%s'";
tool = "pbcopy";
#else
if (getenv("WAYLAND_DISPLAY")) {
fmt = "wl-copy < '%s'";
tool = "wl-clipboard (wl-copy)";
} else if (getenv("DISPLAY")) {
fmt = "xclip -selection clipboard -i < '%s'";
tool = "xclip";
}
#endif
if (!fmt) {
set_err(err, err_sz, "no clipboard here (no Wayland or X11 display)");
return -1;
}
char tmp[OS_PATH_MAX];
int fd = os_mkstemp(tmp, sizeof(tmp), "cst-clip");
if (fd < 0) {
set_err(err, err_sz, "could not create a temporary file");
return -1;
}
FILE *fp = fdopen(fd, "wb");
int ok = fp && fwrite(text, 1, n, fp) == n && fclose(fp) == 0;
if (!ok) {
if (fp)
fclose(fp);
remove(tmp);
set_err(err, err_sz, "could not write the text");
return -1;
}
char cmd[OS_PATH_MAX + 64];
snprintf(cmd, sizeof(cmd), fmt, tmp);
os_run_result_t r;
memset(&r, 0, sizeof(r));
int rc = os_run_shell_c(NULL, cmd, 4096, &r);
int code = r.exit_code;
os_run_result_free(&r);
remove(tmp);
if (rc != 0 || code != 0) {
char msg[96];
snprintf(msg, sizeof(msg), "writing the clipboard needs %s", tool);
set_err(err, err_sz, msg);
return -1;
}
return 0;
}
#endif
static int hexval(int c) {
if (c >= '0' && c <= '9')
return c - '0';
if (c >= 'a' && c <= 'f')
return c - 'a' + 10;
if (c >= 'A' && c <= 'F')
return c - 'A' + 10;
return -1;
}
/* file://[host]/path -> /path, percent-decoded; 0 on success. */
static int from_file_uri(char *s) {
const char *p = s + 7;
const char *slash = strchr(p, '/');
if (!slash)
return -1;
if (slash != p && !(slash - p == 9 && strncmp(p, "localhost", 9) == 0))
return -1;
char *o = s;
for (const char *q = slash; *q; q++) {
if (*q == '%' && hexval(q[1]) >= 0 && hexval(q[2]) >= 0) {
*o++ = (char)(hexval(q[1]) * 16 + hexval(q[2]));
q += 2;
} else {
*o++ = *q;
}
}
*o = 0;
#ifdef _WIN32
if (s[0] == '/' && isalpha((unsigned char)s[1]) && s[2] == ':')
memmove(s, s + 1, strlen(s));
#endif
return s[0] ? 0 : -1;
}
static int is_space(char c) {
return c == ' ' || c == '\t' || c == '\r' || c == '\n';
}
int cst_paste_paths(const char *text, size_t len,
char out[][CST_PASTE_PATH_MAX], int max) {
int n = 0;
size_t i = 0;
while (i < len) {
while (i < len && is_space(text[i]))
i++;
if (i >= len)
break;
if (n >= max)
return -1;
char *tok = out[n];
size_t k = 0;
char q = 0;
while (i < len) {
char c = text[i];
if (!c)
return -1;
if (q) {
if (c == q) {
q = 0;
i++;
continue;
}
} else {
if (c == '\'' || c == '"') {
q = c;
i++;
continue;
}
if (is_space(c))
break;
#ifndef _WIN32
if (c == '\\' && i + 1 < len)
c = text[++i];
#endif
}
if (k + 1 >= CST_PASTE_PATH_MAX)
return -1;
tok[k++] = c;
i++;
}
if (q || !k)
return -1;
tok[k] = 0;
if (strncmp(tok, "file://", 7) == 0 && from_file_uri(tok) != 0)
return -1;
n++;
}
return n > 0 ? n : -1;
}
+18 -10
View File
@@ -12,6 +12,7 @@
#ifdef _WIN32
#include <io.h>
#include <windows.h>
#else
#include <fcntl.h>
#include <termios.h>
@@ -51,7 +52,7 @@ int cst_creds_load(const char *provider, char *out, size_t out_sz) {
char path[OS_PATH_MAX];
if (credentials_path(path, sizeof(path)) != 0)
return -1;
FILE *fp = fopen(path, "r");
FILE *fp = fopen(path, "rb");
if (!fp)
return -1;
char want[64];
@@ -104,7 +105,9 @@ int cst_creds_save(const char *provider, const char *key) {
size_t wl = strlen(want);
#ifdef _WIN32
FILE *out = fopen(tmp, "w");
FILE *out = fopen(tmp, "wb");
if (out)
(void)os_chmod_private(tmp); /* label before the key is written */
#else
/* never exists world-readable, even briefly */
int fd = open(tmp, O_WRONLY | O_CREAT | O_TRUNC, 0600);
@@ -114,7 +117,7 @@ int cst_creds_save(const char *provider, const char *key) {
#endif
if (!out)
return -1;
FILE *in = fopen(path, "r");
FILE *in = fopen(path, "rb");
if (in) {
char line[1024];
while (fgets(line, sizeof(line), in)) {
@@ -128,10 +131,7 @@ int cst_creds_save(const char *provider, const char *key) {
remove(tmp);
return -1;
}
#ifdef _WIN32
remove(path);
#endif
if (rename(tmp, path) != 0) {
if (os_rename_replace(tmp, path) != 0) {
remove(tmp);
return -1;
}
@@ -167,7 +167,13 @@ int cst_creds_prompt_console(const char *provider, char *out, size_t out_sz) {
fprintf(stderr, "API key for %s (input hidden, saved to the config dir): ",
provider ? provider : "?");
fflush(stderr);
#ifndef _WIN32
#ifdef _WIN32
HANDLE hin = GetStdHandle(STD_INPUT_HANDLE);
DWORD old_mode = 0;
int tty = GetConsoleMode(hin, &old_mode) != 0;
if (tty)
SetConsoleMode(hin, old_mode & ~(DWORD)ENABLE_ECHO_INPUT);
#else
struct termios old, quiet;
int tty = isatty(STDIN_FILENO) && tcgetattr(STDIN_FILENO, &old) == 0;
if (tty) {
@@ -177,12 +183,14 @@ int cst_creds_prompt_console(const char *provider, char *out, size_t out_sz) {
}
#endif
char *got = fgets(out, (int)out_sz, stdin);
#ifndef _WIN32
if (tty) {
#ifdef _WIN32
SetConsoleMode(hin, old_mode);
#else
tcsetattr(STDIN_FILENO, TCSAFLUSH, &old);
#endif
fputc('\n', stderr);
}
#endif
if (!got) {
out[0] = 0;
return -1;
+770
View File
@@ -0,0 +1,770 @@
#ifndef _DEFAULT_SOURCE
#define _DEFAULT_SOURCE
#endif
#include "cst_http.h"
#include "cst_util.h"
#include "os_compat.h"
#include <errno.h>
#include <pthread.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define HTTP_HDR_MAX 32768 /* one header block */
#define HTTP_LINE_MAX 8192 /* one header line */
#define HTTP_DEFAULT_BODY (1024 * 1024)
#define HTTP_DEFAULT_CONN 32
#define HTTP_IDLE_TIMEOUT 60000
#define HTTP_READ_TIMEOUT 15000
struct cst_http_server {
os_socket_t listener;
int port;
pthread_t accept_th;
int accept_started;
pthread_mutex_t mu;
pthread_cond_t cv;
int stopping;
int nconn;
cst_http_conn_t *conns_first;
cst_http_opts_t o;
cst_http_route_t *routes;
int nroutes;
};
struct cst_http_conn {
cst_http_server_t *srv;
os_socket_t sock;
cst_http_req_t req;
char rbuf[8192];
size_t rlen, rpos;
char *body; /* owned while the request is served */
char hdrs[HTTP_HDR_MAX]; /* header values are copied here */
size_t hdrs_used;
int sent;
int streaming;
int failed;
int nreq; /* requests served on this connection */
int expect_continue; /* the client waits for "100 Continue" */
int keep_alive_req; /* what the client asked for */
cst_http_conn_t *prev, *next;
};
static const char *status_text(int code) {
switch (code) {
case 200: return "OK";
case 201: return "Created";
case 202: return "Accepted";
case 204: return "No Content";
case 302: return "Found";
case 400: return "Bad Request";
case 401: return "Unauthorized";
case 403: return "Forbidden";
case 404: return "Not Found";
case 405: return "Method Not Allowed";
case 409: return "Conflict";
case 411: return "Length Required";
case 413: return "Payload Too Large";
case 415: return "Unsupported Media Type";
case 429: return "Too Many Requests";
case 500: return "Internal Server Error";
case 503: return "Service Unavailable";
default: return "Error";
}
}
/* --- reading -------------------------------------------------------------- */
/* deadline: os_now_ms() value; the whole read must be done by then, so a
* client trickling one byte at a time cannot hold the thread forever. */
static int fill(cst_http_conn_t *c, double deadline) {
if (c->rpos < c->rlen)
return 1;
double left = deadline - os_now_ms();
if (left <= 0)
return -2;
long r = os_net_recv(c->sock, c->rbuf, sizeof(c->rbuf), (int)left + 1);
if (r <= 0)
return (int)r; /* 0 eof, -1 error, -2 timeout */
c->rpos = 0;
c->rlen = (size_t)r;
return 1;
}
#define RD_TOO_LONG (-3)
static int rd_line(cst_http_conn_t *c, char *out, size_t cap, double deadline) {
size_t n = 0;
for (;;) {
int rc = fill(c, deadline);
if (rc <= 0)
return rc;
while (c->rpos < c->rlen) {
char ch = c->rbuf[c->rpos++];
if (ch == '\n') {
if (n && out[n - 1] == '\r')
n--;
out[n] = 0;
return 1;
}
if (n + 2 >= cap)
return RD_TOO_LONG;
out[n++] = ch;
}
}
}
/* Exactly n bytes, using what the read buffer already holds. malloc'd. */
static char *rd_body(cst_http_conn_t *c, size_t n, int timeout_ms) {
char *buf = malloc(n + 1);
if (!buf)
return NULL;
size_t got = 0;
while (got < n) {
if (c->rpos < c->rlen) {
size_t take = c->rlen - c->rpos;
if (take > n - got)
take = n - got;
memcpy(buf + got, c->rbuf + c->rpos, take);
c->rpos += take;
got += take;
continue;
}
long r = os_net_recv(c->sock, c->rbuf, sizeof(c->rbuf), timeout_ms);
if (r <= 0) {
free(buf);
return NULL;
}
c->rpos = 0;
c->rlen = (size_t)r;
}
buf[n] = 0;
return buf;
}
/* -1 when the input encodes a NUL (out then holds what came before it). */
static int url_decode(const char *in, char *out, size_t cap, int plus_space) {
size_t o = 0;
for (const char *p = in; *p && o + 1 < cap; p++) {
if (*p == '%' && p[1] && p[2]) {
int hi = p[1] >= '0' && p[1] <= '9' ? p[1] - '0'
: p[1] >= 'a' && p[1] <= 'f' ? p[1] - 'a' + 10
: p[1] >= 'A' && p[1] <= 'F' ? p[1] - 'A' + 10
: -1;
int lo = p[2] >= '0' && p[2] <= '9' ? p[2] - '0'
: p[2] >= 'a' && p[2] <= 'f' ? p[2] - 'a' + 10
: p[2] >= 'A' && p[2] <= 'F' ? p[2] - 'A' + 10
: -1;
if (hi >= 0 && lo >= 0) {
char ch = (char)((hi << 4) | lo);
if (ch == 0) {
out[o] = 0;
return -1;
}
out[o++] = ch;
p += 2;
continue;
}
}
out[o++] = (*p == '+' && plus_space) ? ' ' : *p;
}
out[o] = 0;
return 0;
}
/* Comma-separated header tokens, compared without case ("Close"). */
static int has_token(const char *val, const char *tok) {
size_t n = strlen(tok);
for (const char *p = val; p && *p;) {
while (*p == ' ' || *p == '\t' || *p == ',')
p++;
size_t i = 0;
while (i < n && p[i] && (p[i] | 0x20) == (tok[i] | 0x20))
i++;
if (i == n && (p[i] == 0 || p[i] == ',' || p[i] == ' ' || p[i] == '\t'))
return 1;
p = strchr(p, ',');
}
return 0;
}
static int key_is(const char *line, const char *key) {
size_t n = strlen(key);
for (size_t i = 0; i < n; i++) {
char a = line[i], b = key[i];
if (a >= 'A' && a <= 'Z')
a = (char)(a - 'A' + 'a');
if (b >= 'A' && b <= 'Z')
b = (char)(b - 'A' + 'a');
if (a != b || !a)
return 0;
}
return line[n] == ':';
}
/* Copy a header value into the connection (they must not point into the
* line buffer, which is reused for the next header). */
static const char *store_str(cst_http_conn_t *c, const char *val) {
size_t n = strlen(val);
if (c->hdrs_used + n + 1 > sizeof(c->hdrs))
return NULL;
char *dst = c->hdrs + c->hdrs_used;
memcpy(dst, val, n + 1);
c->hdrs_used += n + 1;
return dst;
}
/* 1 = parsed, 0 = the client went away or waited too long, -1 = malformed. */
static int read_request(cst_http_conn_t *c, size_t *want_body) {
cst_http_server_t *s = c->srv;
char line[HTTP_LINE_MAX];
cst_http_req_t *r = &c->req;
int first = c->nreq == 0; /* a keep-alive pause gets the idle timeout */
memset(r, 0, sizeof(*r));
c->hdrs_used = 0;
c->expect_continue = 0;
*want_body = 0;
int rc = rd_line(c, line, sizeof(line),
os_now_ms() + (first ? s->o.read_timeout_ms
: s->o.idle_timeout_ms));
if (rc == RD_TOO_LONG)
return -1;
if (rc <= 0)
return 0;
char *sp1 = strchr(line, ' ');
if (!sp1)
return -1;
*sp1 = 0;
char *target = sp1 + 1;
char *version = strchr(target, ' ');
if (version) {
*version++ = 0;
} else {
version = (char *)"HTTP/1.0";
}
if (!line[0] || strlen(line) >= sizeof(r->method))
return -1;
snprintf(r->method, sizeof(r->method), "%s", line);
r->keep_alive = strncmp(version, "HTTP/1.", 7) == 0 &&
version[7] >= '1' && version[7] <= '9';
/* absolute-form ("http://host/path") is legal: keep the path */
if (target[0] != '/') {
char *scheme = strstr(target, "://");
target = scheme ? strchr(scheme + 3, '/') : NULL;
}
if (!target || target[0] != '/')
return -1;
{
char raw[sizeof(r->path)];
snprintf(raw, sizeof(raw), "%s", target);
char *q = strchr(raw, '?');
if (q) {
*q = 0;
snprintf(r->query, sizeof(r->query), "%s", q + 1);
}
if (url_decode(raw, r->path, sizeof(r->path), 0) != 0)
return -1;
size_t plen = strlen(r->path);
if (plen == 0 || strstr(r->path, ".."))
return -1;
}
size_t hdr_bytes = 0;
size_t content_length = 0;
int have_length = 0;
double deadline = os_now_ms() + s->o.read_timeout_ms;
for (;;) {
rc = rd_line(c, line, sizeof(line), deadline);
if (rc <= 0)
return -1;
if (!line[0])
break;
hdr_bytes += strlen(line) + 2;
if (hdr_bytes > HTTP_HDR_MAX)
return -1;
const char *colon = strchr(line, ':');
if (!colon)
return -1;
const char *val = colon + 1;
while (*val == ' ' || *val == '\t')
val++;
if (key_is(line, "content-length")) {
/* digits only, and one value: "5x" or two lengths would let the
* rest of the body run as a second request behind a proxy */
char *end = NULL;
errno = 0;
unsigned long long v = strtoull(val, &end, 10);
while (end && (*end == ' ' || *end == '\t'))
end++;
if (*val < '0' || *val > '9' || !end || *end || errno == ERANGE ||
v > (unsigned long long)SIZE_MAX ||
(have_length && content_length != (size_t)v))
return -1;
content_length = (size_t)v;
have_length = 1;
} else if (key_is(line, "transfer-encoding")) {
return -1; /* chunked request bodies are not supported */
} else if (key_is(line, "connection")) {
if (has_token(val, "close"))
r->keep_alive = 0;
else if (has_token(val, "keep-alive"))
r->keep_alive = 1;
} else if (key_is(line, "expect")) {
/* answered once auth and the size cap have passed */
c->expect_continue = has_token(val, "100-continue");
} else if (key_is(line, "authorization")) {
r->authorization = store_str(c, val);
} else if (key_is(line, "cookie")) {
r->cookie = store_str(c, val);
} else if (key_is(line, "origin")) {
r->origin = store_str(c, val);
} else if (key_is(line, "host")) {
r->host = store_str(c, val);
} else if (key_is(line, "content-type")) {
r->content_type = store_str(c, val);
} else if (key_is(line, "user-agent")) {
r->user_agent = store_str(c, val);
} else if (key_is(line, "last-event-id")) {
r->last_event_id = store_str(c, val);
}
}
if (have_length)
*want_body = content_length;
else if (strcmp(r->method, "POST") == 0)
/* No Content-Length means no body (RFC 9112 6.3): curl -X POST sends
* exactly that, and the zero-body endpoints (cancel, undo, quit) would
* otherwise be unreachable from a hand-written client. Bytes that
* follow are not ours to guess at, so the connection ends here. */
r->keep_alive = 0;
return 1;
}
/* --- writing -------------------------------------------------------------- */
static int send_head(cst_http_conn_t *c, int status, const char *ctype,
size_t len, int chunked) {
cst_buf_t out = {0};
cst_buf_printf(&out, "HTTP/1.1 %d %s\r\nContent-Type: %s\r\n",
status, status_text(status), ctype ? ctype : "text/plain");
cst_buf_puts(&out, "Cache-Control: no-store\r\n");
cst_buf_puts(&out, c->req.keep_alive ? "Connection: keep-alive\r\n"
: "Connection: close\r\n");
if (chunked) {
cst_buf_puts(&out, "Transfer-Encoding: chunked\r\n");
} else {
cst_buf_printf(&out, "Content-Length: %zu\r\n", len);
}
cst_buf_puts(&out, "\r\n");
int rc = out.buf ? os_net_send_all(c->sock, out.buf, out.len, 0) : -1;
cst_buf_free(&out);
return rc;
}
int cst_http_send(cst_http_conn_t *c, int status, const char *ctype,
const void *body, size_t n) {
if (!c || c->sent)
return -1;
c->sent = 1;
if (send_head(c, status, ctype, n, 0) != 0) {
c->failed = 1;
return -1;
}
if (n && os_net_send_all(c->sock, body, n, 0) != 0) {
c->failed = 1;
return -1;
}
return 0;
}
int cst_http_send_str(cst_http_conn_t *c, int status, const char *ctype,
const char *s) {
return cst_http_send(c, status, ctype, s ? s : "", s ? strlen(s) : 0);
}
int cst_http_send_json(cst_http_conn_t *c, int status, const char *json) {
return cst_http_send_str(c, status, "application/json; charset=utf-8",
json ? json : "null");
}
int cst_http_send_err(cst_http_conn_t *c, int status, const char *msg) {
cst_buf_t b = {0};
cst_buf_puts(&b, "{\"error\":\"");
for (const char *p = msg ? msg : "error"; *p; p++) {
if (*p == '"' || *p == '\\')
cst_buf_printf(&b, "\\%c", *p);
else if ((unsigned char)*p < 0x20)
cst_buf_printf(&b, "\\u%04x", (unsigned char)*p);
else
cst_buf_append(&b, p, 1);
}
cst_buf_puts(&b, "\"}");
int rc = cst_http_send_json(c, status, b.buf);
cst_buf_free(&b);
return rc;
}
int cst_http_sent(const cst_http_conn_t *c) { return c ? c->sent : 0; }
int cst_http_stream_begin(cst_http_conn_t *c, const char *ctype) {
if (!c || c->sent)
return -1;
c->sent = 1;
c->streaming = 1;
if (send_head(c, 200, ctype, 0, 1) != 0) {
c->failed = 1;
return -1;
}
return 0;
}
int cst_http_stream_write(cst_http_conn_t *c, const void *data, size_t n) {
if (!c || !c->streaming || c->failed)
return -1;
if (n == 0)
return 0;
cst_buf_t b = {0};
cst_buf_printf(&b, "%zx\r\n", n);
cst_buf_append(&b, data, n);
cst_buf_puts(&b, "\r\n");
int rc = b.buf ? os_net_send_all(c->sock, b.buf, b.len, 0) : -1;
cst_buf_free(&b);
if (rc != 0)
c->failed = 1;
return rc;
}
int cst_http_stream_end(cst_http_conn_t *c) {
if (!c || !c->streaming)
return -1;
c->streaming = 0;
if (c->failed)
return -1;
const char *end = "0\r\n\r\n";
if (os_net_send_all(c->sock, end, strlen(end), 0) != 0) {
c->failed = 1;
return -1;
}
return 0;
}
const cst_http_req_t *cst_http_request(cst_http_conn_t *c) {
return c ? &c->req : NULL;
}
int cst_http_query(const cst_http_req_t *req, const char *key, char *out,
size_t out_sz) {
if (!req || !key || !out || out_sz == 0)
return -1;
size_t klen = strlen(key);
const char *p = req->query;
while (p && *p) {
const char *amp = strchr(p, '&');
size_t seg = amp ? (size_t)(amp - p) : strlen(p);
const char *eq = memchr(p, '=', seg);
size_t name_len = eq ? (size_t)(eq - p) : seg;
if (name_len == klen && strncmp(p, key, klen) == 0) {
char raw[1024];
size_t vlen = eq ? seg - name_len - 1 : 0;
if (vlen >= sizeof(raw))
vlen = sizeof(raw) - 1;
if (eq)
memcpy(raw, eq + 1, vlen);
raw[vlen] = 0;
url_decode(raw, out, out_sz, 1);
return 0;
}
p = amp ? amp + 1 : NULL;
}
return -1;
}
/* --- connections ---------------------------------------------------------- */
static const cst_http_route_t *find_route(const cst_http_server_t *s,
const cst_http_req_t *req) {
const cst_http_route_t *fallback = NULL;
for (int i = 0; i < s->nroutes; i++) {
const cst_http_route_t *r = &s->routes[i];
if (strcmp(r->method, "*") != 0 && strcmp(r->method, req->method) != 0)
continue;
if (strcmp(r->path, "*") == 0) {
fallback = r;
continue;
}
if (strcmp(r->path, req->path) == 0)
return r;
}
return fallback;
}
/* Route, auth and the size cap are settled before the body is read: an
* unauthenticated client must not make us buffer 32 MiB. */
static void serve(cst_http_server_t *s, cst_http_conn_t *c, size_t want) {
const cst_http_route_t *route = find_route(s, &c->req);
size_t cap = route && route->max_body ? route->max_body : s->o.max_body;
if (want > 0)
c->req.keep_alive = 0; /* until the body is read, the stream is ours */
if (!route) {
cst_http_send_err(c, 404, "not found");
return;
}
if (route->auth && s->o.auth && s->o.auth(&c->req, s->o.auth_ud) != 0) {
cst_http_send_err(c, 401, "unauthorized");
return;
}
if (want > cap) {
cst_http_send_err(c, 413, "request body too large");
return;
}
if (want > 0) {
if (c->expect_continue) {
const char *cont = "HTTP/1.1 100 Continue\r\n\r\n";
os_net_send_all(c->sock, cont, strlen(cont), 5000);
}
char *body = rd_body(c, want, s->o.read_timeout_ms);
if (!body) {
c->failed = 1;
return;
}
c->body = body;
c->req.body = body;
c->req.body_len = want;
c->req.keep_alive = c->keep_alive_req;
}
int rc = route->fn(c, route->ud);
if (!c->sent)
cst_http_send_err(c, rc == 0 ? 500 : 404,
rc == 0 ? "handler sent no response" : "not found");
}
static void *conn_main(void *arg) {
cst_http_conn_t *c = arg;
cst_http_server_t *s = c->srv;
for (;;) {
size_t want = 0;
int rc = read_request(c, &want);
if (rc < 0) {
/* malformed (bad request line, traversal, unsupported transfer
* encoding): say so instead of closing silently */
c->sent = 0;
c->req.keep_alive = 0;
cst_http_send_err(c, 400, "malformed request");
break;
}
if (rc == 0)
break;
c->nreq++;
c->sent = 0;
c->streaming = 0;
c->keep_alive_req = c->req.keep_alive;
serve(s, c, want);
if (s->o.on_log) {
char line[512];
snprintf(line, sizeof(line), "%s %.480s -> %d", c->req.method, c->req.path,
c->failed ? 499 : (c->sent ? 200 : 500));
s->o.on_log(line, s->o.log_ud);
}
free(c->body);
c->body = NULL;
c->req.body = NULL;
c->req.body_len = 0;
c->req.authorization = c->req.cookie = c->req.origin = NULL;
c->req.host = c->req.content_type = c->req.user_agent = NULL;
c->req.last_event_id = NULL;
if (!c->req.keep_alive || s->stopping || c->failed)
break;
}
/* unlink first: stop() shuts down every listed socket, and a closed fd
* number may already belong to someone else */
pthread_mutex_lock(&s->mu);
if (c->prev)
c->prev->next = c->next;
else
s->conns_first = c->next;
if (c->next)
c->next->prev = c->prev;
pthread_mutex_unlock(&s->mu);
os_net_shutdown(c->sock);
os_net_close(c->sock);
free(c);
pthread_mutex_lock(&s->mu);
s->nconn--;
pthread_cond_broadcast(&s->cv);
pthread_mutex_unlock(&s->mu);
return NULL;
}
static void *accept_main(void *arg) {
cst_http_server_t *s = arg;
while (!s->stopping) {
int timed_out = 0;
os_socket_t sock = os_net_accept(s->listener, 500, &timed_out);
if (sock == OS_SOCKET_BAD) {
if (!timed_out) { /* EMFILE and friends: the listener stays readable */
struct timespec ts;
clock_gettime(CLOCK_REALTIME, &ts);
ts.tv_nsec += 100000000;
if (ts.tv_nsec >= 1000000000) {
ts.tv_sec++;
ts.tv_nsec -= 1000000000;
}
pthread_mutex_lock(&s->mu);
if (!s->stopping)
pthread_cond_timedwait(&s->cv, &s->mu, &ts);
pthread_mutex_unlock(&s->mu);
}
continue;
}
if (s->stopping) {
os_net_close(sock);
break;
}
pthread_mutex_lock(&s->mu);
int over = s->nconn >= s->o.max_conn;
if (!over)
s->nconn++;
pthread_mutex_unlock(&s->mu);
if (over) {
const char *busy = "HTTP/1.1 503 Service Unavailable\r\n"
"Content-Length: 0\r\nConnection: close\r\n\r\n";
os_net_send_all(sock, busy, strlen(busy), 2000);
os_net_close(sock);
continue;
}
cst_http_conn_t *c = calloc(1, sizeof(*c));
if (!c) {
pthread_mutex_lock(&s->mu);
s->nconn--;
pthread_cond_broadcast(&s->cv);
pthread_mutex_unlock(&s->mu);
os_net_close(sock);
continue;
}
c->srv = s;
c->sock = sock;
pthread_mutex_lock(&s->mu);
c->next = s->conns_first;
if (c->next)
c->next->prev = c;
s->conns_first = c;
pthread_mutex_unlock(&s->mu);
pthread_t th;
if (pthread_create(&th, NULL, conn_main, c) != 0) {
pthread_mutex_lock(&s->mu);
s->conns_first = c->next;
if (c->next)
c->next->prev = NULL;
s->nconn--;
pthread_cond_broadcast(&s->cv);
pthread_mutex_unlock(&s->mu);
os_net_close(sock);
free(c);
continue;
}
pthread_detach(th);
}
return NULL;
}
cst_http_server_t *cst_http_start(const cst_http_opts_t *o, char *err,
size_t err_sz) {
if (!o)
return NULL;
if (os_net_init() != 0) {
if (err)
snprintf(err, err_sz, "network init failed");
return NULL;
}
cst_http_server_t *s = calloc(1, sizeof(*s));
if (!s)
return NULL;
s->o = *o;
s->o.max_body = o->max_body ? o->max_body : HTTP_DEFAULT_BODY;
s->o.max_conn = o->max_conn ? o->max_conn : HTTP_DEFAULT_CONN;
if (!s->o.read_timeout_ms)
s->o.read_timeout_ms = HTTP_READ_TIMEOUT;
if (!s->o.idle_timeout_ms)
s->o.idle_timeout_ms = HTTP_IDLE_TIMEOUT;
if (o->nroutes > 0) {
s->routes = calloc((size_t)o->nroutes, sizeof(*s->routes));
if (!s->routes) {
free(s);
return NULL;
}
memcpy(s->routes, o->routes, (size_t)o->nroutes * sizeof(*s->routes));
s->nroutes = o->nroutes;
}
pthread_mutex_init(&s->mu, NULL);
pthread_cond_init(&s->cv, NULL);
s->listener = os_net_listen(o->host, o->port, err, err_sz);
if (s->listener == OS_SOCKET_BAD) {
free(s->routes);
free(s);
return NULL;
}
s->port = os_net_port(s->listener);
if (pthread_create(&s->accept_th, NULL, accept_main, s) != 0) {
os_net_close(s->listener);
free(s->routes);
free(s);
if (err)
snprintf(err, err_sz, "cannot start the accept thread");
return NULL;
}
s->accept_started = 1;
return s;
}
int cst_http_port(const cst_http_server_t *s) { return s ? s->port : 0; }
int cst_http_stopping(const cst_http_server_t *s) {
return s ? s->stopping : 1;
}
int cst_http_stop(cst_http_server_t *s) {
if (!s)
return 0;
pthread_mutex_lock(&s->mu);
s->stopping = 1;
for (cst_http_conn_t *c = s->conns_first; c; c = c->next)
os_net_shutdown(c->sock);
pthread_mutex_unlock(&s->mu);
os_net_shutdown(s->listener);
if (s->accept_started) {
pthread_join(s->accept_th, NULL);
s->accept_started = 0;
}
/* Handler threads own their connection until they return; the shutdown
* above is what wakes them. Bounded wait so free() stays safe. */
pthread_mutex_lock(&s->mu);
for (int i = 0; i < 300 && s->nconn > 0; i++) {
struct timespec ts;
clock_gettime(CLOCK_REALTIME, &ts);
ts.tv_nsec += 100000000; /* 100 ms */
if (ts.tv_nsec >= 1000000000) {
ts.tv_sec++;
ts.tv_nsec -= 1000000000;
}
pthread_cond_timedwait(&s->cv, &s->mu, &ts);
}
int left = s->nconn;
pthread_mutex_unlock(&s->mu);
os_net_close(s->listener);
return left;
}
void cst_http_free(cst_http_server_t *s) {
if (!s)
return;
pthread_mutex_lock(&s->mu);
int left = s->nconn;
pthread_mutex_unlock(&s->mu);
if (left > 0)
return; /* a handler still runs on it (see stop()): leak, not crash */
pthread_mutex_destroy(&s->mu);
pthread_cond_destroy(&s->cv);
free(s->routes);
free(s);
}
+5 -1
View File
@@ -1,6 +1,7 @@
#include "cst_keymap.h"
#include "cJSON.h"
#include "cst_util.h"
#include "os_compat.h"
#include <ctype.h>
@@ -47,6 +48,8 @@ static const cst_key_action_info_t INFO[CST_KA_COUNT] = {
[CST_KA_COPY] = {"copy", "Session", "copy the last reply", "/copy"},
[CST_KA_EXPORT] = {"export", "Session", "export the transcript",
"/export"},
[CST_KA_PASTE_IMAGE] = {"paste_image", "Session",
"attach the clipboard image", "/attach paste"},
[CST_KA_MODEL] = {"model", "Model", "pick the model", "/model"},
[CST_KA_PROVIDER] = {"provider", "Model", "switch provider", "/provider"},
[CST_KA_EFFORT] = {"effort", "Model", "reasoning effort", "/effort"},
@@ -88,6 +91,7 @@ static const struct {
{CST_KA_COMPACT, "<leader> c"},
{CST_KA_COPY, "<leader> y"},
{CST_KA_EXPORT, "<leader> x"},
{CST_KA_PASTE_IMAGE, "ctrl+v"},
{CST_KA_MODEL, "<leader> m"},
{CST_KA_PROVIDER, "<leader> p"},
{CST_KA_EFFORT, "<leader> e"},
@@ -491,7 +495,7 @@ int cst_keymap_load(cst_keymap_t *km, char *err, size_t err_sz) {
size_t n = fread(buf, 1, sizeof(buf) - 1, fp);
fclose(fp);
buf[n] = 0;
return cst_keymap_apply_json(km, buf, err, err_sz);
return cst_keymap_apply_json(km, buf + cst_bom_len(buf, n), err, err_sz);
}
int cst_keymap_lookup(const cst_keymap_t *km, const int *seq, int n,
+591 -26
View File
@@ -22,6 +22,80 @@
#define ERRBODY_MAX 4096
#define LLM_MAX_RETRIES 3
static int ca_file_ok(const char *path) {
if (!path || !path[0])
return 0;
FILE *f = fopen(path, "rb");
if (!f)
return 0;
fclose(f);
return 1;
}
/* Point libcurl at a CA bundle.
*
* libcurl keeps the bundle path it was built with. On Windows an OpenSSL
* build carries MSYS2's "/mingw64/etc/ssl/certs/ca-bundle.crt"; the static
* Linux release is built on one distro and runs on all of them, whose bundles
* live in different places. A path that does not exist fails every HTTPS
* request with CURLE_SSL_CACERT_BADFILE (77).
*
* Order: $CST_CA_BUNDLE; on Windows ca-bundle.crt beside the executable (if
* absent nothing is set, so Schannel uses the certificate store); elsewhere
* $SSL_CERT_FILE, libcurl's own default if that file exists, then the usual
* distro locations. Decided once per process. */
void cst_curl_apply_ca(void *curl) {
static char ca[OS_PATH_MAX];
static atomic_int state; /* 0 unchecked, 1 checking, 2 done */
int expect = 0;
if (atomic_compare_exchange_strong(&state, &expect, 1)) {
const char *env = getenv("CST_CA_BUNDLE");
if (ca_file_ok(env)) {
snprintf(ca, sizeof(ca), "%s", env);
} else {
#if defined(_WIN32)
char dir[OS_PATH_MAX], p[OS_PATH_MAX];
if (os_exe_dir(dir, sizeof(dir)) == 0 &&
os_path_join(p, sizeof(p), dir, "ca-bundle.crt") == 0 &&
ca_file_ok(p))
snprintf(ca, sizeof(ca), "%s", p);
#else
static const char *const known[] = {
"/etc/ssl/certs/ca-certificates.crt", /* Debian, Ubuntu, Arch, Alpine */
"/etc/pki/tls/certs/ca-bundle.crt", /* Fedora, RHEL */
"/etc/ssl/cert.pem", /* macOS, BSD, Alpine */
"/etc/ssl/ca-bundle.pem", /* openSUSE */
"/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem",
};
const char *pick = NULL;
const char *sslenv = getenv("SSL_CERT_FILE");
const curl_version_info_data *vi = curl_version_info(CURLVERSION_NOW);
if (ca_file_ok(sslenv))
pick = sslenv;
else if (vi && ca_file_ok(vi->cainfo))
pick = NULL; /* libcurl's built-in default already works */
else
for (size_t i = 0; i < sizeof(known) / sizeof(known[0]); i++)
if (ca_file_ok(known[i])) {
pick = known[i];
break;
}
if (pick)
snprintf(ca, sizeof(ca), "%s", pick);
#endif
}
atomic_store(&state, 2);
} else {
while (atomic_load(&state) != 2)
;
}
if (ca[0])
curl_easy_setopt((CURL *)curl, CURLOPT_CAINFO, ca);
}
#include "cst_attach.h"
#include "cst_models.h"
/* --- wire body ----------------------------------------------------------- */
static size_t wire_tool_cap(const cst_llm_request_t *req) {
@@ -77,6 +151,272 @@ static cJSON *clean_string(const char *s, size_t cap) {
return j;
}
/* --- multimodal content --------------------------------------------------
* OpenAI-compatible servers take a parts array where a plain string is also
* accepted, so text-only messages keep the string form and only those with
* images switch to parts. Images live in the session store and are read (and
* base64'd) here, once per request. */
/* The message's "content" as a parts array, converting a string in place. */
static cJSON *content_parts(cJSON *msg) {
cJSON *c = cJSON_GetObjectItemCaseSensitive(msg, "content");
if (cJSON_IsArray(c))
return c;
cJSON *arr = cJSON_CreateArray();
if (cJSON_IsString(c)) {
cJSON *t = cJSON_CreateObject();
cJSON_AddStringToObject(t, "type", "text");
cJSON_AddStringToObject(t, "text", c->valuestring);
cJSON_AddItemToArray(arr, t);
}
cJSON_ReplaceItemInObjectCaseSensitive(msg, "content", arr);
return arr;
}
/* Append text, joining it onto a trailing text part when there is one.
* Takes ownership of txt. */
static void parts_add_text(cJSON *parts, cJSON *txt) {
if (!txt)
txt = cJSON_CreateString("");
cJSON *last = cJSON_GetArrayItem(parts, cJSON_GetArraySize(parts) - 1);
const cJSON *lt = last ? cJSON_GetObjectItemCaseSensitive(last, "type") : NULL;
if (cJSON_IsString(lt) && strcmp(lt->valuestring, "text") == 0) {
const cJSON *old = cJSON_GetObjectItemCaseSensitive(last, "text");
cst_buf_t b = {0};
if (cJSON_IsString(old))
cst_buf_puts(&b, old->valuestring);
if (b.len)
cst_buf_puts(&b, "\n\n");
cst_buf_puts(&b, txt->valuestring ? txt->valuestring : "");
cJSON_ReplaceItemInObjectCaseSensitive(
last, "text", cJSON_CreateString(b.buf ? b.buf : ""));
cst_buf_free(&b);
cJSON_Delete(txt);
return;
}
if (!txt->valuestring || !txt->valuestring[0]) {
cJSON_Delete(txt);
return;
}
cJSON *t = cJSON_CreateObject();
cJSON_AddStringToObject(t, "type", "text");
cJSON_AddItemToObject(t, "text", txt);
cJSON_AddItemToArray(parts, t);
}
/* One stored image as an image_url part, or NULL with why filled in. */
static cJSON *image_part(const cst_llm_request_t *req,
const cst_attachment_t *a, size_t *budget, char *why,
size_t why_sz) {
why[0] = 0;
if (!req->session_path || !req->session_path[0]) {
snprintf(why, why_sz, "there is no session store to read it from");
return NULL;
}
char err[256];
size_t n = 0;
char *data = cst_attach_read(req->session_path, a, &n, err, sizeof(err));
if (!data) {
snprintf(why, why_sz, "%s", err);
return NULL;
}
if (n > *budget) {
snprintf(why, why_sz, "%d KB left in the %d MB image budget",
(int)(*budget / 1024), (int)(CST_ATTACH_MAX_TOTAL >> 20));
free(data);
return NULL;
}
*budget -= n;
char mime[32];
cst_attach_sniff(data, n, mime, sizeof(mime)); /* read verified it */
char *b64 = cst_base64_encode(data, n);
free(data);
if (!b64) {
snprintf(why, why_sz, "out of memory");
return NULL;
}
cst_buf_t b = {0};
cst_buf_puts(&b, "data:");
cst_buf_puts(&b, mime);
cst_buf_puts(&b, ";base64,");
cst_buf_puts(&b, b64);
free(b64);
if (!b.buf) {
snprintf(why, why_sz, "out of memory");
return NULL;
}
cJSON *p = cJSON_CreateObject();
cJSON_AddStringToObject(p, "type", "image_url");
cJSON *iu = cJSON_AddObjectToObject(p, "image_url");
cJSON_AddStringToObject(iu, "url", b.buf);
if (a->detail[0] && strcmp(a->detail, "auto") != 0)
cJSON_AddStringToObject(iu, "detail", a->detail);
cst_buf_free(&b);
return p;
}
/* One stored document as a text part: its text, framed so the model knows
* where it came from. Documents are plain text on the wire, so a text-only
* model and CST_NO_IMAGES both still get them. */
static cJSON *document_part(const cst_llm_request_t *req,
const cst_attachment_t *a, size_t *budget,
char *why, size_t why_sz) {
why[0] = 0;
if (!req->session_path || !req->session_path[0]) {
snprintf(why, why_sz, "there is no session store to read it from");
return NULL;
}
char err[256];
size_t n = 0;
char *text = cst_attach_read(req->session_path, a, &n, err, sizeof(err));
if (!text) {
snprintf(why, why_sz, "%s", err);
return NULL;
}
int cut = 0;
if (n > *budget) {
n = *budget;
cut = 1;
}
*budget -= n;
char desc[128];
cst_attach_describe(a, desc, sizeof(desc));
cst_buf_t b = {0};
cst_buf_printf(&b, "[document \"%.120s\" (%s)%s]\n", a->name, desc,
cut ? " -- only the start is included" : "");
cst_buf_append(&b, text, n);
cst_buf_puts(&b, cut ? "\n[-- document truncated --]"
: "\n[-- end of document --]");
free(text);
cJSON *p = cJSON_CreateObject();
cJSON_AddStringToObject(p, "type", "text");
cJSON_AddStringToObject(p, "text", b.buf ? b.buf : "");
cst_buf_free(&b);
return p;
}
/* Append every attachment of m: images as image parts (or a note saying why
* not), documents inline as text. */
static void parts_add_attachments(const cst_llm_request_t *req, cJSON *parts,
const cst_message_t *m, size_t *budget,
size_t *doc_budget, const char *img_off) {
int n = m->nattach;
if (n > CST_ATTACH_MAX_COUNT) {
char note[160];
snprintf(note, sizeof(note),
"[only the first %d of %d attachments are sent]",
CST_ATTACH_MAX_COUNT, n);
parts_add_text(parts, cJSON_CreateString(note));
n = CST_ATTACH_MAX_COUNT;
}
for (int i = 0; i < n; i++) {
const cst_attachment_t *a = &m->attach[i];
int is_img = cst_attach_is_image(a);
char why[256];
cJSON *p = NULL;
if (is_img && img_off)
snprintf(why, sizeof(why), "%s", img_off);
else if (is_img)
p = image_part(req, a, budget, why, sizeof(why));
else
p = document_part(req, a, doc_budget, why, sizeof(why));
if (p) {
cJSON_AddItemToArray(parts, p);
continue;
}
char note[512];
snprintf(note, sizeof(note), "[%s \"%.120s\" not sent: %.200s]",
is_img ? "image" : "document", a->name, why);
parts_add_text(parts, cJSON_CreateString(note));
}
}
/* Images a tool result produced are deferred: an assistant tool_call must be
* answered by tool messages only, so they ride in one extra user message
* after the run of tool messages ends. *pending collects it. */
static void defer_attachments(const cst_llm_request_t *req,
const cst_message_t *m, cJSON **pending,
size_t *budget, size_t *doc_budget,
const char *img_off) {
cJSON *p = *pending;
if (!p) {
p = cJSON_CreateObject();
cJSON_AddStringToObject(p, "role", "user");
cJSON_AddArrayToObject(p, "content");
*pending = p;
}
parts_add_attachments(req, cJSON_GetObjectItemCaseSensitive(p, "content"), m,
budget, doc_budget, img_off);
}
/* m's text plus one line per image it would have carried (malloc'd); a
* document is not affected (it is text and is sent as such). */
static char *image_notes(const cst_message_t *m, const char *why) {
cst_buf_t b = {0};
cst_buf_puts(&b, m->content ? m->content : "");
for (int i = 0; i < m->nattach; i++) {
if (!cst_attach_is_image(&m->attach[i]))
continue;
char d[128];
cst_attach_describe(&m->attach[i], d, sizeof(d));
cst_buf_printf(&b, "%s[image \"%.120s\" (%s) not sent: %s]",
b.len ? "\n" : "", m->attach[i].name, d, why);
}
return cst_buf_steal(&b);
}
/* --- cache breakpoints (Anthropic-style) ----------------------------------
* At most 4 per request. The system prompt one also covers tools[]; the
* newest message one moves forward every step; the newest real user message
* keeps a hit when a long tool run pushes the previous one out of the
* provider's lookback. */
static void mark_message(cJSON *msg, const char *ttl) {
const cJSON *c = cJSON_GetObjectItemCaseSensitive(msg, "content");
if (cJSON_IsString(c) ? !c->valuestring[0] : !cJSON_IsArray(c))
return;
cJSON *parts = content_parts(msg);
cJSON *last = cJSON_GetArrayItem(parts, cJSON_GetArraySize(parts) - 1);
if (!cJSON_IsObject(last) ||
cJSON_GetObjectItemCaseSensitive(last, "cache_control"))
return;
cJSON *cc = cJSON_AddObjectToObject(last, "cache_control");
cJSON_AddStringToObject(cc, "type", "ephemeral");
if (ttl && ttl[0])
cJSON_AddStringToObject(cc, "ttl", ttl);
}
static int msg_role_is(const cJSON *msg, const char *role) {
const cJSON *r = cJSON_GetObjectItemCaseSensitive(msg, "role");
return cJSON_IsString(r) && strcmp(r->valuestring, role) == 0;
}
static int msg_has_content(const cJSON *msg) {
const cJSON *c = cJSON_GetObjectItemCaseSensitive(msg, "content");
return cJSON_IsString(c) ? c->valuestring[0] != 0 : cJSON_IsArray(c);
}
static void cache_marks(cJSON *msgs, const char *ttl) {
int n = cJSON_GetArraySize(msgs);
cJSON *first = cJSON_GetArrayItem(msgs, 0);
if (first && msg_role_is(first, "system"))
mark_message(first, ttl);
int last = -1;
for (int i = n - 1; i > 0 && last < 0; i--)
if (msg_has_content(cJSON_GetArrayItem(msgs, i)))
last = i;
if (last < 0)
return;
mark_message(cJSON_GetArrayItem(msgs, last), ttl);
for (int i = last - 1; i > 0; i--) {
cJSON *m = cJSON_GetArrayItem(msgs, i);
if (msg_role_is(m, "user") && msg_has_content(m)) {
mark_message(m, ttl);
break;
}
}
}
static const char *role_api(cst_role_t r) {
switch (r) {
case CST_ROLE_SYSTEM:
@@ -107,34 +447,92 @@ char *cst_llm_build_body(const cst_llm_request_t *req) {
cJSON *msgs = cJSON_AddArrayToObject(root, "messages");
size_t tcap = wire_tool_cap(req);
cJSON *prev_user = NULL; /* merge consecutive user messages */
cJSON *pending = NULL; /* images from tool results, waiting for a break */
size_t img_budget = CST_ATTACH_MAX_TOTAL;
size_t doc_budget = CST_ATTACH_MAX_TOTAL;
const char *noimg = getenv("CST_NO_IMAGES");
const char *img_off =
noimg && noimg[0] && noimg[0] != '0'
? "image sending is disabled (CST_NO_IMAGES)"
: req->images_off ? "this model does not take images"
: NULL;
for (int i = 0; i < req->nmsgs; i++) {
const cst_message_t *m = &req->msgs[i];
cst_message_t mm = req->msgs[i];
const cst_message_t *m = &mm;
cst_role_t role = m->role;
if (role == CST_ROLE_TOOL && !m->tool_call_id[0])
role = CST_ROLE_USER;
int n_att = m->attach ? m->nattach : 0;
int has_img = 0, has_doc = 0;
for (int k = 0; k < n_att; k++) {
if (cst_attach_is_image(&m->attach[k]))
has_img = 1;
else
has_doc = 1;
}
char *noted = NULL;
if (has_img && img_off) {
has_img = 0;
if (!has_doc) {
/* plain text: text-only APIs may refuse content arrays */
noted = image_notes(m, img_off);
if (noted)
mm.content = noted;
}
}
/* a document needs a content array even when no image can go out */
int att_parts = has_img || has_doc;
/* Every tool_call must be answered by tool messages, so tool-produced
* images only leave the queue when the run of tool results is over. */
if (role != CST_ROLE_TOOL && pending) {
cJSON_AddItemToArray(msgs, pending);
prev_user = pending;
pending = NULL;
}
if (role == CST_ROLE_USER) {
/* Text-protocol result frames are capped by the agent before framing
* (their byte counts must stay exact), so never cap user text here. */
cJSON *c = clean_string(m->content, 0);
if (prev_user) {
cJSON *pc = cJSON_GetObjectItemCaseSensitive(prev_user, "content");
cst_buf_t b = {0};
cst_buf_puts(&b, pc->valuestring);
cst_buf_puts(&b, "\n\n");
cst_buf_puts(&b, c->valuestring);
cJSON_ReplaceItemInObjectCaseSensitive(prev_user, "content",
cJSON_CreateString(b.buf));
cst_buf_free(&b);
cJSON_Delete(c);
cJSON *pc = prev_user
? cJSON_GetObjectItemCaseSensitive(prev_user, "content")
: NULL;
if (!att_parts && !cJSON_IsArray(pc)) {
cJSON *c = clean_string(m->content, 0);
if (prev_user) {
cst_buf_t b = {0};
cst_buf_puts(&b, pc->valuestring);
cst_buf_puts(&b, "\n\n");
cst_buf_puts(&b, c->valuestring);
cJSON_ReplaceItemInObjectCaseSensitive(prev_user, "content",
cJSON_CreateString(b.buf));
cst_buf_free(&b);
cJSON_Delete(c);
free(noted);
continue;
}
cJSON *o = cJSON_CreateObject();
cJSON_AddStringToObject(o, "role", "user");
cJSON_AddItemToObject(o, "content", c);
cJSON_AddItemToArray(msgs, o);
prev_user = o;
free(noted);
continue;
}
cJSON *o = cJSON_CreateObject();
cJSON_AddStringToObject(o, "role", "user");
cJSON_AddItemToObject(o, "content", c);
cJSON_AddItemToArray(msgs, o);
prev_user = o;
cJSON *o = prev_user;
if (!o) {
o = cJSON_CreateObject();
cJSON_AddStringToObject(o, "role", "user");
cJSON_AddArrayToObject(o, "content");
cJSON_AddItemToArray(msgs, o);
prev_user = o;
}
cJSON *parts = content_parts(o);
parts_add_text(parts, clean_string(m->content, 0));
if (att_parts)
parts_add_attachments(req, parts, m, &img_budget, &doc_budget, img_off);
free(noted);
continue;
}
prev_user = NULL;
@@ -159,17 +557,39 @@ char *cst_llm_build_body(const cst_llm_request_t *req) {
} else {
cJSON_AddItemToObject(o, "content", clean_string(m->content, 0));
}
if (role == CST_ROLE_ASSISTANT) {
if (req->send_reasoning && m->reasoning && m->reasoning[0])
cJSON_AddItemToObject(o, "reasoning_content",
clean_string(m->reasoning, 0));
if (m->prefix)
cJSON_AddTrueToObject(o, "prefix");
}
cJSON_AddItemToArray(msgs, o);
if (role == CST_ROLE_TOOL && n_att)
defer_attachments(req, m, &pending, &img_budget, &doc_budget, img_off);
free(noted);
}
if (pending)
cJSON_AddItemToArray(msgs, pending);
if (req->enable_tools) {
cJSON *tools = cst_tools_schema_cfg(req->tool_cfg);
cJSON *tools = req->tools_json ? cJSON_Parse(req->tools_json)
: cst_tools_schema_cfg(req->tool_cfg);
if (tools) {
cJSON_AddItemToObject(root, "tools", tools);
cJSON_AddStringToObject(root, "tool_choice",
req->tool_choice_none ? "none" : "auto");
}
}
if (req->cache_style == CST_CACHE_OPENAI && req->cache_key &&
req->cache_key[0]) {
cJSON_AddStringToObject(root, "prompt_cache_key", req->cache_key);
const char *ret = getenv("CST_CACHE_RETENTION");
if (ret && ret[0])
cJSON_AddStringToObject(root, "prompt_cache_retention", ret);
} else if (req->cache_style == CST_CACHE_ANTHROPIC) {
cache_marks(msgs, req->cache_ttl);
}
char *out = cJSON_PrintUnformatted(root);
cJSON_Delete(root);
return out;
@@ -332,11 +752,22 @@ static void handle_payload(sse_t *s, const char *payload) {
const cJSON *pt = cJSON_GetObjectItemCaseSensitive(usage, "prompt_tokens");
const cJSON *ct =
cJSON_GetObjectItemCaseSensitive(usage, "completion_tokens");
/* DeepSeek: prompt_cache_hit/miss_tokens. OpenAI and routers:
* prompt_tokens_details.cached_tokens (+ cache_write_tokens).
* Anthropic fields passed through: cache_read/creation_input_tokens. */
const cJSON *hit =
cJSON_GetObjectItemCaseSensitive(usage, "prompt_cache_hit_tokens");
const cJSON *miss =
cJSON_GetObjectItemCaseSensitive(usage, "prompt_cache_miss_tokens");
const cJSON *det =
cJSON_GetObjectItemCaseSensitive(usage, "prompt_tokens_details");
const cJSON *cached = cJSON_GetObjectItemCaseSensitive(det, "cached_tokens");
const cJSON *dwrite =
cJSON_GetObjectItemCaseSensitive(det, "cache_write_tokens");
const cJSON *aread =
cJSON_GetObjectItemCaseSensitive(usage, "cache_read_input_tokens");
const cJSON *awrite =
cJSON_GetObjectItemCaseSensitive(usage, "cache_creation_input_tokens");
if (cJSON_IsNumber(pt))
s->st->prompt_tokens = (long)pt->valuedouble;
if (cJSON_IsNumber(ct))
@@ -345,6 +776,19 @@ static void handle_payload(sse_t *s, const char *payload) {
s->st->cached_tokens = (long)hit->valuedouble;
else if (cJSON_IsNumber(cached))
s->st->cached_tokens = (long)cached->valuedouble;
else if (cJSON_IsNumber(aread))
s->st->cached_tokens = (long)aread->valuedouble;
if (cJSON_IsNumber(dwrite))
s->st->cache_write_tokens = (long)dwrite->valuedouble;
else if (cJSON_IsNumber(awrite))
s->st->cache_write_tokens = (long)awrite->valuedouble;
if (cJSON_IsNumber(miss)) {
s->st->cache_miss_tokens = (long)miss->valuedouble;
} else {
long m = s->st->prompt_tokens - s->st->cached_tokens -
s->st->cache_write_tokens;
s->st->cache_miss_tokens = m > 0 ? m : 0;
}
}
const cJSON *choices = cJSON_GetObjectItemCaseSensitive(root, "choices");
const cJSON *ch = cJSON_IsArray(choices) ? cJSON_GetArrayItem(choices, 0)
@@ -487,15 +931,37 @@ static void trace_path(const char *dir, int seq, const char *ext, char *out,
snprintf(out, n, "%s/%04d.%s", dir, seq, ext);
}
static void format_http_error(sse_t *s, long http, char *err, size_t err_sz);
/* NNNN.sse is fed as the stream; with an NNNN.status other than 200 it is
* the error body of that HTTP status instead. */
static int replay_run(sse_t *s, const char *dir, int seq, char *err,
size_t err_sz) {
char path[OS_PATH_MAX];
trace_path(dir, seq, "status", path, sizeof(path));
FILE *sf = fopen(path, "rb");
long status = 200;
if (sf) {
if (fscanf(sf, "%ld", &status) != 1)
status = 200;
fclose(sf);
}
trace_path(dir, seq, "sse", path, sizeof(path));
FILE *fp = fopen(path, "rb");
if (!fp) {
snprintf(err, err_sz, "replay: missing %s", path);
return -1;
}
if (status != 200) {
char eb[4096];
size_t got = fread(eb, 1, sizeof(eb), fp);
fclose(fp);
cst_buf_append(&s->errbody, eb, got);
if (s->st)
s->st->http_status = status;
format_http_error(s, status, err, err_sz);
return -1;
}
size_t chunk = 16;
const char *e = getenv("CST_REPLAY_CHUNK");
if (e && atoi(e) > 0)
@@ -621,6 +1087,7 @@ int cst_llm_chat_stream(const cst_llm_request_t *req, char *err,
ret = -1;
break;
}
cst_curl_apply_ca(curl);
sse_reset(&s);
s.curl = curl;
if (rec_dir && rec_dir[0]) {
@@ -702,7 +1169,7 @@ finish:
/* --- model listing -------------------------------------------------------- */
#define MODELS_BODY_MAX (2 * 1024 * 1024)
#define MODELS_BODY_MAX (8 * 1024 * 1024)
static size_t models_write_cb(char *ptr, size_t size, size_t nmemb, void *ud) {
cst_buf_t *b = ud;
@@ -712,18 +1179,97 @@ static size_t models_write_cb(char *ptr, size_t size, size_t nmemb, void *ud) {
return n;
}
static int add_model(char out[][128], int n, int max_out, const char *id) {
if (!id || !id[0] || n >= max_out || strlen(id) >= 128)
/* A number, or a numeric string (OpenRouter prices are strings). */
static int json_num(const cJSON *v, double *out) {
if (cJSON_IsNumber(v)) {
*out = v->valuedouble;
return 0;
}
if (cJSON_IsString(v) && v->valuestring[0]) {
char *end;
double d = strtod(v->valuestring, &end);
if (end != v->valuestring && !*end) {
*out = d;
return 0;
}
}
return -1;
}
static int has_image(const cJSON *arr) {
const cJSON *x;
cJSON_ArrayForEach(x, arr) {
if (cJSON_IsString(x) && strcmp(x->valuestring, "image") == 0)
return 1;
}
return 0;
}
/* Optional OpenRouter-style metadata: context_length (or context_window /
* max_model_len), architecture.input_modalities, modalities, and pricing in
* USD per token. */
static void model_meta(const cJSON *it, cst_model_meta_t *m) {
static const char *const ctx_keys[] = {"context_length", "context_window",
"max_model_len"};
for (size_t k = 0; k < sizeof(ctx_keys) / sizeof(ctx_keys[0]); k++) {
double d;
if (json_num(cJSON_GetObjectItemCaseSensitive(it, ctx_keys[k]), &d) == 0 &&
d > 0) {
m->context = (long)d;
break;
}
}
const cJSON *arch = cJSON_GetObjectItemCaseSensitive(it, "architecture");
const cJSON *in = cJSON_GetObjectItemCaseSensitive(arch, "input_modalities");
const cJSON *mods = cJSON_GetObjectItemCaseSensitive(it, "modalities");
if (cJSON_IsObject(mods))
mods = cJSON_GetObjectItemCaseSensitive(mods, "input");
const cJSON *modality = cJSON_GetObjectItemCaseSensitive(arch, "modality");
if (cJSON_IsArray(in))
m->vision = has_image(in);
else if (cJSON_IsArray(mods))
m->vision = has_image(mods);
else if (cJSON_IsString(modality)) {
const char *arrow = strstr(modality->valuestring, "->");
size_t l = arrow ? (size_t)(arrow - modality->valuestring)
: strlen(modality->valuestring);
char left[64];
snprintf(left, sizeof(left), "%.*s", (int)l, modality->valuestring);
m->vision = strstr(left, "image") != NULL;
}
const cJSON *pr = cJSON_GetObjectItemCaseSensitive(it, "pricing");
double miss, out;
if (json_num(cJSON_GetObjectItemCaseSensitive(pr, "prompt"), &miss) == 0 &&
json_num(cJSON_GetObjectItemCaseSensitive(pr, "completion"), &out) == 0 &&
miss >= 0 && out >= 0) {
double hit = miss, write = miss;
json_num(cJSON_GetObjectItemCaseSensitive(pr, "input_cache_read"), &hit);
json_num(cJSON_GetObjectItemCaseSensitive(pr, "input_cache_write"), &write);
m->has_price = 1;
m->miss = miss * 1e6;
m->out = out * 1e6;
m->hit = (hit >= 0 ? hit : miss) * 1e6;
m->write = (write >= 0 ? write : miss) * 1e6;
}
}
static int add_model(cst_model_meta_t *out, int n, int max_out,
const char *id, const cJSON *it) {
if (!id || !id[0] || n >= max_out || strlen(id) >= sizeof(out->id))
return n;
for (int i = 0; i < n; i++)
if (strcmp(out[i], id) == 0)
if (strcmp(out[i].id, id) == 0)
return n;
snprintf(out[n], 128, "%s", id);
memset(&out[n], 0, sizeof(out[n]));
out[n].vision = -1;
snprintf(out[n].id, sizeof(out[n].id), "%s", id);
if (cJSON_IsObject(it))
model_meta(it, &out[n]);
return n + 1;
}
/* {"data":[{"id":..}]}, {"models":[{"id"|"name":..}]} or a bare array. */
static int parse_models(const char *json, char out[][128], int max_out) {
static int parse_models(const char *json, cst_model_meta_t *out, int max_out) {
cJSON *root = cJSON_Parse(json);
if (!root)
return -1;
@@ -741,9 +1287,9 @@ static int parse_models(const char *json, char out[][128], int max_out) {
if (!cJSON_IsString(id))
id = cJSON_GetObjectItemCaseSensitive(it, "name");
if (cJSON_IsString(id))
n = add_model(out, n, max_out, id->valuestring);
n = add_model(out, n, max_out, id->valuestring, it);
else if (cJSON_IsString(it))
n = add_model(out, n, max_out, it->valuestring);
n = add_model(out, n, max_out, it->valuestring, NULL);
}
}
cJSON_Delete(root);
@@ -753,6 +1299,24 @@ static int parse_models(const char *json, char out[][128], int max_out) {
int cst_llm_list_models(const char *api_base, const char *api_key,
char out[][128], int max_out, char *err,
size_t err_sz) {
if (!out || max_out <= 0)
return cst_llm_list_models_meta(api_base, api_key, NULL, 0, err, err_sz);
cst_model_meta_t *m = calloc((size_t)max_out, sizeof(*m));
if (!m) {
if (err && err_sz)
snprintf(err, err_sz, "out of memory");
return -1;
}
int n = cst_llm_list_models_meta(api_base, api_key, m, max_out, err, err_sz);
for (int i = 0; i < n; i++)
snprintf(out[i], 128, "%s", m[i].id);
free(m);
return n;
}
int cst_llm_list_models_meta(const char *api_base, const char *api_key,
cst_model_meta_t *out, int max_out, char *err,
size_t err_sz) {
char dummy[8];
if (!err || !err_sz) {
err = dummy;
@@ -768,6 +1332,7 @@ int cst_llm_list_models(const char *api_base, const char *api_key,
snprintf(err, err_sz, "curl init failed");
return -1;
}
cst_curl_apply_ca(curl);
char url[1024];
size_t bl = strlen(api_base);
while (bl && api_base[bl - 1] == '/')
+192 -11
View File
@@ -1,7 +1,9 @@
#include "cst_agent.h"
#include "cst_ext.h"
#include "cst_models.h"
#include "cst_tools.h"
#include "cst_version.h"
#include "cst_webui.h"
#include "os_compat.h"
#include <curl/curl.h>
@@ -20,14 +22,19 @@
static void usage(const char *argv0) {
fprintf(stderr,
"Usage: %s [options] (works in the current directory)\n"
" -P, --provider NAME minimax | deepseek | openai (last used)\n"
" -P, --provider NAME minimax | deepseek | openai | teamorouter\n"
" (last used)\n"
" -m, --model NAME model (last used for the provider)\n"
" -e, --effort LEVEL low | medium | high | max (last used)\n"
" -M, --mode NAME coding (default, code framed) | general\n"
" (documents, data, research; last used)\n"
" -w, --workspace DIR project directory (default: current)\n"
" -c, --continue reopen the last session (default)\n"
" -N, --new start a new session\n"
" -R, --resume ID open a session by id or unique prefix\n"
" -L, --list-sessions list sessions and exit\n"
" --list-models [P] what is known about each provider's models\n"
" (built-in, <config>/models.json, GET /models)\n"
" -A, --approval MODE ask | auto (default: edits run, shell in\n"
" the sandbox or by safe rules) | yolo\n"
" -y, --yolo same as --approval yolo (this run only)\n"
@@ -37,6 +44,13 @@ static void usage(const char *argv0) {
" -T, --tools PROFILE default (all tools) | minimal (paper set)\n"
" -n, --max-steps N model calls per turn (30)\n"
" -H, --headless -p TEXT run one prompt without the TUI\n"
" --web serve the UI over HTTP on 127.0.0.1:8787\n"
" --web-port N port (default 8787, 0 = pick a free one)\n"
" --web-host ADDR bind address (default 127.0.0.1; a\n"
" non-loopback address requires a token)\n"
" --web-token T require ?token=T / Authorization: Bearer T\n"
" --web-dev DIR serve the UI from DIR (development)\n"
" --web-open open the browser at startup\n"
" -a, --ask-key enter and save the provider's API key\n"
" -k, --api-key KEY use KEY for this run only (not saved)\n"
" -b, --api-base URL OpenAI-compatible base URL (…/v1)\n"
@@ -47,13 +61,13 @@ static void usage(const char *argv0) {
"API keys: the TUI asks for a missing key and saves it per provider\n"
" in the config dir (credentials, mode 0600); /key changes it.\n"
" Fallbacks: <provider env> (DEEPSEEK_API_KEY, ...), CST_API_KEY.\n"
"Settings: provider, models and effort are remembered in\n"
"Settings: provider, models, effort and mode are remembered in\n"
" settings.json next to the credentials.\n"
"Env: CST_PROVIDER CST_MODEL CST_EFFORT CST_PROTOCOL CST_TOOLS\n"
" CST_CONFIRM CST_MAX_STEPS CST_TOOL_OUT_CAP CST_CTX_WINDOW\n"
" CST_CTX_TOKENS CST_AUTO_COMPACT=0 CST_SHELL_TIMEOUT\n"
" CST_METRICS=0 CST_RECORD_SSE CST_REPLAY_SSE CST_MODELS\n"
" CST_APPROVAL=ask|auto|yolo CST_AUTO_APPROVE=1 (=yolo)\n"
"Env: CST_PROVIDER CST_MODEL CST_EFFORT CST_MODE CST_PROTOCOL\n"
" CST_TOOLS CST_CONFIRM CST_MAX_STEPS CST_TOOL_OUT_CAP\n"
" CST_CTX_WINDOW CST_CTX_TOKENS CST_AUTO_COMPACT=0\n"
" CST_SHELL_TIMEOUT CST_METRICS=0 CST_RECORD_SSE CST_REPLAY_SSE\n"
" CST_MODELS CST_APPROVAL=ask|auto|yolo CST_AUTO_APPROVE=1 (=yolo)\n"
" CST_SANDBOX=off CST_SHELL CST_NO_PROJECT_NOTES=1 NO_COLOR\n"
"TUI: type /help inside for commands and keys\n",
argv0);
@@ -80,6 +94,62 @@ static int parse_confirm(const char *s, cst_agent_config_t *cfg) {
return 0;
}
static void list_one_model(const char *provider, const char *id,
const char *where) {
cst_model_info_t mi;
cst_model_info(provider, id, &mi);
char tag[48], price[128] = "";
cst_model_info_label(&mi, tag, sizeof(tag));
if (mi.has_price)
snprintf(price, sizeof(price), " %s%g/%g/%g per 1M (miss/hit/out)",
mi.price.currency, mi.price.miss, mi.price.hit, mi.price.out);
printf(" %-40s %-18s %-8s%s [%s]\n", id, tag, where, price, mi.source);
}
/* --list-models [provider]: the merged registry, to check models.json. */
static int list_models(const char *only) {
char user[1100] = "";
cst_models_user_path(user, sizeof(user));
FILE *uf = user[0] ? fopen(user, "rb") : NULL;
printf("user file: %s%s\n", user[0] ? user : "(no config dir)",
uf ? "" : " (none)");
if (uf)
fclose(uf);
if (cst_models_load_error()[0])
printf("warning: %s\n", cst_models_load_error());
int shown = 0;
static cst_model_meta_t cached[1024];
for (int i = 0; i < cst_provider_count(); i++) {
const cst_provider_t *p = cst_provider_at(i);
if (only && only[0] && strcmp(p->name, only) != 0)
continue;
shown++;
printf("\n%s %s (%s; key %s)\n", p->name, p->api_base,
cst_provider_source(p), p->key_env[0] ? p->key_env : "-");
char listed[16][128];
int nl = cst_provider_models(p, listed, 16);
for (int k = 0; k < nl; k++)
list_one_model(p->name, listed[k],
strcmp(listed[k], p->model) == 0 ? "default" : "listed");
long age = 0;
int nc = cst_models_cache_load(p->name, cached, 1024, &age);
if (nc > 0) {
printf(" -- %d from GET /models, fetched %ldh ago%s\n", nc,
age / 3600, age >= CST_MODELS_CACHE_TTL ? " (stale)" : "");
for (int k = 0; k < nc; k++)
list_one_model(p->name, cached[k].id, "api");
} else if (!nl) {
printf(" (no models listed; the first one GET /models returns is "
"used)\n");
}
}
if (only && only[0] && !shown) {
fprintf(stderr, "unknown provider: %s\n", only);
return 1;
}
return 0;
}
static int list_sessions(const char *ws) {
static char ids[200][128];
int n = cst_agent_store_list(ws, ids, 200);
@@ -187,8 +257,22 @@ static int demo_ctl_impl(const char *workspace) {
return 0;
}
/* 0..65535, digits only: atoi turned "70000" into port 4464, "abc" into 0. */
static int parse_port(const char *s, int *out) {
char *end = NULL;
long v = strtol(s, &end, 10);
if (!s[0] || *end || v < 0 || v > 65535)
return -1;
*out = (int)v;
return 0;
}
int main(int argc, char **argv) {
if (os_console_init() != 0)
fprintf(stderr, "warning: the process code page is not UTF-8 (Windows 10 "
"1903 or later is required); non-ASCII text may break\n");
os_set_prog_path(argv[0]);
os_temp_dir_export();
curl_global_init(CURL_GLOBAL_DEFAULT);
cst_agent_config_t cfg = {0};
@@ -256,6 +340,19 @@ int main(int argc, char **argv) {
const char *prompt = NULL;
int new_session = 0, do_list = 0;
const char *resume_id = NULL;
/* WebUI (--web): a second front end on the same engine. */
int web_mode = 0;
cst_webui_opts_t web;
memset(&web, 0, sizeof(web));
web.port = CST_WEBUI_DEFAULT_PORT; /* 0 would mean "pick a free one" */
const char *env = getenv("CST_WEB_PORT");
if (env && env[0] && parse_port(env, &web.port) != 0) {
fprintf(stderr, "CST_WEB_PORT=%s is not a port (0-65535)\n", env);
return 1;
}
web.host = getenv("CST_WEB_HOST");
web.token = getenv("CST_WEB_TOKEN");
web.root = getenv("CST_WEB_ROOT");
static struct option opts[] = {
{"workspace", required_argument, 0, 'w'},
@@ -279,12 +376,20 @@ int main(int argc, char **argv) {
{"approval", required_argument, 0, 'A'},
{"confirm", required_argument, 0, 'C'},
{"effort", required_argument, 0, 'e'},
{"mode", required_argument, 0, 'M'},
{"help", no_argument, 0, 'h'},
{"version", no_argument, 0, 'V'},
{"list-models", optional_argument, 0, 0x100},
{"web", no_argument, 0, 0x101},
{"web-port", required_argument, 0, 0x102},
{"web-host", required_argument, 0, 0x103},
{"web-token", required_argument, 0, 0x104},
{"web-dev", required_argument, 0, 0x105},
{"web-open", no_argument, 0, 0x106},
{0, 0, 0, 0}};
int c;
while ((c = getopt_long(argc, argv, "w:s:b:k:m:P:r:n:T:Hp:dahVNcR:LyA:C:e:", opts, NULL)) !=
-1) {
while ((c = getopt_long(argc, argv, "w:s:b:k:m:P:r:n:T:Hp:dahVNcR:LyA:C:e:M:",
opts, NULL)) != -1) {
switch (c) {
case 'w':
snprintf(cfg.workspace_root, sizeof(cfg.workspace_root), "%s", optarg);
@@ -376,10 +481,53 @@ int main(int argc, char **argv) {
}
snprintf(cfg.effort, sizeof(cfg.effort), "%s", optarg);
break;
case 'M':
if (!cst_mode_valid(optarg)) {
fprintf(stderr, "bad --mode %s (coding | general)\n", optarg);
curl_global_cleanup();
return 1;
}
snprintf(cfg.mode, sizeof(cfg.mode), "%s", optarg);
break;
case 'h':
usage(argv[0]);
curl_global_cleanup();
return 0;
case 0x100: {
const char *only = optarg;
if (!only && optind < argc && argv[optind][0] != '-')
only = argv[optind++];
int rc = list_models(only);
curl_global_cleanup();
return rc;
}
case 0x101:
web_mode = 1;
break;
case 0x102:
if (parse_port(optarg, &web.port) != 0) {
fprintf(stderr, "--web-port %s is not a port (0-65535)\n", optarg);
curl_global_cleanup();
return 1;
}
web_mode = 1;
break;
case 0x103:
web.host = optarg;
web_mode = 1;
break;
case 0x104:
web.token = optarg;
web_mode = 1;
break;
case 0x105:
web.root = optarg;
web_mode = 1;
break;
case 0x106:
web.open_browser = 1;
web_mode = 1;
break;
case 'V':
printf("cst-agent %s\n", CST_VERSION);
curl_global_cleanup();
@@ -420,6 +568,12 @@ int main(int argc, char **argv) {
snprintf(cfg.effort, sizeof(cfg.effort), "%s",
cst_effort_valid(e) ? e : CST_DEFAULT_EFFORT);
}
if (!cfg.mode[0]) {
const char *m = getenv("CST_MODE");
if (!cst_mode_valid(m))
m = saved.mode;
snprintf(cfg.mode, sizeof(cfg.mode), "%s", cst_mode_name(m));
}
if (!cfg.api_base[0]) {
const char *env_base = getenv("CST_API_BASE");
if (!env_base || !env_base[0])
@@ -527,7 +681,7 @@ int main(int argc, char **argv) {
return 0;
}
}
if (!cfg.api_key[0] && cfg.headless) {
if (!cfg.api_key[0] && (cfg.headless || web_mode)) {
fprintf(stderr,
"no API key for %s: start cst-agent once to enter it, or run "
"cst-agent --ask-key --provider %s\n",
@@ -535,6 +689,30 @@ int main(int argc, char **argv) {
curl_global_cleanup();
return 1;
}
/* Router presets have no fixed model: take the first one the API lists
* (the TUI does the same after its /models probe). */
if (!cfg.model[0] && (cfg.headless || web_mode)) {
static cst_model_meta_t list[512];
char err[256] = "";
long age = 0;
int n = cst_models_cache_load(cfg.provider, list, 1, &age);
if (n <= 0 || age >= CST_MODELS_CACHE_TTL) {
int r = cst_llm_list_models_meta(cfg.api_base, cfg.api_key, list, 512,
err, sizeof(err));
if (r >= 0)
cst_models_cache_save(cfg.provider, cfg.api_base, list, r);
if (r >= 0 || n <= 0)
n = r;
}
if (n > 0) {
snprintf(cfg.model, sizeof(cfg.model), "%s", list[0].id);
} else {
fprintf(stderr, "no model for %s: pass --model (%s)\n", cfg.provider,
err[0] ? err : "the API listed none");
curl_global_cleanup();
return 1;
}
}
fprintf(stderr,
"using %s model=%s effort=%s protocol=%s tools=%s max_steps=%d\n",
@@ -557,7 +735,10 @@ int main(int argc, char **argv) {
}
int rc = 0;
if (cfg.headless) {
if (web_mode) {
web.prompt = prompt; /* --web -p "...": run one turn at startup */
rc = cst_webui_run(agent, &web);
} else if (cfg.headless) {
if (!prompt) {
fprintf(stderr, "--headless requires --prompt\n");
rc = 1;
+62 -3
View File
@@ -6,6 +6,8 @@
#endif
#include "cst_mcp.h"
#include "cst_agent.h"
#include "cst_attach.h"
#include <pthread.h>
#include <stdatomic.h>
@@ -268,6 +270,7 @@ static char *slurp(const char *path, size_t max) {
fclose(fp);
if (!b.buf)
cst_buf_puts(&b, "");
cst_strip_bom(b.buf, &b.len);
return cst_buf_steal(&b);
}
@@ -601,7 +604,56 @@ static int auto_approved(const mcp_srv_t *s, const char *tool) {
return 0;
}
static void put_content(cst_buf_t *b, const cJSON *item) {
/* Images a tool call returned, stored beside the session. */
typedef struct {
const char *session;
const char *server;
cst_attachment_t *list;
int n, cap;
} mcp_imgs_t;
/* An image content item goes into the session store and rides along with the
* tool result; the text keeps a one-line description. 0 when stored. */
static int put_image(cst_buf_t *b, const cJSON *item, mcp_imgs_t *im) {
const cJSON *d = cJSON_GetObjectItemCaseSensitive(item, "data");
if (!im || !cJSON_IsString(d))
return -1;
char err[256] = "no session to store the image in";
if (im->n >= CST_ATTACH_MAX_COUNT) {
snprintf(err, sizeof(err), "more than %d images", CST_ATTACH_MAX_COUNT);
} else if (im->session && im->session[0]) {
size_t n = 0;
char *raw = cst_base64_decode(d->valuestring, strlen(d->valuestring), &n);
char name[96];
snprintf(name, sizeof(name), "%s-%d", im->server, im->n + 1);
cst_attachment_t *a =
raw ? cst_attach_import_bytes(im->session, name, raw, n, err,
sizeof(err))
: NULL;
if (!raw)
snprintf(err, sizeof(err), "bad base64");
free(raw);
if (a) {
snprintf(a->origin, sizeof(a->origin), "mcp");
int pushed = cst_attachments_push(&im->list, &im->n, &im->cap, a) == 0;
char desc[128];
cst_attach_describe(a, desc, sizeof(desc));
cst_attach_free(a);
if (pushed) {
cst_buf_printf(b, "[image %s (shown to you as an image)]", desc);
return 0;
}
snprintf(err, sizeof(err), "out of memory");
}
}
const cJSON *mt = cJSON_GetObjectItemCaseSensitive(item, "mimeType");
cst_buf_printf(b, "[image %s, %zu bytes base64, not shown: %s]",
cJSON_IsString(mt) ? mt->valuestring : "",
strlen(d->valuestring), err);
return -1;
}
static void put_content(cst_buf_t *b, const cJSON *item, mcp_imgs_t *im) {
const cJSON *type = cJSON_GetObjectItemCaseSensitive(item, "type");
const char *t = cJSON_IsString(type) ? type->valuestring : "";
if (b->len && b->buf[b->len - 1] != '\n')
@@ -610,6 +662,9 @@ static void put_content(cst_buf_t *b, const cJSON *item) {
const cJSON *x = cJSON_GetObjectItemCaseSensitive(item, "text");
if (cJSON_IsString(x))
cst_buf_puts(b, x->valuestring);
} else if (strcmp(t, "image") == 0 &&
cJSON_IsString(cJSON_GetObjectItemCaseSensitive(item, "data"))) {
put_image(b, item, im);
} else if (strcmp(t, "image") == 0 || strcmp(t, "audio") == 0) {
const cJSON *mt = cJSON_GetObjectItemCaseSensitive(item, "mimeType");
const cJSON *d = cJSON_GetObjectItemCaseSensitive(item, "data");
@@ -659,7 +714,6 @@ static int set_out(cst_tool_result_t *r, int code, const char *fmt, ...) {
static int mcp_tool_run(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r, void *ud) {
(void)ctx;
mtool_t *mt = ud;
mcp_srv_t *s = mt->srv;
pthread_mutex_lock(&s->mu);
@@ -689,9 +743,12 @@ static int mcp_tool_run(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
const cJSON *res = cJSON_GetObjectItemCaseSensitive(resp, "result");
const cJSON *content = cJSON_GetObjectItemCaseSensitive(res, "content");
cst_buf_t b = {0};
mcp_imgs_t im = {0};
im.session = ctx && ctx->agent ? cst_agent_session_path(ctx->agent) : NULL;
im.server = s->name;
const cJSON *it = NULL;
cJSON_ArrayForEach(it, content) {
put_content(&b, it);
put_content(&b, it, &im);
if (b.len > MCP_OUT_MAX)
break;
}
@@ -715,6 +772,8 @@ static int mcp_tool_run(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cJSON_Delete(resp);
r->out_len = b.len;
r->out = cst_buf_steal(&b);
r->attach = im.list; /* one block, freed by cst_tool_result_free */
r->nattach = im.n;
return 0;
}
+1 -1
View File
@@ -47,7 +47,7 @@ cst_metrics_t *cst_metrics_open(const char *workspace, const char *session_path)
snprintf(file, sizeof(file), "%s.jsonl", leaf);
os_path_join(m->path, sizeof(m->path), dir, file);
}
m->fp = fopen(m->path, "a");
m->fp = fopen(m->path, "ab");
if (!m->fp) {
free(m);
return NULL;
+818
View File
@@ -0,0 +1,818 @@
#include "cst_models.h"
#include "cJSON.h"
#include "os_compat.h"
#include <ctype.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static const char BUILTIN[] = {
#include "models_builtin.inc"
0};
const char *cst_models_builtin_json(void) { return BUILTIN; }
#define MAX_PROV 32
#define MAX_PMODELS 16
#define MAX_RULES 256
typedef struct {
cst_provider_t pub; /* points into the buffers below */
char name[64];
char api_base[256];
char model[128];
char key_env[64];
int passback, prefix;
char effort[16]; /* "", "passthrough", "openai" */
char models[MAX_PMODELS][128];
int nmodels;
const char *source;
} prov_t;
enum { F_CONTEXT = 1, F_VISION = 2, F_CACHE = 4, F_REASONING = 8, F_PRICE = 16 };
typedef struct {
char match[128]; /* lowercase */
int spec; /* literal characters: higher is more specific */
unsigned has; /* F_* set by this rule */
long context;
int vision, cache, reasoning;
cst_price_t price;
} rule_t;
typedef struct {
char provider[64];
long fetched_at;
cst_model_meta_t m;
} cached_t;
static pthread_mutex_t g_mu = PTHREAD_MUTEX_INITIALIZER;
static struct {
int loaded;
prov_t prov[MAX_PROV];
int nprov;
rule_t builtin[MAX_RULES];
int nbuiltin;
rule_t user[MAX_RULES];
int nuser;
cached_t *cache;
int ncache;
char err[1400];
} R;
/* --- parsing ------------------------------------------------------------- */
static void lower_copy(char *out, size_t n, const char *s) {
size_t i = 0;
for (; s && s[i] && i + 1 < n; i++)
out[i] = (char)tolower((unsigned char)s[i]);
out[i] = 0;
}
static const char *jstr(const cJSON *o, const char *k) {
const cJSON *v = cJSON_GetObjectItemCaseSensitive(o, k);
return cJSON_IsString(v) ? v->valuestring : NULL;
}
static void fix_pub(prov_t *p) {
p->pub.name = p->name;
p->pub.api_base = p->api_base;
p->pub.model = p->model;
p->pub.key_env = p->key_env;
}
static prov_t *prov_by_name(const char *name) {
for (int i = 0; i < R.nprov; i++)
if (strcmp(R.prov[i].name, name) == 0)
return &R.prov[i];
return NULL;
}
static void parse_providers(const cJSON *obj, const char *source) {
const cJSON *it = NULL;
cJSON_ArrayForEach(it, obj) {
if (!cJSON_IsObject(it) || !it->string || !it->string[0])
continue;
prov_t *p = prov_by_name(it->string);
if (!p) {
if (R.nprov >= MAX_PROV)
continue;
p = &R.prov[R.nprov++];
memset(p, 0, sizeof(*p));
snprintf(p->name, sizeof(p->name), "%s", it->string);
fix_pub(p);
}
p->source = source;
const char *s;
if ((s = jstr(it, "api_base")))
snprintf(p->api_base, sizeof(p->api_base), "%s", s);
if ((s = jstr(it, "key_env")))
snprintf(p->key_env, sizeof(p->key_env), "%s", s);
if ((s = jstr(it, "default_model")))
snprintf(p->model, sizeof(p->model), "%s", s);
if ((s = jstr(it, "effort")))
snprintf(p->effort, sizeof(p->effort), "%s", s);
const cJSON *b = cJSON_GetObjectItemCaseSensitive(it, "reasoning_passback");
if (cJSON_IsBool(b))
p->passback = cJSON_IsTrue(b);
b = cJSON_GetObjectItemCaseSensitive(it, "prefix_completion");
if (cJSON_IsBool(b))
p->prefix = cJSON_IsTrue(b);
const cJSON *ms = cJSON_GetObjectItemCaseSensitive(it, "models");
if (cJSON_IsArray(ms)) {
p->nmodels = 0;
const cJSON *m = NULL;
cJSON_ArrayForEach(m, ms) {
if (cJSON_IsString(m) && m->valuestring[0] && p->nmodels < MAX_PMODELS)
snprintf(p->models[p->nmodels++], 128, "%s", m->valuestring);
}
}
}
}
static int parse_price(const cJSON *o, cst_price_t *pr) {
const cJSON *miss = cJSON_GetObjectItemCaseSensitive(o, "miss");
const cJSON *out = cJSON_GetObjectItemCaseSensitive(o, "out");
if (!cJSON_IsNumber(miss) || !cJSON_IsNumber(out))
return -1;
memset(pr, 0, sizeof(*pr));
pr->miss = miss->valuedouble;
pr->out = out->valuedouble;
const cJSON *v = cJSON_GetObjectItemCaseSensitive(o, "hit");
pr->hit = cJSON_IsNumber(v) ? v->valuedouble : pr->miss;
v = cJSON_GetObjectItemCaseSensitive(o, "write");
pr->write = cJSON_IsNumber(v) ? v->valuedouble : pr->miss;
const char *c = jstr(o, "currency");
snprintf(pr->currency, sizeof(pr->currency), "%s", c ? c : "$");
pr->offpeak_half =
cJSON_IsTrue(cJSON_GetObjectItemCaseSensitive(o, "offpeak_half"));
return 0;
}
static int parse_rule(const cJSON *o, rule_t *r) {
const char *m = jstr(o, "match");
if (!m || !m[0])
return -1;
memset(r, 0, sizeof(*r));
lower_copy(r->match, sizeof(r->match), m);
for (const char *q = r->match; *q; q++)
if (*q != '*')
r->spec++;
const cJSON *v = cJSON_GetObjectItemCaseSensitive(o, "context");
if (cJSON_IsNumber(v) && v->valuedouble > 0) {
r->context = (long)v->valuedouble;
r->has |= F_CONTEXT;
}
v = cJSON_GetObjectItemCaseSensitive(o, "vision");
if (cJSON_IsBool(v)) {
r->vision = cJSON_IsTrue(v);
r->has |= F_VISION;
}
v = cJSON_GetObjectItemCaseSensitive(o, "reasoning");
if (cJSON_IsBool(v)) {
r->reasoning = cJSON_IsTrue(v);
r->has |= F_REASONING;
}
const char *c = jstr(o, "cache");
if (c) {
if (strcmp(c, "openai") == 0)
r->cache = CST_CACHE_OPENAI;
else if (strcmp(c, "anthropic") == 0)
r->cache = CST_CACHE_ANTHROPIC;
else
r->cache = CST_CACHE_AUTO;
r->has |= F_CACHE;
}
v = cJSON_GetObjectItemCaseSensitive(o, "price");
if (cJSON_IsObject(v) && parse_price(v, &r->price) == 0)
r->has |= F_PRICE;
return 0;
}
static int parse_doc(const char *json, int user, char *err, size_t err_sz) {
cJSON *doc = cJSON_Parse(json);
if (!cJSON_IsObject(doc)) {
cJSON_Delete(doc);
snprintf(err, err_sz, "not a JSON object");
return -1;
}
parse_providers(cJSON_GetObjectItemCaseSensitive(doc, "providers"),
user ? "user" : "built-in");
rule_t *rules = user ? R.user : R.builtin;
int *n = user ? &R.nuser : &R.nbuiltin;
const cJSON *it = NULL;
cJSON_ArrayForEach(it, cJSON_GetObjectItemCaseSensitive(doc, "models")) {
if (*n < MAX_RULES && parse_rule(it, &rules[*n]) == 0)
(*n)++;
}
cJSON_Delete(doc);
return 0;
}
static char *read_file(const char *path) {
FILE *f = fopen(path, "rb");
if (!f)
return NULL;
char *buf = NULL;
size_t len = 0;
char chunk[8192];
size_t got;
while ((got = fread(chunk, 1, sizeof(chunk), f)) > 0 && len < (8u << 20)) {
char *nb = realloc(buf, len + got + 1);
if (!nb)
break;
buf = nb;
memcpy(buf + len, chunk, got);
len += got;
}
fclose(f);
if (buf)
buf[len] = 0;
return buf;
}
static int cache_path(const char *provider, char *out, size_t n) {
char dir[1024];
if (!provider || !provider[0] || os_config_dir(dir, sizeof(dir)) != 0)
return -1;
for (const char *q = provider; *q; q++)
if (!isalnum((unsigned char)*q) && !strchr("._-", *q))
return -1;
int w = snprintf(out, n, "%s/models-cache/%s.json", dir, provider);
return w > 0 && (size_t)w < n ? 0 : -1;
}
static void meta_from_json(const cJSON *o, cst_model_meta_t *m) {
memset(m, 0, sizeof(*m));
m->vision = -1;
const char *id = jstr(o, "id");
snprintf(m->id, sizeof(m->id), "%s", id ? id : "");
const cJSON *v = cJSON_GetObjectItemCaseSensitive(o, "context");
if (cJSON_IsNumber(v) && v->valuedouble > 0)
m->context = (long)v->valuedouble;
v = cJSON_GetObjectItemCaseSensitive(o, "vision");
if (cJSON_IsBool(v))
m->vision = cJSON_IsTrue(v);
v = cJSON_GetObjectItemCaseSensitive(o, "price");
cst_price_t pr;
if (cJSON_IsObject(v) && parse_price(v, &pr) == 0) {
m->has_price = 1;
m->miss = pr.miss;
m->hit = pr.hit;
m->write = pr.write;
m->out = pr.out;
}
}
/* Appends provider's cache file to R.cache (caller holds the lock). */
static void load_cache(const char *provider) {
char path[1100];
if (cache_path(provider, path, sizeof(path)) != 0)
return;
char *txt = read_file(path);
cJSON *doc = txt ? cJSON_Parse(txt) : NULL;
free(txt);
const cJSON *fa = cJSON_GetObjectItemCaseSensitive(doc, "fetched_at");
long at = cJSON_IsNumber(fa) ? (long)fa->valuedouble : 0;
const cJSON *it = NULL;
cJSON_ArrayForEach(it, cJSON_GetObjectItemCaseSensitive(doc, "models")) {
cst_model_meta_t m;
meta_from_json(it, &m);
if (!m.id[0])
continue;
cached_t *nc = realloc(R.cache, (size_t)(R.ncache + 1) * sizeof(*nc));
if (!nc)
break;
R.cache = nc;
cached_t *c = &R.cache[R.ncache++];
snprintf(c->provider, sizeof(c->provider), "%s", provider);
c->fetched_at = at;
c->m = m;
}
cJSON_Delete(doc);
}
static void drop_cache(const char *provider) {
int w = 0;
for (int i = 0; i < R.ncache; i++)
if (provider && strcmp(R.cache[i].provider, provider) != 0)
R.cache[w++] = R.cache[i];
R.ncache = w;
}
static void reset_locked(void) {
R.nprov = 0;
R.nbuiltin = 0;
R.nuser = 0;
drop_cache(NULL);
R.err[0] = 0;
}
static void load_caches_locked(void) {
for (int i = 0; i < R.nprov; i++)
load_cache(R.prov[i].name);
}
int cst_models_user_path(char *out, size_t out_sz) {
char dir[1024];
if (os_config_dir(dir, sizeof(dir)) != 0)
return -1;
int w = snprintf(out, out_sz, "%s/models.json", dir);
return w > 0 && (size_t)w < out_sz ? 0 : -1;
}
static void load_default_locked(void) {
reset_locked();
char e[128];
if (parse_doc(BUILTIN, 0, e, sizeof(e)) != 0)
snprintf(R.err, sizeof(R.err), "built-in model data: %s", e);
char path[1100];
char *user = cst_models_user_path(path, sizeof(path)) == 0
? read_file(path)
: NULL;
if (user && parse_doc(user, 1, e, sizeof(e)) != 0)
snprintf(R.err, sizeof(R.err), "%s: %s", path, e);
free(user);
load_caches_locked();
R.loaded = 1;
}
static void ensure_locked(void) {
if (!R.loaded)
load_default_locked();
}
void cst_models_reload(void) {
pthread_mutex_lock(&g_mu);
load_default_locked();
pthread_mutex_unlock(&g_mu);
}
int cst_models_load_json(const char *builtin_json, const char *user_json,
char *err, size_t err_sz) {
char dummy[8];
if (!err || !err_sz) {
err = dummy;
err_sz = sizeof(dummy);
}
err[0] = 0;
pthread_mutex_lock(&g_mu);
reset_locked();
int rc = 0;
if (builtin_json && parse_doc(builtin_json, 0, err, err_sz) != 0)
rc = -1;
if (rc == 0 && user_json && parse_doc(user_json, 1, err, err_sz) != 0)
rc = -1;
load_caches_locked();
R.loaded = 1;
pthread_mutex_unlock(&g_mu);
return rc;
}
const char *cst_models_load_error(void) {
pthread_mutex_lock(&g_mu);
ensure_locked();
pthread_mutex_unlock(&g_mu);
return R.err;
}
/* --- providers ----------------------------------------------------------- */
const cst_provider_t *cst_provider_find(const char *name) {
if (!name || !name[0])
return NULL;
pthread_mutex_lock(&g_mu);
ensure_locked();
prov_t *p = prov_by_name(name);
pthread_mutex_unlock(&g_mu);
return p ? &p->pub : NULL;
}
const cst_provider_t *cst_provider_default(void) {
const cst_provider_t *p = cst_provider_find(getenv("CST_PROVIDER"));
return p ? p : cst_provider_at(0);
}
int cst_provider_count(void) {
pthread_mutex_lock(&g_mu);
ensure_locked();
int n = R.nprov;
pthread_mutex_unlock(&g_mu);
return n;
}
const cst_provider_t *cst_provider_at(int i) {
pthread_mutex_lock(&g_mu);
ensure_locked();
const cst_provider_t *p = i >= 0 && i < R.nprov ? &R.prov[i].pub : NULL;
pthread_mutex_unlock(&g_mu);
return p;
}
const char *cst_provider_source(const cst_provider_t *p) {
for (int i = 0; p && i < MAX_PROV; i++)
if (p == &R.prov[i].pub)
return R.prov[i].source ? R.prov[i].source : "built-in";
return "";
}
int cst_provider_models(const cst_provider_t *p, char out[][128], int max) {
int n = 0;
pthread_mutex_lock(&g_mu);
for (int i = 0; p && i < R.nprov; i++) {
if (p != &R.prov[i].pub)
continue;
for (int k = 0; k < R.prov[i].nmodels && n < max; k++)
snprintf(out[n++], 128, "%s", R.prov[i].models[k]);
}
pthread_mutex_unlock(&g_mu);
return n;
}
static int push_name(char out[][128], int *n, int max, const char *name) {
if (!name || !name[0] || *n >= max)
return 0;
for (int i = 0; i < *n; i++)
if (strcmp(out[i], name) == 0)
return 0;
snprintf(out[(*n)++], 128, "%s", name);
return 1;
}
int cst_models_list(const char *current, char out[][128], int max_out) {
if (!out || max_out <= 0)
return 0;
int n = 0;
const char *env = getenv("CST_MODELS");
if (env && env[0]) {
char buf[1024];
snprintf(buf, sizeof(buf), "%s", env);
for (char *tok = buf; *tok;) {
while (*tok == ':' || *tok == ',' || *tok == ' ')
tok++;
char *end = tok;
while (*end && *end != ':' && *end != ',')
end++;
char save = *end;
*end = 0;
char *t = end;
while (t > tok && (t[-1] == ' ' || t[-1] == '\t'))
*--t = 0;
push_name(out, &n, max_out, tok);
*end = save;
tok = *end ? end + 1 : end;
}
} else {
pthread_mutex_lock(&g_mu);
ensure_locked();
for (int i = 0; i < R.nprov; i++)
for (int k = 0; k < R.prov[i].nmodels; k++)
push_name(out, &n, max_out, R.prov[i].models[k]);
pthread_mutex_unlock(&g_mu);
}
push_name(out, &n, max_out, current);
return n;
}
/* --- model lookup -------------------------------------------------------- */
/* '*' matches any run; without one the pattern is a prefix. */
static int glob(const char *p, const char *s) {
if (!strchr(p, '*'))
return strncmp(s, p, strlen(p)) == 0;
while (*p) {
if (*p == '*') {
while (*p == '*')
p++;
if (!*p)
return 1;
for (; *s; s++)
if (glob(p, s))
return 1;
return 0;
}
if (*p != *s)
return 0;
p++;
s++;
}
return !*s;
}
static void set_field(cst_model_info_t *o, unsigned f, const rule_t *r) {
switch (f) {
case F_CONTEXT:
o->context = r->context;
break;
case F_VISION:
o->vision = r->vision;
break;
case F_CACHE:
o->cache_style = r->cache;
break;
case F_REASONING:
o->reasoning = r->reasoning;
break;
case F_PRICE:
o->price = r->price;
o->has_price = 1;
break;
}
}
static void add_source(cst_model_info_t *o, const char *src) {
if (strstr(o->source, src))
return;
size_t l = strlen(o->source);
snprintf(o->source + l, sizeof(o->source) - l, "%s%s", l ? "+" : "", src);
}
static void apply_rules(const rule_t *rules, int n, const char *full,
const char *bare, unsigned *have, cst_model_info_t *o,
const char *src) {
for (unsigned f = 1; f <= F_PRICE; f <<= 1) {
if (*have & f)
continue;
const rule_t *best = NULL;
for (int i = 0; i < n; i++) {
const rule_t *r = &rules[i];
if (!(r->has & f) || (best && r->spec <= best->spec))
continue;
if (glob(r->match, strchr(r->match, '/') ? full : bare))
best = r;
}
if (best) {
set_field(o, f, best);
*have |= f;
add_source(o, src);
}
}
}
static void apply_cache(const char *provider, const char *model,
unsigned *have, cst_model_info_t *o) {
for (int i = 0; i < R.ncache; i++) {
const cached_t *c = &R.cache[i];
if ((provider && strcmp(c->provider, provider) != 0) ||
strcmp(c->m.id, model) != 0)
continue;
unsigned before = *have;
if (!(*have & F_CONTEXT) && c->m.context > 0) {
o->context = c->m.context;
*have |= F_CONTEXT;
}
if (!(*have & F_VISION) && c->m.vision >= 0) {
o->vision = c->m.vision;
*have |= F_VISION;
}
if (!(*have & F_PRICE) && c->m.has_price) {
memset(&o->price, 0, sizeof(o->price));
snprintf(o->price.currency, sizeof(o->price.currency), "$");
o->price.miss = c->m.miss;
o->price.hit = c->m.hit;
o->price.write = c->m.write;
o->price.out = c->m.out;
o->has_price = 1;
*have |= F_PRICE;
}
if (*have != before)
add_source(o, "api");
return;
}
}
static int env_price(cst_price_t *out) {
const char *e = getenv("CST_PRICE");
if (!e || !e[0])
return -1;
memset(out, 0, sizeof(*out));
char cur[8] = "";
int n = sscanf(e, "%lf,%lf,%lf,%lf,%7s", &out->miss, &out->hit, &out->out,
&out->write, cur);
if (n < 3)
return -1;
if (n < 4)
out->write = out->miss;
snprintf(out->currency, sizeof(out->currency), "%s", cur);
return 0;
}
void cst_model_info(const char *provider, const char *model,
cst_model_info_t *out) {
memset(out, 0, sizeof(*out));
out->vision = -1;
unsigned have = 0;
const char *e = getenv("CST_CTX_TOKENS");
if (e && atol(e) > 0) {
out->context = atol(e);
have |= F_CONTEXT;
add_source(out, "env");
}
if (env_price(&out->price) == 0) {
out->has_price = 1;
have |= F_PRICE;
add_source(out, "env");
}
if (model && model[0]) {
char full[256], bare[256];
lower_copy(full, sizeof(full), model);
const char *slash = strrchr(full, '/');
snprintf(bare, sizeof(bare), "%s", slash && slash[1] ? slash + 1 : full);
pthread_mutex_lock(&g_mu);
ensure_locked();
apply_rules(R.user, R.nuser, full, bare, &have, out, "user");
if (provider && !prov_by_name(provider))
provider = NULL;
apply_cache(provider, model, &have, out);
apply_rules(R.builtin, R.nbuiltin, full, bare, &have, out, "built-in");
pthread_mutex_unlock(&g_mu);
}
if (!(have & F_CONTEXT)) {
out->context = 128000;
add_source(out, "default");
}
}
void cst_model_info_label(const cst_model_info_t *mi, char *out, size_t n) {
char ctx[32];
if (mi->context >= 1000000)
snprintf(ctx, sizeof(ctx), "%.0fM", (double)mi->context / 1e6);
else
snprintf(ctx, sizeof(ctx), "%ldK", (mi->context + 500) / 1000);
snprintf(out, n, "%s%s", ctx,
mi->vision == 1 ? " · vision"
: mi->vision == 0 ? " · text only"
: "");
}
/* --- wrappers kept for the rest of the code ------------------------------ */
long cst_model_context_tokens(const char *model) {
cst_model_info_t i;
cst_model_info(NULL, model, &i);
return i.context;
}
int cst_model_price(const char *model, cst_price_t *out) {
cst_model_info_t i;
cst_model_info(NULL, model, &i);
*out = i.price;
if (!i.has_price)
memset(out, 0, sizeof(*out));
return i.has_price ? 0 : -1;
}
/* "https://api.deepseek.com/v1" -> "api.deepseek.com" */
static void host_of(const char *url, char *out, size_t n) {
const char *s = strstr(url, "://");
s = s ? s + 3 : url;
size_t i = 0;
for (; s[i] && s[i] != '/' && i + 1 < n; i++)
out[i] = s[i];
out[i] = 0;
}
void cst_model_caps(const char *provider, const char *api_base,
const char *model, cst_model_caps_t *out) {
memset(out, 0, sizeof(*out));
pthread_mutex_lock(&g_mu);
ensure_locked();
/* provider extensions follow the provider, or its host behind any name */
for (int i = 0; i < R.nprov; i++) {
const prov_t *p = &R.prov[i];
if (!p->passback && !p->prefix)
continue;
char host[256];
host_of(p->api_base, host, sizeof(host));
if ((provider && strcmp(provider, p->name) == 0) ||
(api_base && host[0] && strstr(api_base, host))) {
out->reasoning_passback |= p->passback;
out->prefix_completion |= p->prefix;
}
}
pthread_mutex_unlock(&g_mu);
cst_model_info_t mi;
cst_model_info(provider, model, &mi);
out->vision = mi.vision;
const char *e = getenv("CST_NO_CACHE_HINTS");
if ((e && e[0] == '1') || !model)
return;
out->cache_style = mi.cache_style;
}
const char *cst_effort_wire(const char *provider, const char *model,
const char *effort) {
if (!cst_effort_valid(effort) || !provider || !model)
return NULL;
char mode[16] = "";
pthread_mutex_lock(&g_mu);
ensure_locked();
const prov_t *p = prov_by_name(provider);
if (p)
snprintf(mode, sizeof(mode), "%s", p->effort);
pthread_mutex_unlock(&g_mu);
if (strcmp(mode, "passthrough") == 0)
return effort;
if (strcmp(mode, "openai") == 0) {
cst_model_info_t mi;
cst_model_info(provider, model, &mi);
if (mi.reasoning)
return strcmp(effort, "max") == 0 ? "high" : effort;
}
return NULL;
}
/* --- fetched cache ------------------------------------------------------- */
int cst_models_cache_save(const char *provider, const char *api_base,
const cst_model_meta_t *list, int n) {
char path[1100];
if (!list || n < 0 || cache_path(provider, path, sizeof(path)) != 0)
return -1;
char dir[1100];
snprintf(dir, sizeof(dir), "%s", path);
char *sl = strrchr(dir, '/');
if (sl)
*sl = 0;
os_mkdir_p(dir, 0700);
long now = (long)time(NULL);
cJSON *doc = cJSON_CreateObject();
cJSON_AddNumberToObject(doc, "fetched_at", (double)now);
if (api_base)
cJSON_AddStringToObject(doc, "api_base", api_base);
cJSON *arr = cJSON_AddArrayToObject(doc, "models");
for (int i = 0; i < n; i++) {
const cst_model_meta_t *m = &list[i];
cJSON *o = cJSON_CreateObject();
cJSON_AddStringToObject(o, "id", m->id);
if (m->context > 0)
cJSON_AddNumberToObject(o, "context", (double)m->context);
if (m->vision >= 0)
cJSON_AddBoolToObject(o, "vision", m->vision);
if (m->has_price) {
cJSON *p = cJSON_AddObjectToObject(o, "price");
cJSON_AddStringToObject(p, "currency", "$");
cJSON_AddNumberToObject(p, "miss", m->miss);
cJSON_AddNumberToObject(p, "hit", m->hit);
cJSON_AddNumberToObject(p, "write", m->write);
cJSON_AddNumberToObject(p, "out", m->out);
}
cJSON_AddItemToArray(arr, o);
}
char *txt = cJSON_Print(doc);
cJSON_Delete(doc);
if (!txt)
return -1;
char tmp[1200];
snprintf(tmp, sizeof(tmp), "%s.tmp", path);
FILE *f = fopen(tmp, "wb");
int rc = -1;
if (f) {
size_t len = strlen(txt);
rc = fwrite(txt, 1, len, f) == len ? 0 : -1;
if (fclose(f) != 0)
rc = -1;
if (rc == 0)
rc = os_rename_replace(tmp, path);
if (rc != 0)
remove(tmp);
}
free(txt);
if (rc == 0) {
pthread_mutex_lock(&g_mu);
ensure_locked();
drop_cache(provider);
load_cache(provider);
pthread_mutex_unlock(&g_mu);
}
return rc;
}
int cst_models_cache_load(const char *provider, cst_model_meta_t *out, int max,
long *age_s) {
if (age_s)
*age_s = 0;
if (!provider)
return -1;
pthread_mutex_lock(&g_mu);
ensure_locked();
int n = 0, found = 0;
for (int i = 0; i < R.ncache; i++) {
const cached_t *c = &R.cache[i];
if (strcmp(c->provider, provider) != 0)
continue;
if (!found && age_s)
*age_s = (long)time(NULL) - c->fetched_at;
found = 1;
if (out && n < max)
out[n++] = c->m;
}
pthread_mutex_unlock(&g_mu);
return found ? n : -1;
}
+25 -123
View File
@@ -1,113 +1,37 @@
#ifndef _DEFAULT_SOURCE
#define _DEFAULT_SOURCE
#endif
#include "cst_agent.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define MODEL_MAX_LEN 128
/* Provider and model data live in model_registry.c (cst_models.h). */
static const cst_provider_t PROVIDERS[] = {
{"minimax", "https://api.minimax.cn/v1", "MiniMax-M2.7", "MINIMAX_API_KEY"},
{"deepseek", "https://api.deepseek.com/v1", "deepseek-flash",
"DEEPSEEK_API_KEY"},
{"openai", "https://api.openai.com/v1", "gpt-4.1-mini", "OPENAI_API_KEY"},
};
const cst_provider_t *cst_provider_find(const char *name) {
if (!name || !name[0])
return NULL;
for (size_t i = 0; i < sizeof(PROVIDERS) / sizeof(PROVIDERS[0]); i++) {
if (strcmp(PROVIDERS[i].name, name) == 0)
return &PROVIDERS[i];
}
return NULL;
}
const cst_provider_t *cst_provider_default(void) {
const cst_provider_t *p = cst_provider_find(getenv("CST_PROVIDER"));
return p ? p : &PROVIDERS[0];
}
static int already(char out[][MODEL_MAX_LEN], int n, const char *name) {
for (int i = 0; i < n; i++) {
if (strcmp(out[i], name) == 0)
return 1;
}
return 0;
}
static int push(char out[][MODEL_MAX_LEN], int *n, int max, const char *name) {
if (!name || !name[0] || *n >= max)
int cst_price_is_peak(time_t when) {
time_t bj = when + 8 * 3600;
struct tm tmv;
#ifdef _WIN32
gmtime_s(&tmv, &bj);
#else
gmtime_r(&bj, &tmv);
#endif
if (tmv.tm_wday == 0 || tmv.tm_wday == 6)
return 0;
if (already(out, *n, name))
return 0;
snprintf(out[*n], MODEL_MAX_LEN, "%s", name);
(*n)++;
return 1;
return (tmv.tm_hour >= 9 && tmv.tm_hour < 12) ||
(tmv.tm_hour >= 14 && tmv.tm_hour < 18);
}
int cst_models_list(const char *current, char out[][128], int max_out) {
if (!out || max_out <= 0)
double cst_price_cost(const cst_price_t *p, long miss, long hit, long write,
long out, time_t when) {
if (!p)
return 0;
int n = 0;
const char *env = getenv("CST_MODELS");
if (env && env[0]) {
char buf[1024];
snprintf(buf, sizeof(buf), "%s", env);
char *p = buf;
while (*p && n < max_out) {
while (*p == ':' || *p == ' ' || *p == ',')
p++;
if (!*p)
break;
char *start = p;
while (*p && *p != ':' && *p != ',')
p++;
char save = *p;
*p = 0;
char *end = start + strlen(start);
while (end > start && (end[-1] == ' ' || end[-1] == '\t'))
*--end = 0;
push(out, &n, max_out, start);
*p = save;
if (*p)
p++;
}
} else {
push(out, &n, max_out, "deepseek-flash");
push(out, &n, max_out, "deepseek-v4-pro");
push(out, &n, max_out, "MiniMax-M2.7");
push(out, &n, max_out, "MiniMax-M2.5");
push(out, &n, max_out, "gpt-4.1-mini");
push(out, &n, max_out, "gpt-4o");
}
if (current && current[0])
push(out, &n, max_out, current);
return n;
}
/* Context window in tokens (CST_CTX_TOKENS overrides). Prefix match. */
long cst_model_context_tokens(const char *model) {
const char *e = getenv("CST_CTX_TOKENS");
if (e && atol(e) > 0)
return atol(e);
static const struct {
const char *prefix;
long tokens;
} T[] = {{"deepseek", 128000}, {"MiniMax", 196608}, {"minimax", 196608},
{"gpt-4.1", 1047576}, {"gpt-4o", 128000}, {"gpt-5", 400000},
{"o3", 200000}, {"o4", 200000}, {"claude", 200000},
{"qwen", 131072}, {"glm", 128000}, {"kimi", 131072}};
if (model) {
for (size_t i = 0; i < sizeof(T) / sizeof(T[0]); i++) {
if (strncmp(model, T[i].prefix, strlen(T[i].prefix)) == 0)
return T[i].tokens;
}
}
return 128000;
double c = (miss * p->miss + hit * p->hit + write * p->write +
out * p->out) / 1e6;
if (p->offpeak_half && !cst_price_is_peak(when))
c /= 2;
return c;
}
int cst_effort_valid(const char *effort) {
@@ -118,25 +42,3 @@ int cst_effort_valid(const char *effort) {
}
return 0;
}
const char *cst_effort_wire(const char *provider, const char *model,
const char *effort) {
if (!cst_effort_valid(effort) || !provider || !model)
return NULL;
if (strcmp(provider, "deepseek") == 0)
return effort;
/* OpenAI reasoning models take low | medium | high. */
if (strcmp(provider, "openai") == 0 &&
(strncmp(model, "o1", 2) == 0 || strncmp(model, "o3", 2) == 0 ||
strncmp(model, "o4", 2) == 0 || strncmp(model, "gpt-5", 5) == 0))
return strcmp(effort, "max") == 0 ? "high" : effort;
return NULL;
}
int cst_provider_count(void) {
return (int)(sizeof(PROVIDERS) / sizeof(PROVIDERS[0]));
}
const cst_provider_t *cst_provider_at(int i) {
return i >= 0 && i < cst_provider_count() ? &PROVIDERS[i] : NULL;
}
+73
View File
@@ -0,0 +1,73 @@
{
"providers": {
"minimax": {
"api_base": "https://api.minimax.cn/v1",
"key_env": "MINIMAX_API_KEY",
"default_model": "MiniMax-M2.7",
"models": ["MiniMax-M2.7", "MiniMax-M2.5"]
},
"deepseek": {
"api_base": "https://api.deepseek.com/v1",
"key_env": "DEEPSEEK_API_KEY",
"default_model": "deepseek-flash",
"models": ["deepseek-flash", "deepseek-v4-pro"],
"reasoning_passback": true,
"prefix_completion": true,
"effort": "passthrough"
},
"openai": {
"api_base": "https://api.openai.com/v1",
"key_env": "OPENAI_API_KEY",
"default_model": "gpt-4.1-mini",
"models": ["gpt-4.1-mini", "gpt-4o"],
"effort": "openai"
},
"teamorouter": {
"api_base": "https://api.teamorouter.com/v1",
"key_env": "TEAMOROUTER_API_KEY",
"default_model": ""
}
},
"models": [
{"match": "deepseek", "context": 1000000, "vision": false,
"price": {"currency": "¥", "miss": 2, "hit": 0.04, "out": 8, "offpeak_half": true}},
{"match": "deepseek-flash", "vision": true},
{"match": "deepseek-chat", "vision": true},
{"match": "deepseek-reasoner", "vision": true},
{"match": "deepseek-v4-pro",
"price": {"currency": "¥", "miss": 9, "hit": 0.3, "out": 27, "offpeak_half": true}},
{"match": "deepseek-vl", "vision": true},
{"match": "minimax", "context": 196608, "vision": false},
{"match": "gpt-", "cache": "openai"},
{"match": "gpt-4.1", "context": 1047576, "vision": true,
"price": {"currency": "$", "miss": 2, "hit": 0.5, "out": 8}},
{"match": "gpt-4.1-mini",
"price": {"currency": "$", "miss": 0.4, "hit": 0.1, "out": 1.6}},
{"match": "gpt-4o", "context": 128000, "vision": true,
"price": {"currency": "$", "miss": 2.5, "hit": 1.25, "out": 10}},
{"match": "gpt-4o-mini",
"price": {"currency": "$", "miss": 0.15, "hit": 0.075, "out": 0.6}},
{"match": "gpt-5", "context": 400000, "vision": true, "reasoning": true},
{"match": "o1", "vision": true, "cache": "openai", "reasoning": true},
{"match": "o3", "context": 200000, "vision": true, "cache": "openai", "reasoning": true},
{"match": "o4", "context": 200000, "vision": true, "cache": "openai", "reasoning": true},
{"match": "anthropic/", "cache": "anthropic"},
{"match": "claude", "context": 200000, "vision": true, "cache": "anthropic"},
{"match": "claude-sonnet",
"price": {"currency": "$", "miss": 3, "hit": 0.3, "write": 3.75, "out": 15}},
{"match": "gemini", "context": 1048576, "vision": true},
{"match": "grok", "context": 256000},
{"match": "grok-4", "vision": true},
{"match": "qwen", "context": 131072},
{"match": "glm", "context": 128000},
{"match": "glm-*v*", "vision": true},
{"match": "kimi", "context": 131072},
{"match": "pixtral", "vision": true},
{"match": "llava", "vision": true},
{"match": "*-vl*", "vision": true},
{"match": "*vision*", "vision": true}
]
}
+334
View File
@@ -0,0 +1,334 @@
#ifndef _DEFAULT_SOURCE
#define _DEFAULT_SOURCE
#endif
#include "cst_net.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#if !defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0600
#undef _WIN32_WINNT
#define _WIN32_WINNT 0x0600 /* WSAPoll */
#endif
#include <winsock2.h>
#include <ws2tcpip.h>
#else
#include <arpa/inet.h>
#include <errno.h>
#include <fcntl.h>
#include <netdb.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <poll.h>
#include <signal.h>
#include <sys/socket.h>
#include <unistd.h>
#endif
#ifndef MSG_NOSIGNAL
#define MSG_NOSIGNAL 0
#endif
#define NET_SEND_TIMEOUT_MS 20000
#ifdef _WIN32
static void set_err_win(char *err, size_t err_sz, const char *what) {
if (err && err_sz)
snprintf(err, err_sz, "%s (winsock error %d)", what, WSAGetLastError());
}
#define SET_ERR(e, n, w) set_err_win(e, n, w)
#define NET_FD(s) ((SOCKET)(s))
static void net_close_raw(os_socket_t s) { closesocket((SOCKET)s); }
static int net_last_would_block(void) {
int e = WSAGetLastError();
return e == WSAEWOULDBLOCK || e == WSAEINTR;
}
static int net_last_conn_reset(void) {
int e = WSAGetLastError();
return e == WSAECONNRESET || e == WSAECONNABORTED || e == WSAENOTSOCK;
}
#else
static void set_err(char *err, size_t err_sz, const char *what) {
if (err && err_sz)
snprintf(err, err_sz, "%s", what);
}
#define SET_ERR(e, n, w) set_err(e, n, w)
#define NET_FD(s) ((int)(s))
static void net_close_raw(os_socket_t s) { close(s); }
static int net_last_would_block(void) { return errno == EINTR; }
static int net_last_conn_reset(void) {
return errno == ECONNRESET || errno == EPIPE || errno == EBADF;
}
#endif
int os_net_init(void) {
#ifdef _WIN32
WSADATA w;
if (WSAStartup(MAKEWORD(2, 2), &w) != 0)
return -1;
#else
/* A hung-up client must not kill the process with SIGPIPE. */
signal(SIGPIPE, SIG_IGN);
#endif
return 0;
}
void os_net_cleanup(void) {
#ifdef _WIN32
WSACleanup();
#endif
}
/* poll, not select: FD_SET past FD_SETSIZE writes out of bounds, and a
* failed connect shows up as an error event, not as writable, on Windows. */
static int wait_io(os_socket_t s, int want_write, int timeout_ms) {
#ifdef _WIN32
WSAPOLLFD p;
p.fd = NET_FD(s);
p.events = want_write ? POLLWRNORM : POLLRDNORM;
p.revents = 0;
int rc = WSAPoll(&p, 1, timeout_ms < 0 ? -1 : timeout_ms);
#else
struct pollfd p;
p.fd = NET_FD(s);
p.events = want_write ? POLLOUT : POLLIN;
p.revents = 0;
int rc = poll(&p, 1, timeout_ms < 0 ? -1 : timeout_ms);
#endif
if (rc > 0)
return 1;
if (rc == 0)
return 0; /* timed out */
return -1;
}
/* Host to bind: AF_INET unless the name is an IPv6 literal, so "localhost"
* and "127.0.0.1" always mean the IPv4 loopback a browser will reach. */
static int resolve_bind(const char *host, int port, struct sockaddr_storage *sa,
socklen_t *sa_len) {
memset(sa, 0, sizeof(*sa));
int family = AF_INET;
const char *h = host && host[0] ? host : "127.0.0.1";
if (strchr(h, ':'))
family = AF_INET6;
struct addrinfo hints, *res = NULL;
memset(&hints, 0, sizeof(hints));
hints.ai_family = family;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE;
char portstr[16];
snprintf(portstr, sizeof(portstr), "%d", port);
if (getaddrinfo(h, portstr, &hints, &res) != 0 || !res)
return -1;
memcpy(sa, res->ai_addr, res->ai_addrlen);
*sa_len = (socklen_t)res->ai_addrlen;
freeaddrinfo(res);
return 0;
}
os_socket_t os_net_listen(const char *host, int port, char *err, size_t err_sz) {
struct sockaddr_storage sa;
socklen_t sa_len;
if (resolve_bind(host, port, &sa, &sa_len) != 0) {
SET_ERR(err, err_sz, "cannot resolve the bind address");
return OS_SOCKET_BAD;
}
os_socket_t s = (os_socket_t)socket(sa.ss_family, SOCK_STREAM, 0);
if (s == OS_SOCKET_BAD) {
SET_ERR(err, err_sz, "cannot create a socket");
return OS_SOCKET_BAD;
}
int one = 1;
#ifdef _WIN32
/* SO_REUSEADDR on Windows lets another process bind the same port and
* take over our connections */
setsockopt(NET_FD(s), SOL_SOCKET, SO_EXCLUSIVEADDRUSE, (const char *)&one,
sizeof(one));
#else
setsockopt(NET_FD(s), SOL_SOCKET, SO_REUSEADDR, (const char *)&one,
sizeof(one));
#endif
#ifdef IPV6_V6ONLY
if (sa.ss_family == AF_INET6)
setsockopt(NET_FD(s), IPPROTO_IPV6, IPV6_V6ONLY, (const char *)&one,
sizeof(one));
#endif
if (bind(NET_FD(s), (const struct sockaddr *)&sa, sa_len) != 0) {
SET_ERR(err, err_sz, "the address is already in use");
net_close_raw(s);
return OS_SOCKET_BAD;
}
if (listen(NET_FD(s), 32) != 0) {
SET_ERR(err, err_sz, "listen failed");
net_close_raw(s);
return OS_SOCKET_BAD;
}
return s;
}
int os_net_port(os_socket_t s) {
struct sockaddr_storage sa;
socklen_t len = sizeof(sa);
if (getsockname(NET_FD(s), (struct sockaddr *)&sa, &len) != 0)
return 0;
if (sa.ss_family == AF_INET)
return ntohs(((struct sockaddr_in *)&sa)->sin_port);
if (sa.ss_family == AF_INET6)
return ntohs(((struct sockaddr_in6 *)&sa)->sin6_port);
return 0;
}
void os_net_peer(os_socket_t s, char *out, size_t out_sz) {
if (!out || out_sz == 0)
return;
out[0] = 0;
struct sockaddr_storage sa;
socklen_t len = sizeof(sa);
if (getpeername(NET_FD(s), (struct sockaddr *)&sa, &len) != 0)
return;
char ip[64] = "";
int port = 0;
if (sa.ss_family == AF_INET) {
if (!inet_ntop(AF_INET, &((struct sockaddr_in *)&sa)->sin_addr, ip,
sizeof(ip)))
return;
port = ntohs(((struct sockaddr_in *)&sa)->sin_port);
} else if (sa.ss_family == AF_INET6) {
if (!inet_ntop(AF_INET6, &((struct sockaddr_in6 *)&sa)->sin6_addr, ip,
sizeof(ip)))
return;
port = ntohs(((struct sockaddr_in6 *)&sa)->sin6_port);
} else {
return;
}
if (strchr(ip, ':'))
snprintf(out, out_sz, "[%s]:%d", ip, port);
else
snprintf(out, out_sz, "%s:%d", ip, port);
}
os_socket_t os_net_connect(const char *host, int port, int timeout_ms) {
const char *h = host && host[0] ? host : "127.0.0.1";
struct addrinfo hints, *res = NULL;
memset(&hints, 0, sizeof(hints));
hints.ai_family = strchr(h, ':') ? AF_INET6 : AF_INET;
hints.ai_socktype = SOCK_STREAM;
char portstr[16];
snprintf(portstr, sizeof(portstr), "%d", port);
if (getaddrinfo(h, portstr, &hints, &res) != 0 || !res)
return OS_SOCKET_BAD;
os_socket_t s = (os_socket_t)socket(res->ai_family, SOCK_STREAM, 0);
if (s == OS_SOCKET_BAD) {
freeaddrinfo(res);
return OS_SOCKET_BAD;
}
/* Bounded wait: connect() non-blocking, poll, then read SO_ERROR. A dead
* server must not hang the caller. */
#ifdef _WIN32
u_long nb = 1;
ioctlsocket(NET_FD(s), FIONBIO, &nb);
#else
int fl = fcntl(s, F_GETFL, 0);
fcntl(s, F_SETFL, fl | O_NONBLOCK);
#endif
int rc = connect(NET_FD(s), res->ai_addr, (int)res->ai_addrlen);
freeaddrinfo(res);
if (rc != 0) {
#ifdef _WIN32
int in_progress = WSAGetLastError() == WSAEWOULDBLOCK;
#else
int in_progress = errno == EINPROGRESS || errno == EINTR;
#endif
if (!in_progress || wait_io(s, 1, timeout_ms) <= 0) {
net_close_raw(s);
return OS_SOCKET_BAD;
}
int soerr = 0;
socklen_t elen = sizeof(soerr);
if (getsockopt(NET_FD(s), SOL_SOCKET, SO_ERROR, (char *)&soerr, &elen) != 0 ||
soerr != 0) {
net_close_raw(s);
return OS_SOCKET_BAD;
}
}
#ifdef _WIN32
nb = 0;
ioctlsocket(NET_FD(s), FIONBIO, &nb);
#else
fcntl(s, F_SETFL, fl);
#endif
return s;
}
os_socket_t os_net_accept(os_socket_t s, int timeout_ms, int *timed_out) {
if (timed_out)
*timed_out = 0;
int w = wait_io(s, 0, timeout_ms);
if (w == 0) {
if (timed_out)
*timed_out = 1;
return OS_SOCKET_BAD;
}
if (w < 0)
return OS_SOCKET_BAD;
os_socket_t c = (os_socket_t)accept(NET_FD(s), NULL, NULL);
if (c == OS_SOCKET_BAD)
return OS_SOCKET_BAD;
int one = 1;
setsockopt(NET_FD(c), IPPROTO_TCP, TCP_NODELAY, (const char *)&one,
sizeof(one));
return c;
}
long os_net_recv(os_socket_t s, void *buf, size_t n, int timeout_ms) {
int w = wait_io(s, 0, timeout_ms);
if (w == 0)
return -2;
if (w < 0)
return -1;
size_t want = n > (size_t)(1 << 20) ? (size_t)(1 << 20) : n;
int r = (int)recv(NET_FD(s), (char *)buf, (int)want, 0);
if (r == 0)
return 0;
if (r < 0)
return net_last_conn_reset() ? 0 : (net_last_would_block() ? -2 : -1);
return r;
}
int os_net_send_all(os_socket_t s, const void *buf, size_t n, int timeout_ms) {
const char *p = buf;
size_t off = 0;
while (off < n) {
int w = wait_io(s, 1, timeout_ms ? timeout_ms : NET_SEND_TIMEOUT_MS);
if (w <= 0)
return -1;
long r = (long)send(NET_FD(s), p + off, (int)(n - off), MSG_NOSIGNAL);
if (r <= 0) {
if (net_last_would_block())
continue;
return -1;
}
off += (size_t)r;
}
return 0;
}
void os_net_shutdown(os_socket_t s) {
#ifdef _WIN32
shutdown((SOCKET)s, SD_BOTH);
#else
shutdown(s, SHUT_RDWR);
#endif
}
void os_net_close(os_socket_t s) {
if (s == OS_SOCKET_BAD)
return;
net_close_raw(s);
}
+361 -361
View File
@@ -10,8 +10,10 @@
#include <stdint.h>
#endif
#include "os_compat.h"
#include "os_internal.h"
#include <errno.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -26,6 +28,7 @@
#include <process.h>
#include <windows.h>
#else
#include <dirent.h>
#include <fcntl.h>
#include <poll.h>
#include <pthread.h>
@@ -93,26 +96,10 @@ int os_path_join(char *out, size_t out_sz, const char *a, const char *b) {
return 0;
}
#ifndef _WIN32
int os_realpath(const char *path, char *out, size_t out_sz) {
if (!path || !out || out_sz < 2)
return -1;
#ifdef _WIN32
char buf[OS_PATH_MAX];
DWORD n = GetFullPathNameA(path, (DWORD)sizeof(buf), buf, NULL);
if (n == 0 || n >= sizeof(buf))
return -1;
/* Prefer forward slashes internally for sandbox compares. */
os_path_norm_slashes(buf);
if (strlen(buf) + 1 > out_sz)
return -1;
memcpy(out, buf, strlen(buf) + 1);
/* Lowercase drive letter for stable compares */
if (out[0] && out[1] == ':') {
if (out[0] >= 'A' && out[0] <= 'Z')
out[0] = (char)(out[0] - 'A' + 'a');
}
return 0;
#else
char *r = realpath(path, NULL);
if (!r)
return -1;
@@ -123,7 +110,6 @@ int os_realpath(const char *path, char *out, size_t out_sz) {
memcpy(out, r, strlen(r) + 1);
free(r);
return 0;
#endif
}
int os_path_under(const char *root, const char *path) {
@@ -132,14 +118,9 @@ int os_path_under(const char *root, const char *path) {
size_t rl = strlen(root);
if (strncmp(path, root, rl) != 0)
return 0;
if (path[rl] == '\0')
return 1;
#ifdef _WIN32
return path[rl] == '/' || path[rl] == '\\';
#else
return path[rl] == '/';
#endif
return path[rl] == '\0' || path[rl] == '/';
}
#endif
int os_mkdir(const char *path, int mode) {
if (!path)
@@ -172,16 +153,93 @@ int os_mkdir_p(const char *path, int mode) {
return os_mkdir(tmp, mode);
}
#ifndef _WIN32
struct os_dir {
DIR *d;
char path[OS_PATH_MAX];
os_dirent_t ent;
};
os_dir_t *os_dir_open(const char *path) {
if (!path)
return NULL;
os_dir_t *d = calloc(1, sizeof(*d));
if (!d)
return NULL;
snprintf(d->path, sizeof(d->path), "%s", path);
d->d = opendir(path);
if (!d->d) {
free(d);
return NULL;
}
return d;
}
const os_dirent_t *os_dir_next(os_dir_t *d) {
struct dirent *de;
while (d && (de = readdir(d->d)) != NULL) {
if (strcmp(de->d_name, ".") == 0 || strcmp(de->d_name, "..") == 0)
continue;
d->ent.name = de->d_name;
d->ent.is_dir = d->ent.is_link = 0;
#ifdef DT_DIR
if (de->d_type != DT_UNKNOWN) {
d->ent.is_dir = de->d_type == DT_DIR;
d->ent.is_link = de->d_type == DT_LNK;
return &d->ent;
}
#endif
char p[OS_PATH_MAX];
struct stat st;
if (snprintf(p, sizeof(p), "%s/%s", d->path, de->d_name) <
(int)sizeof(p) &&
lstat(p, &st) == 0) {
d->ent.is_dir = S_ISDIR(st.st_mode);
d->ent.is_link = S_ISLNK(st.st_mode);
}
return &d->ent;
}
return NULL;
}
void os_dir_close(os_dir_t *d) {
if (!d)
return;
closedir(d->d);
free(d);
}
int os_lstat(const char *path, os_stat_t *st) {
struct stat s;
memset(st, 0, sizeof(*st));
if (!path || lstat(path, &s) != 0)
return -1;
st->is_dir = S_ISDIR(s.st_mode);
st->is_reg = S_ISREG(s.st_mode);
st->is_link = S_ISLNK(s.st_mode);
st->size = (long long)s.st_size;
st->mtime = (double)s.st_mtime;
return 0;
}
int os_rename_replace(const char *from, const char *to) {
return rename(from, to) == 0 ? 0 : -1;
}
#endif
int os_config_dir(char *out, size_t out_sz) {
if (!out || out_sz < 8)
return -1;
#ifdef _WIN32
const char *base = getenv("APPDATA");
const char *base = getenv("XDG_CONFIG_HOME");
if (!base || !os_path_is_abs(base))
base = getenv("APPDATA");
if (!base || !base[0])
base = getenv("LOCALAPPDATA");
if (!base || !base[0])
return -1;
int n = snprintf(out, out_sz, "%s\\cst-agent", base);
int n = snprintf(out, out_sz, "%s/cst-agent", base);
os_path_norm_slashes(out);
#else
const char *xdg = getenv("XDG_CONFIG_HOME");
const char *home = getenv("HOME");
@@ -198,6 +256,66 @@ int os_config_dir(char *out, size_t out_sz) {
return 0;
}
#ifndef _WIN32
/* $TMPDIR is not always usable: Termux has no /tmp unless TMPDIR points at
* $PREFIX/tmp, and Android may deny /tmp or /data/local/tmp outright. Take
* the first candidate where a file can actually be created. */
static char g_tmp[OS_PATH_MAX];
static pthread_once_t g_tmp_once = PTHREAD_ONCE_INIT;
static int tmp_writable(const char *dir) {
char probe[OS_PATH_MAX];
struct stat st;
if (!dir || dir[0] != '/' || stat(dir, &st) != 0 || !S_ISDIR(st.st_mode))
return 0;
int n = snprintf(probe, sizeof(probe), "%s/.cst-probe-XXXXXX", dir);
if (n < 0 || (size_t)n >= sizeof(probe))
return 0;
int fd = mkstemp(probe);
if (fd < 0)
return 0;
close(fd);
unlink(probe);
return 1;
}
static void tmp_trim(char *s) {
size_t n = strlen(s);
while (n > 1 && s[n - 1] == '/')
s[--n] = 0;
}
static void tmp_pick(void) {
char cand[5][OS_PATH_MAX];
int nc = 0;
const char *env = getenv("TMPDIR");
const char *prefix = getenv("PREFIX");
const char *home = getenv("HOME");
const char *cache = getenv("XDG_CACHE_HOME");
if (env && env[0])
snprintf(cand[nc++], OS_PATH_MAX, "%s", env);
if (prefix && prefix[0] == '/')
snprintf(cand[nc++], OS_PATH_MAX, "%s/tmp", prefix);
snprintf(cand[nc++], OS_PATH_MAX, "/tmp");
snprintf(cand[nc++], OS_PATH_MAX, "/var/tmp");
if (cache && cache[0] == '/')
snprintf(cand[nc++], OS_PATH_MAX, "%s/cst-agent/tmp", cache);
else if (home && home[0] == '/')
snprintf(cand[nc++], OS_PATH_MAX, "%s/.cache/cst-agent/tmp", home);
for (int i = 0; i < nc; i++) {
tmp_trim(cand[i]);
if (i == nc - 1)
os_mkdir_p(cand[i], 0700);
if (tmp_writable(cand[i])) {
memcpy(g_tmp, cand[i], strlen(cand[i]) + 1);
return;
}
}
/* nothing works: keep the old answer so callers fail as before */
memcpy(g_tmp, cand[0], strlen(cand[0]) + 1);
}
#endif
int os_temp_dir(char *out, size_t out_sz) {
if (!out || out_sz < 2)
return -1;
@@ -211,11 +329,18 @@ int os_temp_dir(char *out, size_t out_sz) {
out[--L] = 0;
return 0;
#else
pthread_once(&g_tmp_once, tmp_pick);
int n = snprintf(out, out_sz, "%s", g_tmp);
return n < 0 || (size_t)n >= out_sz ? -1 : 0;
#endif
}
void os_temp_dir_export(void) {
#ifndef _WIN32
char d[OS_PATH_MAX];
const char *t = getenv("TMPDIR");
if (!t || !t[0])
t = "/tmp";
snprintf(out, out_sz, "%s", t);
return 0;
if (os_temp_dir(d, sizeof(d)) == 0 && (!t || strcmp(t, d) != 0))
setenv("TMPDIR", d, 1);
#endif
}
@@ -379,23 +504,78 @@ int os_resolve_shell(char *out, size_t out_sz) {
#endif
}
/* Per thread: parallel tool runs and sub-agents each watch their own flag. */
static _Thread_local volatile int *g_run_cancel;
static _Thread_local int g_run_timeout_s;
/* Per thread: parallel tool runs and sub-agents each watch their own flag.
* Thread keys rather than _Thread_local, which tcc does not implement; each
* value fits in the key's pointer slot, so nothing is allocated. */
#ifdef _WIN32
static INIT_ONCE g_run_once = INIT_ONCE_STATIC_INIT;
static DWORD g_key_cancel = TLS_OUT_OF_INDEXES, g_key_timeout = TLS_OUT_OF_INDEXES;
static BOOL CALLBACK run_keys_init(PINIT_ONCE once, PVOID param, PVOID *ctx) {
(void)once;
(void)param;
(void)ctx;
g_key_cancel = TlsAlloc();
g_key_timeout = TlsAlloc();
return TRUE;
}
static void *run_key_get(DWORD *key) {
InitOnceExecuteOnce(&g_run_once, run_keys_init, NULL, NULL);
if (*key == TLS_OUT_OF_INDEXES)
return NULL;
DWORD err = GetLastError(); /* TlsGetValue clears it */
void *v = TlsGetValue(*key);
SetLastError(err);
return v;
}
static void run_key_set(DWORD *key, void *v) {
InitOnceExecuteOnce(&g_run_once, run_keys_init, NULL, NULL);
if (*key != TLS_OUT_OF_INDEXES)
TlsSetValue(*key, v);
}
#else
static pthread_once_t g_run_once = PTHREAD_ONCE_INIT;
static pthread_key_t g_key_cancel, g_key_timeout;
static int g_run_keys_ok;
static void run_keys_init(void) {
g_run_keys_ok = pthread_key_create(&g_key_cancel, NULL) == 0 &&
pthread_key_create(&g_key_timeout, NULL) == 0;
}
static void *run_key_get(pthread_key_t *key) {
pthread_once(&g_run_once, run_keys_init);
return g_run_keys_ok ? pthread_getspecific(*key) : NULL;
}
static void run_key_set(pthread_key_t *key, void *v) {
pthread_once(&g_run_once, run_keys_init);
if (g_run_keys_ok)
pthread_setspecific(*key, v);
}
#endif
void os_set_cancel_flag(volatile int *flag) { g_run_cancel = flag; }
static volatile int *run_cancel(void) {
return (volatile int *)run_key_get(&g_key_cancel);
}
static int run_timeout(void) {
return (int)(intptr_t)run_key_get(&g_key_timeout);
}
void os_set_run_timeout(int seconds) { g_run_timeout_s = seconds; }
void os_set_cancel_flag(volatile int *flag) {
run_key_set(&g_key_cancel, (void *)(uintptr_t)flag);
}
int os_get_run_timeout(void) { return g_run_timeout_s; }
void os_set_run_timeout(int seconds) {
run_key_set(&g_key_timeout, (void *)(intptr_t)seconds);
}
static int run_timeout_s(void) {
if (g_run_timeout_s > 0)
return g_run_timeout_s;
int os_get_run_timeout(void) { return run_timeout(); }
int os_run_timeout_effective(void) {
int t = run_timeout();
if (t > 0)
return t;
const char *e = getenv("CST_SHELL_TIMEOUT");
if (e && atoi(e) > 0)
return atoi(e);
return 120;
return 0; /* no limit: a build, a test suite or a long clone must finish */
}
double os_now_ms(void) {
@@ -414,8 +594,8 @@ double os_now_ms(void) {
}
/* Append to capture buffer, keeping at most max_out bytes. */
static int cap_append(char **buf, size_t *len, size_t *cap, size_t max_out,
const char *data, size_t n, int *truncated) {
int os_cap_append(char **buf, size_t *len, size_t *cap, size_t max_out,
const char *data, size_t n, int *truncated) {
if (*len >= max_out) {
*truncated = 1;
return 0;
@@ -440,7 +620,7 @@ static int cap_append(char **buf, size_t *len, size_t *cap, size_t max_out,
return 0;
}
static void cap_note(char **buf, size_t *len, size_t *cap, const char *note) {
void os_cap_note(char **buf, size_t *len, size_t *cap, const char *note) {
size_t n = strlen(note);
if (*len + n + 1 > *cap) {
char *nb = realloc(*buf, *len + n + 1);
@@ -453,154 +633,7 @@ static void cap_note(char **buf, size_t *len, size_t *cap, const char *note) {
*len += n;
}
#ifdef _WIN32
int os_run_capture(const char *cwd, const char *const argv[], size_t max_out,
os_run_result_t *r) {
if (!argv || !argv[0] || !r)
return -1;
if (max_out == 0)
max_out = 512 * 1024;
memset(r, 0, sizeof(*r));
char cmdline[8192];
size_t pos = 0;
cmdline[0] = 0;
for (int i = 0; argv[i]; i++) {
const char *a = argv[i];
int need_q = strchr(a, ' ') || strchr(a, '\t') || strchr(a, '"');
if (i && pos + 1 < sizeof(cmdline))
cmdline[pos++] = ' ';
if (need_q) {
if (pos + 1 < sizeof(cmdline))
cmdline[pos++] = '"';
for (const char *p = a; *p && pos + 2 < sizeof(cmdline); p++) {
if (*p == '"') {
cmdline[pos++] = '\\';
cmdline[pos++] = '"';
} else {
cmdline[pos++] = *p;
}
}
if (pos + 1 < sizeof(cmdline))
cmdline[pos++] = '"';
} else {
size_t L = strlen(a);
if (pos + L >= sizeof(cmdline))
break;
memcpy(cmdline + pos, a, L);
pos += L;
}
cmdline[pos] = 0;
}
SECURITY_ATTRIBUTES sa = {0};
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
HANDLE rd = NULL, wr = NULL;
if (!CreatePipe(&rd, &wr, &sa, 0))
return -1;
SetHandleInformation(rd, HANDLE_FLAG_INHERIT, 0);
STARTUPINFOA si = {0};
si.cb = sizeof(si);
si.dwFlags = STARTF_USESTDHANDLES;
si.hStdOutput = wr;
si.hStdError = wr;
HANDLE nul = CreateFileA("NUL", GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE,
&sa, OPEN_EXISTING, 0, NULL);
si.hStdInput = nul != INVALID_HANDLE_VALUE ? nul : GetStdHandle(STD_INPUT_HANDLE);
PROCESS_INFORMATION pi = {0};
char *cmd_mut = _strdup(cmdline);
if (!cmd_mut) {
CloseHandle(rd);
CloseHandle(wr);
return -1;
}
BOOL ok = CreateProcessA(NULL, cmd_mut, NULL, NULL, TRUE, 0, NULL,
cwd && cwd[0] ? cwd : NULL, &si, &pi);
free(cmd_mut);
CloseHandle(wr);
if (nul != INVALID_HANDLE_VALUE)
CloseHandle(nul);
if (!ok) {
CloseHandle(rd);
return -1;
}
size_t cap = 4096, len = 0;
char *buf = malloc(cap);
if (!buf) {
CloseHandle(rd);
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
return -1;
}
buf[0] = 0;
int truncated = 0, killed = 0;
double deadline = os_now_ms() + run_timeout_s() * 1000.0;
for (;;) {
DWORD avail = 0;
if (!PeekNamedPipe(rd, NULL, 0, NULL, &avail, NULL))
break; /* writer closed */
if (avail > 0) {
char tmp[4096];
DWORD n = 0;
if (!ReadFile(rd, tmp, sizeof(tmp), &n, NULL) || n == 0)
break;
cap_append(&buf, &len, &cap, max_out, tmp, n, &truncated);
continue;
}
if (WaitForSingleObject(pi.hProcess, 50) == WAIT_OBJECT_0) {
/* drain what is left */
while (PeekNamedPipe(rd, NULL, 0, NULL, &avail, NULL) && avail > 0) {
char tmp[4096];
DWORD n = 0;
if (!ReadFile(rd, tmp, sizeof(tmp), &n, NULL) || n == 0)
break;
cap_append(&buf, &len, &cap, max_out, tmp, n, &truncated);
}
break;
}
if ((g_run_cancel && *g_run_cancel) || os_now_ms() > deadline) {
killed = (g_run_cancel && *g_run_cancel) ? 2 : 1;
TerminateProcess(pi.hProcess, killed == 1 ? 124 : 130);
break;
}
}
CloseHandle(rd);
WaitForSingleObject(pi.hProcess, INFINITE);
DWORD code = 1;
GetExitCodeProcess(pi.hProcess, &code);
if (truncated)
cap_note(&buf, &len, &cap, "\n...(output truncated)\n");
if (killed == 1) {
char note[96];
snprintf(note, sizeof(note), "\n[cst: killed after %ds timeout]\n",
run_timeout_s());
cap_note(&buf, &len, &cap, note);
code = 124;
} else if (killed == 2) {
cap_note(&buf, &len, &cap, "\n[cst: cancelled]\n");
code = 130;
}
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
buf[len] = 0;
r->out = buf;
r->out_len = len;
r->exit_code = (int)code;
return 0;
}
int os_run_capture_ex(const char *cwd, const char *const argv[],
size_t max_out, os_run_result_t *r,
os_child_hook_fn child_hook, void *ud) {
(void)child_hook;
(void)ud;
return os_run_capture(cwd, argv, max_out, r);
}
#else
#ifndef _WIN32
static void cloexec(int fd) {
int fl = fcntl(fd, F_GETFD);
if (fl >= 0)
@@ -685,7 +718,8 @@ int os_run_capture_ex(const char *cwd, const char *const argv[],
int truncated = 0, killed = 0;
int status = 0, reaped = 0, bg_killed = 0;
double reaped_at = 0, killed_at = 0;
double deadline = os_now_ms() + run_timeout_s() * 1000.0;
int timeout_s = os_run_timeout_effective();
double deadline = timeout_s > 0 ? os_now_ms() + timeout_s * 1000.0 : 0;
for (;;) {
struct pollfd pfd = {pipefd[0], POLLIN, 0};
int pr = poll(&pfd, 1, 100);
@@ -697,7 +731,7 @@ int os_run_capture_ex(const char *cwd, const char *const argv[],
if (n == 0)
break; /* every writer closed */
if (n > 0)
cap_append(&buf, &len, &cap, max_out, tmp, (size_t)n, &truncated);
os_cap_append(&buf, &len, &cap, max_out, tmp, (size_t)n, &truncated);
else if (errno != EINTR && errno != EAGAIN)
break;
}
@@ -715,8 +749,8 @@ int os_run_capture_ex(const char *cwd, const char *const argv[],
}
/* Checked on every pass, so a chatty command still stops promptly. */
if (!reaped && !killed &&
((g_run_cancel && *g_run_cancel) || now > deadline)) {
killed = (g_run_cancel && *g_run_cancel) ? 2 : 1;
(os_cancel_requested() || (deadline > 0 && now > deadline))) {
killed = os_cancel_requested() ? 2 : 1;
killed_at = now;
kill(-pid, SIGTERM);
}
@@ -733,9 +767,9 @@ int os_run_capture_ex(const char *cwd, const char *const argv[],
waitpid(pid, &status, 0);
}
if (truncated)
cap_note(&buf, &len, &cap, "\n...(output truncated)\n");
os_cap_note(&buf, &len, &cap, "\n...(output truncated)\n");
if (bg_killed)
cap_note(&buf, &len, &cap,
os_cap_note(&buf, &len, &cap,
"\n[cst: background processes left running were stopped]\n");
r->out = buf;
r->out_len = len;
@@ -743,11 +777,11 @@ int os_run_capture_ex(const char *cwd, const char *const argv[],
if (killed == 1) {
char note[96];
snprintf(note, sizeof(note), "\n[cst: killed after %ds timeout]\n",
run_timeout_s());
cap_note(&r->out, &r->out_len, &cap, note);
os_run_timeout_effective());
os_cap_note(&r->out, &r->out_len, &cap, note);
r->exit_code = 124;
} else if (killed == 2) {
cap_note(&r->out, &r->out_len, &cap, "\n[cst: cancelled]\n");
os_cap_note(&r->out, &r->out_len, &cap, "\n[cst: cancelled]\n");
r->exit_code = 130;
}
return 0;
@@ -759,40 +793,64 @@ int os_run_shell_c(const char *cwd, const char *cmd, size_t max_out,
return os_run_shell_ex(cwd, cmd, max_out, r, NULL, NULL);
}
/* busybox: busybox sh -c cmd ; /bin/sh: sh -c cmd. argv needs 5 slots and
* points into shell. */
static int shell_argv(char *shell, size_t shell_sz, const char *cmd,
const char *argv[5]) {
if (!cmd || os_resolve_shell(shell, shell_sz) != 0)
return -1;
const char *base = shell;
const char *s = strrchr(shell, '/');
#ifdef _WIN32
const char *s2 = strrchr(shell, '\\');
if (s2 && (!s || s2 > s))
s = s2;
#endif
if (s)
base = s + 1;
int i = 0;
argv[i++] = shell;
if (strncmp(base, "busybox", 7) == 0)
argv[i++] = "sh";
argv[i++] = "-c";
argv[i++] = cmd;
argv[i] = NULL;
return 0;
}
int os_run_shell_ex(const char *cwd, const char *cmd, size_t max_out,
os_run_result_t *r, os_child_hook_fn child_hook,
void *ud) {
if (!cmd || !r)
return -1;
char shell[OS_PATH_MAX];
if (os_resolve_shell(shell, sizeof(shell)) != 0)
const char *argv[5];
if (!r || shell_argv(shell, sizeof(shell), cmd, argv) != 0)
return -1;
/* busybox: busybox sh -c cmd ; /bin/sh: sh -c cmd */
int is_busybox = 0;
{
const char *base = shell;
const char *s = strrchr(shell, '/');
#ifdef _WIN32
const char *s2 = strrchr(shell, '\\');
if (s2 && (!s || s2 > s))
s = s2;
#endif
if (s)
base = s + 1;
if (strncmp(base, "busybox", 7) == 0)
is_busybox = 1;
}
if (is_busybox) {
const char *argv[] = {shell, "sh", "-c", cmd, NULL};
return os_run_capture_ex(cwd, argv, max_out, r, child_hook, ud);
} else {
const char *argv[] = {shell, "-c", cmd, NULL};
return os_run_capture_ex(cwd, argv, max_out, r, child_hook, ud);
}
return os_run_capture_ex(cwd, argv, max_out, r, child_hook, ud);
}
#ifdef _WIN32
int os_run_shell_token(const char *cwd, const char *cmd, size_t max_out,
os_run_result_t *r, void *token,
const char *const env_add[]) {
char shell[OS_PATH_MAX];
const char *argv[5];
if (!r || shell_argv(shell, sizeof(shell), cmd, argv) != 0)
return -1;
return os_run_capture_token(cwd, argv, max_out, r, token, env_add);
}
#endif
#ifndef _WIN32
int os_console_init(void) { return 0; }
char *os_output_to_utf8(const char *s, size_t n, size_t *out_len) {
(void)s;
(void)n;
(void)out_len;
return NULL;
}
#endif
void os_localtime(const time_t *t, struct tm *out) {
if (!t || !out)
return;
@@ -811,16 +869,13 @@ int os_getpid(void) {
#endif
}
#ifndef _WIN32
int os_chmod_private(const char *path) {
if (!path)
return -1;
#ifdef _WIN32
(void)path;
return 0;
#else
return chmod(path, 0600);
#endif
}
#endif
typedef struct {
os_task_fn fn;
@@ -901,137 +956,14 @@ void os_run_parallel(os_task_fn fn, void *const args[], int n) {
free(tasks);
}
int os_cancel_requested(void) { return g_run_cancel && *g_run_cancel; }
volatile int *os_get_cancel_flag(void) { return g_run_cancel; }
#ifdef _WIN32
static void win_quote_arg(char *out, size_t out_sz, size_t *pos,
const char *a) {
int need_q = !a[0] || strpbrk(a, " \t\"") != NULL;
if (need_q && *pos + 1 < out_sz)
out[(*pos)++] = '"';
for (const char *p = a; *p && *pos + 2 < out_sz; p++) {
if (*p == '"')
out[(*pos)++] = '\\';
out[(*pos)++] = *p;
}
if (need_q && *pos + 1 < out_sz)
out[(*pos)++] = '"';
out[*pos] = 0;
int os_cancel_requested(void) {
volatile int *c = run_cancel();
return c && *c;
}
int os_proc_spawn(const char *cwd, const char *const argv[],
const char *const env_add[], const char *stderr_path,
os_proc_t *p) {
memset(p, 0, sizeof(*p));
if (!argv || !argv[0])
return -1;
char cmdline[16384];
size_t pos = 0;
cmdline[0] = 0;
for (int i = 0; argv[i]; i++) {
if (i && pos + 1 < sizeof(cmdline))
cmdline[pos++] = ' ';
win_quote_arg(cmdline, sizeof(cmdline), &pos, argv[i]);
}
for (int i = 0; env_add && env_add[i]; i++)
_putenv(env_add[i]); /* inherited by the child */
SECURITY_ATTRIBUTES sa = {0};
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
HANDLE in_r = NULL, in_w = NULL, out_r = NULL, out_w = NULL;
if (!CreatePipe(&in_r, &in_w, &sa, 0))
return -1;
if (!CreatePipe(&out_r, &out_w, &sa, 0)) {
CloseHandle(in_r);
CloseHandle(in_w);
return -1;
}
SetHandleInformation(in_w, HANDLE_FLAG_INHERIT, 0);
SetHandleInformation(out_r, HANDLE_FLAG_INHERIT, 0);
HANDLE err = CreateFileA(stderr_path ? stderr_path : "NUL", GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE, &sa,
CREATE_ALWAYS, 0, NULL);
STARTUPINFOA si = {0};
si.cb = sizeof(si);
si.dwFlags = STARTF_USESTDHANDLES;
si.hStdInput = in_r;
si.hStdOutput = out_w;
si.hStdError = err != INVALID_HANDLE_VALUE ? err : out_w;
PROCESS_INFORMATION pi = {0};
BOOL ok = CreateProcessA(NULL, cmdline, NULL, NULL, TRUE, CREATE_NO_WINDOW,
NULL, cwd && cwd[0] ? cwd : NULL, &si, &pi);
CloseHandle(in_r);
CloseHandle(out_w);
if (err != INVALID_HANDLE_VALUE)
CloseHandle(err);
if (!ok) {
CloseHandle(in_w);
CloseHandle(out_r);
return -1;
}
CloseHandle(pi.hThread);
p->proc = pi.hProcess;
p->in_w = in_w;
p->out_r = out_r;
return 0;
}
volatile int *os_get_cancel_flag(void) { return run_cancel(); }
int os_proc_write(os_proc_t *p, const char *data, size_t n) {
while (n > 0) {
DWORD w = 0;
if (!p->in_w || !WriteFile((HANDLE)p->in_w, data, (DWORD)n, &w, NULL) ||
w == 0)
return -1;
data += w;
n -= w;
}
return 0;
}
long os_proc_read(os_proc_t *p, char *buf, size_t n, int timeout_ms) {
if (!p->out_r)
return -1;
double deadline = os_now_ms() + timeout_ms;
for (;;) {
DWORD avail = 0;
if (!PeekNamedPipe((HANDLE)p->out_r, NULL, 0, NULL, &avail, NULL))
return GetLastError() == ERROR_BROKEN_PIPE ? 0 : -1;
if (avail > 0)
break;
if (os_now_ms() >= deadline)
return -2;
Sleep(10);
}
DWORD r = 0;
if (!ReadFile((HANDLE)p->out_r, buf, (DWORD)n, &r, NULL))
return GetLastError() == ERROR_BROKEN_PIPE ? 0 : -1;
return (long)r;
}
void os_proc_close_stdin(os_proc_t *p) {
if (p->in_w) {
CloseHandle((HANDLE)p->in_w);
p->in_w = NULL;
}
}
void os_proc_kill(os_proc_t *p) {
os_proc_close_stdin(p);
if (p->proc) {
if (WaitForSingleObject((HANDLE)p->proc, 500) != WAIT_OBJECT_0)
TerminateProcess((HANDLE)p->proc, 1);
WaitForSingleObject((HANDLE)p->proc, 2000);
CloseHandle((HANDLE)p->proc);
p->proc = NULL;
}
if (p->out_r) {
CloseHandle((HANDLE)p->out_r);
p->out_r = NULL;
}
}
#else
#ifndef _WIN32
int os_proc_spawn(const char *cwd, const char *const argv[],
const char *const env_add[], const char *stderr_path,
os_proc_t *p) {
@@ -1169,6 +1101,74 @@ void os_proc_kill(os_proc_t *p) {
}
#endif
/* --- UTF-8 ---------------------------------------------------------------
* See the declaration in os_compat.h for why wcrtomb() is not used. */
int os_utf8_encode(unsigned long cp, char out[4]) {
if (cp < 0x80) {
out[0] = (char)cp;
return 1;
}
if (cp < 0x800) {
out[0] = (char)(0xC0 | (cp >> 6));
out[1] = (char)(0x80 | (cp & 0x3F));
return 2;
}
if (cp >= 0xD800 && cp <= 0xDFFF)
return 0; /* lone surrogate half from a 16-bit wchar_t */
if (cp < 0x10000) {
out[0] = (char)(0xE0 | (cp >> 12));
out[1] = (char)(0x80 | ((cp >> 6) & 0x3F));
out[2] = (char)(0x80 | (cp & 0x3F));
return 3;
}
if (cp <= 0x10FFFF) {
out[0] = (char)(0xF0 | (cp >> 18));
out[1] = (char)(0x80 | ((cp >> 12) & 0x3F));
out[2] = (char)(0x80 | ((cp >> 6) & 0x3F));
out[3] = (char)(0x80 | (cp & 0x3F));
return 4;
}
return 0;
}
/* See the declaration in os_compat.h for why mbrtowc() is not used. */
int os_utf8_decode(const char *s, size_t avail, unsigned long *cp) {
const unsigned char *p = (const unsigned char *)s;
int need, i;
unsigned long v;
static const unsigned long MINV[4] = {0, 0x80, 0x800, 0x10000};
if (!s || !cp || avail == 0)
return 0;
if (p[0] < 0x80) {
*cp = p[0];
return 1;
}
if ((p[0] & 0xE0) == 0xC0) {
need = 1;
v = p[0] & 0x1Fu;
} else if ((p[0] & 0xF0) == 0xE0) {
need = 2;
v = p[0] & 0x0Fu;
} else if ((p[0] & 0xF8) == 0xF0) {
need = 3;
v = p[0] & 0x07u;
} else {
return 0; /* not a UTF-8 lead byte */
}
if (avail < (size_t)need + 1)
return 0; /* truncated */
for (i = 1; i <= need; i++) {
if ((p[i] & 0xC0) != 0x80)
return 0; /* bad continuation byte */
v = (v << 6) | (p[i] & 0x3Fu);
}
if (v < MINV[need] || (v >= 0xD800 && v <= 0xDFFF) || v > 0x10FFFF)
return 0; /* overlong, surrogate, or out of range */
*cp = v;
return need + 1;
}
/* --- libc gaps on Windows -----------------------------------------------
* strcasestr() is a GNU extension and setenv()/unsetenv() are POSIX; glibc
* has all three, MinGW-w64 has none of them. src/tui.c needs them (command
+14
View File
@@ -0,0 +1,14 @@
/* Shared between os_compat.c and os_win.c; not part of the public API. */
#ifndef CST_OS_INTERNAL_H
#define CST_OS_INTERNAL_H
#include <stddef.h>
/* CST_SHELL_TIMEOUT / os_set_run_timeout / 120 s */
int os_run_timeout_effective(void);
/* Append to a capture buffer, keeping at most max_out bytes. */
int os_cap_append(char **buf, size_t *len, size_t *cap, size_t max_out,
const char *data, size_t n, int *truncated);
void os_cap_note(char **buf, size_t *len, size_t *cap, const char *note);
#endif
+937
View File
@@ -0,0 +1,937 @@
/* Windows side of os_compat: every string crossing the OS boundary is UTF-8
* inside cst-agent and UTF-16 at the Win32 W APIs. The embedded manifest also
* sets the process code page to UTF-8, so CRT calls (fopen, getenv, argv)
* see UTF-8 too. */
#ifdef _WIN32
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include "os_compat.h"
#include "cst_util.h"
#include "os_internal.h"
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <windows.h>
#include <aclapi.h>
#include <sddl.h>
wchar_t *os_utf8_to_wide(const char *s) {
if (!s)
return NULL;
int n = MultiByteToWideChar(CP_UTF8, 0, s, -1, NULL, 0);
if (n <= 0)
return NULL;
wchar_t *w = malloc((size_t)n * sizeof(wchar_t));
if (w && MultiByteToWideChar(CP_UTF8, 0, s, -1, w, n) <= 0) {
free(w);
return NULL;
}
return w;
}
char *os_wide_to_utf8(const wchar_t *w) {
if (!w)
return NULL;
int n = WideCharToMultiByte(CP_UTF8, 0, w, -1, NULL, 0, NULL, NULL);
if (n <= 0)
return NULL;
char *s = malloc((size_t)n);
if (s && WideCharToMultiByte(CP_UTF8, 0, w, -1, s, n, NULL, NULL) <= 0) {
free(s);
return NULL;
}
return s;
}
static UINT g_old_cp, g_old_out_cp;
static void console_restore(void) {
if (g_old_cp)
SetConsoleCP(g_old_cp);
if (g_old_out_cp)
SetConsoleOutputCP(g_old_out_cp);
}
int os_console_init(void) {
static int done;
if (done)
return GetACP() == CP_UTF8 ? 0 : -1;
done = 1;
g_old_cp = GetConsoleCP();
g_old_out_cp = GetConsoleOutputCP();
if (g_old_cp || g_old_out_cp) {
SetConsoleCP(CP_UTF8);
SetConsoleOutputCP(CP_UTF8);
atexit(console_restore);
}
setlocale(LC_ALL, ".UTF-8");
/* children inherit these: Python otherwise writes the ANSI code page to pipes */
if (!getenv("PYTHONUTF8"))
_putenv_s("PYTHONUTF8", "1");
if (!getenv("PYTHONIOENCODING"))
_putenv_s("PYTHONIOENCODING", "utf-8");
return GetACP() == CP_UTF8 ? 0 : -1;
}
/* The system locale's code page. GetOEMCP()/GetACP() say 65001 once the
* manifest makes this process UTF-8, but children without one (cmd, most
* console tools) still write the system's OEM code page, e.g. 936. */
static UINT system_cp(LCTYPE which) {
wchar_t b[8];
if (!GetLocaleInfoW(LOCALE_SYSTEM_DEFAULT, which, b, 8))
return 0;
return (UINT)wcstoul(b, NULL, 10);
}
char *os_output_to_utf8(const char *s, size_t n, size_t *out_len) {
if (!s || cst_utf8_valid(s, n))
return NULL;
UINT cps[] = {GetConsoleOutputCP(), system_cp(LOCALE_IDEFAULTCODEPAGE),
system_cp(LOCALE_IDEFAULTANSICODEPAGE), GetOEMCP(), GetACP()};
for (size_t i = 0; i < sizeof(cps) / sizeof(cps[0]); i++) {
UINT cp = cps[i];
if (!cp || cp == CP_UTF8)
continue;
int wn = MultiByteToWideChar(cp, MB_ERR_INVALID_CHARS, s, (int)n, NULL, 0);
if (wn <= 0)
continue;
wchar_t *w = malloc((size_t)wn * sizeof(wchar_t));
if (!w)
return NULL;
MultiByteToWideChar(cp, 0, s, (int)n, w, wn);
int un = WideCharToMultiByte(CP_UTF8, 0, w, wn, NULL, 0, NULL, NULL);
char *u = un > 0 ? malloc((size_t)un + 1) : NULL;
if (u) {
WideCharToMultiByte(CP_UTF8, 0, w, wn, u, un, NULL, NULL);
u[un] = 0;
if (out_len)
*out_len = (size_t)un;
}
free(w);
return u;
}
return NULL;
}
/* --- paths --------------------------------------------------------------- */
/* UTF-8 path -> wide, with the \\?\ prefix once it gets near MAX_PATH so
* deep trees work without the LongPathsEnabled policy. */
static wchar_t *wpath(const char *p) {
wchar_t *w = os_utf8_to_wide(p);
if (!w || wcslen(w) < MAX_PATH - 12)
return w;
DWORD n = GetFullPathNameW(w, 0, NULL, NULL);
wchar_t *f = n ? malloc(n * sizeof(wchar_t)) : NULL;
wchar_t *out = n ? malloc((n + 8) * sizeof(wchar_t)) : NULL;
if (!f || !out || GetFullPathNameW(w, n, f, NULL) == 0) {
free(f);
free(out);
return w;
}
free(w);
if (wcsncmp(f, L"\\\\?\\", 4) == 0) {
wcscpy(out, f);
} else if (f[0] == L'\\' && f[1] == L'\\') { /* \\srv\share -> \\?\UNC\srv\share */
wcscpy(out, L"\\\\?\\UNC");
wcscat(out, f + 1);
} else {
wcscpy(out, L"\\\\?\\");
wcscat(out, f);
}
free(f);
return out;
}
/* Wide path from the OS -> UTF-8 with '/' separators, no \\?\ prefix and a
* lowercase drive letter (stable for prefix compares). */
static int path_out(const wchar_t *w, char *out, size_t out_sz) {
const char *lead = "";
if (wcsncmp(w, L"\\\\?\\UNC\\", 8) == 0) {
lead = "\\\\"; /* \\?\UNC\srv\share -> \\srv\share */
w += 8;
} else if (wcsncmp(w, L"\\\\?\\", 4) == 0) {
w += 4;
}
char *u = os_wide_to_utf8(w);
if (!u)
return -1;
int n = snprintf(out, out_sz, "%s%s", lead, u);
free(u);
if (n < 0 || (size_t)n >= out_sz)
return -1;
os_path_norm_slashes(out);
if (out[0] >= 'A' && out[0] <= 'Z' && out[1] == ':')
out[0] = (char)(out[0] - 'A' + 'a');
return 0;
}
int os_realpath(const char *path, char *out, size_t out_sz) {
if (!path || !path[0] || !out || out_sz < 2)
return -1;
wchar_t *w = wpath(path);
if (!w)
return -1;
HANDLE h = CreateFileW(w, 0,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL);
free(w);
if (h == INVALID_HANDLE_VALUE)
return -1;
wchar_t stackbuf[MAX_PATH + 16], *buf = stackbuf;
DWORD n = GetFinalPathNameByHandleW(h, buf, MAX_PATH + 16,
FILE_NAME_NORMALIZED | VOLUME_NAME_DOS);
if (n >= MAX_PATH + 16) {
buf = malloc((n + 1) * sizeof(wchar_t));
n = buf ? GetFinalPathNameByHandleW(h, buf, n + 1,
FILE_NAME_NORMALIZED | VOLUME_NAME_DOS)
: 0;
}
CloseHandle(h);
int rc = n ? path_out(buf, out, out_sz) : -1;
if (buf != stackbuf)
free(buf);
return rc;
}
/* NTFS names are case-insensitive: compare the prefix the way the OS does. */
int os_path_under(const char *root, const char *path) {
if (!root || !path)
return 0;
wchar_t *wr = os_utf8_to_wide(root), *wp = os_utf8_to_wide(path);
int ok = 0;
if (wr && wp) {
size_t rl = wcslen(wr);
if (wcslen(wp) >= rl &&
(rl == 0 || CompareStringOrdinal(wr, (int)rl, wp, (int)rl, TRUE) ==
CSTR_EQUAL)) {
wchar_t c = wp[rl];
ok = c == 0 || c == L'/' || c == L'\\' ||
(rl && (wr[rl - 1] == L'/' || wr[rl - 1] == L'\\'));
}
}
free(wr);
free(wp);
return ok;
}
static int reparse_is_link(DWORD attrs, DWORD tag) {
return (attrs & FILE_ATTRIBUTE_REPARSE_POINT) &&
(tag == IO_REPARSE_TAG_SYMLINK || tag == IO_REPARSE_TAG_MOUNT_POINT);
}
/* --- directories ------------------------------------------------------- */
struct os_dir {
HANDLE h;
int pending; /* fd holds an entry not yet returned */
WIN32_FIND_DATAW fd;
char name[MAX_PATH * 4];
os_dirent_t ent;
};
os_dir_t *os_dir_open(const char *path) {
if (!path || !path[0])
return NULL;
size_t L = strlen(path);
char *pat = malloc(L + 3);
if (!pat)
return NULL;
memcpy(pat, path, L);
if (path[L - 1] != '/' && path[L - 1] != '\\')
pat[L++] = '\\';
pat[L++] = '*';
pat[L] = 0;
wchar_t *w = wpath(pat);
free(pat);
os_dir_t *d = w ? calloc(1, sizeof(*d)) : NULL;
if (!d) {
free(w);
return NULL;
}
d->h = FindFirstFileExW(w, FindExInfoBasic, &d->fd, FindExSearchNameMatch,
NULL, FIND_FIRST_EX_LARGE_FETCH);
free(w);
if (d->h == INVALID_HANDLE_VALUE) {
DWORD e = GetLastError();
if (e != ERROR_FILE_NOT_FOUND) { /* an empty drive root has no entries */
free(d);
return NULL;
}
} else {
d->pending = 1;
}
return d;
}
const os_dirent_t *os_dir_next(os_dir_t *d) {
if (!d || d->h == INVALID_HANDLE_VALUE)
return NULL;
for (;;) {
if (!d->pending && !FindNextFileW(d->h, &d->fd))
return NULL;
d->pending = 0;
const wchar_t *n = d->fd.cFileName;
if (wcscmp(n, L".") == 0 || wcscmp(n, L"..") == 0)
continue;
if (WideCharToMultiByte(CP_UTF8, 0, n, -1, d->name, (int)sizeof(d->name),
NULL, NULL) <= 0)
continue;
DWORD a = d->fd.dwFileAttributes;
d->ent.name = d->name;
d->ent.is_link = reparse_is_link(a, d->fd.dwReserved0);
d->ent.is_dir = !d->ent.is_link && (a & FILE_ATTRIBUTE_DIRECTORY);
return &d->ent;
}
}
void os_dir_close(os_dir_t *d) {
if (!d)
return;
if (d->h != INVALID_HANDLE_VALUE)
FindClose(d->h);
free(d);
}
static double filetime_secs(FILETIME ft) {
ULONGLONG t = ((ULONGLONG)ft.dwHighDateTime << 32) | ft.dwLowDateTime;
return (double)t / 1e7 - 11644473600.0;
}
int os_lstat(const char *path, os_stat_t *st) {
memset(st, 0, sizeof(*st));
wchar_t *w = path ? wpath(path) : NULL;
if (!w)
return -1;
WIN32_FILE_ATTRIBUTE_DATA fa;
if (!GetFileAttributesExW(w, GetFileExInfoStandard, &fa)) {
free(w);
return -1;
}
DWORD a = fa.dwFileAttributes;
if (a & FILE_ATTRIBUTE_REPARSE_POINT) {
WIN32_FIND_DATAW fd;
HANDLE h = FindFirstFileW(w, &fd);
if (h != INVALID_HANDLE_VALUE) {
st->is_link = reparse_is_link(a, fd.dwReserved0);
FindClose(h);
}
}
free(w);
st->is_dir = !st->is_link && (a & FILE_ATTRIBUTE_DIRECTORY);
st->is_reg = !st->is_link && !(a & FILE_ATTRIBUTE_DIRECTORY);
st->size = (long long)(((ULONGLONG)fa.nFileSizeHigh << 32) | fa.nFileSizeLow);
st->mtime = filetime_secs(fa.ftLastWriteTime);
return 0;
}
int os_rename_replace(const char *from, const char *to) {
wchar_t *wf = wpath(from), *wt = wpath(to);
int ok = wf && wt &&
MoveFileExW(wf, wt, MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED);
free(wf);
free(wt);
return ok ? 0 : -1;
}
/* --- processes ----------------------------------------------------------- */
/* One argument the way CommandLineToArgvW and the CRT split it again. */
static void cmdline_arg(cst_buf_t *b, const char *a) {
if (a[0] && !strpbrk(a, " \t\n\v\"")) {
cst_buf_puts(b, a);
return;
}
cst_buf_puts(b, "\"");
for (const char *p = a;; p++) {
size_t bs = 0;
while (*p == '\\') {
bs++;
p++;
}
if (!*p) {
for (size_t i = 0; i < bs * 2; i++)
cst_buf_puts(b, "\\");
break;
}
for (size_t i = 0; i < (*p == '"' ? bs * 2 + 1 : bs); i++)
cst_buf_puts(b, "\\");
cst_buf_append(b, p, 1);
}
cst_buf_puts(b, "\"");
}
static wchar_t *build_cmdline(const char *const argv[]) {
cst_buf_t b = {0};
for (int i = 0; argv[i]; i++) {
if (i)
cst_buf_puts(&b, " ");
cmdline_arg(&b, argv[i]);
}
wchar_t *w = b.buf ? os_utf8_to_wide(b.buf) : NULL;
cst_buf_free(&b);
return w;
}
static size_t env_name_len(const wchar_t *e) {
const wchar_t *eq = wcschr(e + 1, L'='); /* "=C:=C:\x" hides a leading '=' */
return eq ? (size_t)(eq - e) : wcslen(e);
}
static int env_cmp(const void *a, const void *b) {
const wchar_t *x = *(const wchar_t *const *)a, *y = *(const wchar_t *const *)b;
return CompareStringOrdinal(x, (int)env_name_len(x), y, (int)env_name_len(y),
TRUE) -
CSTR_EQUAL;
}
/* The inherited environment with env_add ("KEY=VALUE") layered on top, as a
* sorted CREATE_UNICODE_ENVIRONMENT block; NULL = inherit unchanged. */
static wchar_t *env_block(const char *const env_add[]) {
if (!env_add || !env_add[0])
return NULL;
size_t nadd = 0, ncur = 0;
while (env_add[nadd])
nadd++;
wchar_t *cur = GetEnvironmentStringsW();
for (const wchar_t *e = cur; e && *e; e += wcslen(e) + 1)
ncur++;
wchar_t **add = calloc(nadd, sizeof(*add));
wchar_t **v = calloc(ncur + nadd, sizeof(*v));
wchar_t *blk = NULL;
size_t n = 0, total = 1;
if (!add || !v)
goto out;
for (size_t i = 0; i < nadd; i++)
if (!(add[i] = os_utf8_to_wide(env_add[i])))
goto out;
for (wchar_t *e = cur; e && *e; e += wcslen(e) + 1) {
int shadowed = 0;
for (size_t i = 0; i < nadd && !shadowed; i++)
shadowed = env_cmp(&e, &add[i]) == 0;
if (!shadowed)
v[n++] = e;
}
for (size_t i = 0; i < nadd; i++) {
int dup = 0; /* a later entry for the same name wins */
for (size_t k = i + 1; k < nadd && !dup; k++)
dup = env_cmp(&add[i], &add[k]) == 0;
if (!dup)
v[n++] = add[i];
}
qsort(v, n, sizeof(*v), env_cmp);
for (size_t i = 0; i < n; i++)
total += wcslen(v[i]) + 1;
blk = malloc(total * sizeof(wchar_t));
if (blk) {
wchar_t *o = blk;
for (size_t i = 0; i < n; i++) {
size_t l = wcslen(v[i]) + 1;
memcpy(o, v[i], l * sizeof(wchar_t));
o += l;
}
*o = 0;
}
out:
for (size_t i = 0; add && i < nadd; i++)
free(add[i]);
free(add);
free(v);
if (cur)
FreeEnvironmentStringsW(cur);
return blk;
}
static HANDLE new_job(void) {
HANDLE job = CreateJobObjectW(NULL, NULL);
if (!job)
return NULL;
JOBOBJECT_EXTENDED_LIMIT_INFORMATION li;
memset(&li, 0, sizeof(li));
li.BasicLimitInformation.LimitFlags =
JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE |
JOB_OBJECT_LIMIT_DIE_ON_UNHANDLED_EXCEPTION;
SetInformationJobObject(job, JobObjectExtendedLimitInformation, &li,
sizeof(li));
return job;
}
static DWORD job_active(HANDLE job) {
JOBOBJECT_BASIC_ACCOUNTING_INFORMATION ai;
if (!job || !QueryInformationJobObject(job,
JobObjectBasicAccountingInformation,
&ai, sizeof(ai), NULL))
return 0;
return ai.ActiveProcesses;
}
/* Children still running after the shell exited. A child started without
* our console (the sandbox) has its own conhost.exe, which leaves the job a
* moment after its client, so give the job a short while to drain. */
static int job_lingers(HANDLE job) {
for (int i = 0; i < 50 && job_active(job) > 0; i++)
Sleep(10);
return job_active(job) > 0;
}
/* Start argv suspended, put it in a fresh kill-on-close job, then resume.
* Only in/out/err are inherited (PROC_THREAD_ATTRIBUTE_HANDLE_LIST), so a
* concurrent spawn on another thread never leaks into this child. */
static int spawn(const char *cwd, const char *const argv[],
const char *const env_add[], HANDLE token, HANDLE in,
HANDLE out, HANDLE err, HANDLE *proc, HANDLE *job,
DWORD *error) {
*proc = *job = NULL;
*error = ERROR_OUTOFMEMORY;
wchar_t *cmd = build_cmdline(argv);
wchar_t *wcwd = cwd && cwd[0] ? os_utf8_to_wide(cwd) : NULL;
wchar_t *env = env_block(env_add);
HANDLE inherit[3];
int ninh = 0;
HANDLE std3[3] = {in, out, err};
for (int i = 0; i < 3; i++) {
int seen = !std3[i] || std3[i] == INVALID_HANDLE_VALUE;
for (int k = 0; k < ninh && !seen; k++)
seen = inherit[k] == std3[i];
if (!seen)
inherit[ninh++] = std3[i];
}
SIZE_T asz = 0;
InitializeProcThreadAttributeList(NULL, 1, 0, &asz);
LPPROC_THREAD_ATTRIBUTE_LIST attrs = malloc(asz);
int ok = 0;
if (cmd && (!cwd || !cwd[0] || wcwd) && (!env_add || !env_add[0] || env) &&
attrs && InitializeProcThreadAttributeList(attrs, 1, 0, &asz)) {
if (UpdateProcThreadAttribute(attrs, 0, PROC_THREAD_ATTRIBUTE_HANDLE_LIST,
inherit, (SIZE_T)ninh * sizeof(HANDLE), NULL,
NULL)) {
STARTUPINFOEXW si;
memset(&si, 0, sizeof(si));
si.StartupInfo.cb = sizeof(si);
si.StartupInfo.dwFlags = STARTF_USESTDHANDLES;
si.StartupInfo.hStdInput = in;
si.StartupInfo.hStdOutput = out;
si.StartupInfo.hStdError = err;
si.lpAttributeList = attrs;
/* Share our console when there is one (children then see its UTF-8
* code page); the sandbox token cannot attach to it. */
DWORD flags = CREATE_SUSPENDED | CREATE_UNICODE_ENVIRONMENT |
CREATE_NEW_PROCESS_GROUP | EXTENDED_STARTUPINFO_PRESENT;
if (token || !GetConsoleWindow())
flags |= CREATE_NO_WINDOW;
PROCESS_INFORMATION pi;
memset(&pi, 0, sizeof(pi));
ok = token ? CreateProcessAsUserW(token, NULL, cmd, NULL, NULL, TRUE,
flags, env, wcwd, &si.StartupInfo, &pi)
: CreateProcessW(NULL, cmd, NULL, NULL, TRUE, flags, env, wcwd,
&si.StartupInfo, &pi);
if (ok) {
*job = new_job();
if (*job && !AssignProcessToJobObject(*job, pi.hProcess)) {
CloseHandle(*job);
*job = NULL;
}
ResumeThread(pi.hThread);
CloseHandle(pi.hThread);
*proc = pi.hProcess;
} else {
*error = GetLastError();
}
}
DeleteProcThreadAttributeList(attrs);
}
free(attrs);
free(cmd);
free(wcwd);
free(env);
return ok ? 0 : -1;
}
int os_run_capture_token(const char *cwd, const char *const argv[],
size_t max_out, os_run_result_t *r, void *token,
const char *const env_add[]) {
if (!argv || !argv[0] || !r)
return -1;
if (max_out == 0)
max_out = 512 * 1024;
memset(r, 0, sizeof(*r));
SECURITY_ATTRIBUTES sa = {sizeof(sa), NULL, TRUE};
HANDLE rd = NULL, wr = NULL;
if (!CreatePipe(&rd, &wr, &sa, 64 * 1024))
return -1;
SetHandleInformation(rd, HANDLE_FLAG_INHERIT, 0);
HANDLE nul = CreateFileW(L"NUL", GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE, &sa,
OPEN_EXISTING, 0, NULL);
HANDLE proc, job;
DWORD err;
int rc = spawn(cwd, argv, env_add, (HANDLE)token, nul, wr, wr, &proc, &job,
&err);
CloseHandle(wr);
if (nul != INVALID_HANDLE_VALUE)
CloseHandle(nul);
if (rc != 0) {
CloseHandle(rd);
if (err != ERROR_FILE_NOT_FOUND && err != ERROR_PATH_NOT_FOUND)
return -1;
cst_buf_t b = {0};
cst_buf_printf(&b, "%s: command not found\n", argv[0]);
r->out_len = b.len;
r->out = cst_buf_steal(&b);
r->exit_code = 127;
return 0;
}
size_t cap = 4096, len = 0;
char *buf = malloc(cap);
if (!buf) {
TerminateJobObject(job, 1);
TerminateProcess(proc, 1);
CloseHandle(rd);
CloseHandle(proc);
if (job)
CloseHandle(job);
return -1;
}
buf[0] = 0;
int truncated = 0, killed = 0, reaped = 0, bg_killed = 0;
double reaped_at = 0, killed_at = 0;
/* no timeout unless one was asked for: a deadline far in the future keeps
* every `now > deadline` check working without a second code path */
int timeout_s = os_run_timeout_effective();
double deadline = timeout_s > 0 ? os_now_ms() + timeout_s * 1000.0 : 1e18;
int idle = 0; /* empty polls in a row: wait longer once output stalls */
for (;;) {
DWORD avail = 0;
if (!PeekNamedPipe(rd, NULL, 0, NULL, &avail, NULL))
break; /* every writer closed */
if (avail) {
char tmp[65536];
DWORD n = 0;
if (!ReadFile(rd, tmp, avail < sizeof(tmp) ? avail : sizeof(tmp), &n,
NULL) ||
n == 0)
break;
os_cap_append(&buf, &len, &cap, max_out, tmp, n, &truncated);
idle = 0;
} else if (reaped) {
Sleep(idle++ < 50 ? 1 : 20);
}
DWORD wait = avail ? 0 : idle++ < 50 ? 1 : 20;
if (!reaped && WaitForSingleObject(proc, wait) == WAIT_OBJECT_0) {
reaped = 1;
reaped_at = os_now_ms();
}
double now = os_now_ms();
/* The shell exited but a background child still holds the pipe
* (`server &`): give it a moment, then end the tree. */
if (reaped && !killed && !bg_killed && now - reaped_at > 1000) {
TerminateJobObject(job, 1);
bg_killed = 1;
killed_at = now;
}
if (!reaped && !killed && (os_cancel_requested() || now > deadline)) {
killed = os_cancel_requested() ? 2 : 1;
killed_at = now;
TerminateJobObject(job, killed == 1 ? 124 : 130);
if (!job)
TerminateProcess(proc, killed == 1 ? 124 : 130);
}
/* a writer outside the job can keep the pipe open */
if ((killed || bg_killed) && now - killed_at > 3000)
break;
}
CloseHandle(rd);
WaitForSingleObject(proc, killed ? 5000 : INFINITE);
DWORD code = 1;
GetExitCodeProcess(proc, &code);
if (!bg_killed && !killed && job_lingers(job))
bg_killed = 1; /* closing the job below ends them */
CloseHandle(proc);
if (job)
CloseHandle(job);
size_t ulen = 0;
char *u = os_output_to_utf8(buf, len, &ulen);
if (u) {
free(buf);
buf = u;
len = ulen;
cap = ulen + 1;
}
if (truncated)
os_cap_note(&buf, &len, &cap, "\n...(output truncated)\n");
if (bg_killed && !killed)
os_cap_note(&buf, &len, &cap,
"\n[cst: background processes left running were stopped]\n");
if (killed == 1) {
char note[96];
snprintf(note, sizeof(note), "\n[cst: killed after %ds timeout]\n",
os_run_timeout_effective());
os_cap_note(&buf, &len, &cap, note);
code = 124;
} else if (killed == 2) {
os_cap_note(&buf, &len, &cap, "\n[cst: cancelled]\n");
code = 130;
}
r->out = buf;
r->out_len = len;
r->exit_code = (int)code;
return 0;
}
int os_run_capture(const char *cwd, const char *const argv[], size_t max_out,
os_run_result_t *r) {
return os_run_capture_token(cwd, argv, max_out, r, NULL, NULL);
}
int os_run_capture_ex(const char *cwd, const char *const argv[],
size_t max_out, os_run_result_t *r,
os_child_hook_fn child_hook, void *ud) {
(void)child_hook;
(void)ud;
return os_run_capture_token(cwd, argv, max_out, r, NULL, NULL);
}
int os_proc_spawn(const char *cwd, const char *const argv[],
const char *const env_add[], const char *stderr_path,
os_proc_t *p) {
memset(p, 0, sizeof(*p));
if (!argv || !argv[0])
return -1;
SECURITY_ATTRIBUTES sa = {sizeof(sa), NULL, TRUE};
HANDLE in_r = NULL, in_w = NULL, out_r = NULL, out_w = NULL;
if (!CreatePipe(&in_r, &in_w, &sa, 64 * 1024))
return -1;
if (!CreatePipe(&out_r, &out_w, &sa, 64 * 1024)) {
CloseHandle(in_r);
CloseHandle(in_w);
return -1;
}
SetHandleInformation(in_w, HANDLE_FLAG_INHERIT, 0);
SetHandleInformation(out_r, HANDLE_FLAG_INHERIT, 0);
wchar_t *werr = stderr_path ? wpath(stderr_path) : NULL;
HANDLE err = CreateFileW(werr ? werr : L"NUL", GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE, &sa,
werr ? CREATE_ALWAYS : OPEN_EXISTING, 0, NULL);
free(werr);
HANDLE proc, job;
DWORD e;
int rc = spawn(cwd, argv, env_add, NULL, in_r, out_w,
err != INVALID_HANDLE_VALUE ? err : out_w, &proc, &job, &e);
CloseHandle(in_r);
CloseHandle(out_w);
if (err != INVALID_HANDLE_VALUE)
CloseHandle(err);
if (rc != 0) {
CloseHandle(in_w);
CloseHandle(out_r);
return -1;
}
p->proc = proc;
p->job = job;
p->in_w = in_w;
p->out_r = out_r;
return 0;
}
int os_proc_write(os_proc_t *p, const char *data, size_t n) {
while (n > 0) {
DWORD w = 0;
if (!p->in_w || !WriteFile((HANDLE)p->in_w, data, (DWORD)n, &w, NULL) ||
w == 0)
return -1;
data += w;
n -= w;
}
return 0;
}
long os_proc_read(os_proc_t *p, char *buf, size_t n, int timeout_ms) {
if (!p->out_r)
return -1;
double deadline = os_now_ms() + timeout_ms;
DWORD avail = 0;
for (;;) {
if (!PeekNamedPipe((HANDLE)p->out_r, NULL, 0, NULL, &avail, NULL))
return GetLastError() == ERROR_BROKEN_PIPE ? 0 : -1;
if (avail > 0)
break;
if (os_now_ms() >= deadline)
return -2;
Sleep(5);
}
DWORD r = 0;
if (!ReadFile((HANDLE)p->out_r, buf, avail < n ? avail : (DWORD)n, &r, NULL))
return GetLastError() == ERROR_BROKEN_PIPE ? 0 : -1;
return (long)r;
}
void os_proc_close_stdin(os_proc_t *p) {
if (p->in_w) {
CloseHandle((HANDLE)p->in_w);
p->in_w = NULL;
}
}
void os_proc_kill(os_proc_t *p) {
os_proc_close_stdin(p);
if (p->proc) {
/* polite first: most servers exit when stdin closes */
if (WaitForSingleObject((HANDLE)p->proc, 500) != WAIT_OBJECT_0) {
if (p->job)
TerminateJobObject((HANDLE)p->job, 1);
else
TerminateProcess((HANDLE)p->proc, 1);
}
WaitForSingleObject((HANDLE)p->proc, 2000);
CloseHandle((HANDLE)p->proc);
p->proc = NULL;
}
if (p->job) {
CloseHandle((HANDLE)p->job); /* ends anything the server left behind */
p->job = NULL;
}
if (p->out_r) {
CloseHandle((HANDLE)p->out_r);
p->out_r = NULL;
}
}
void os_windows_version(char *out, size_t n) {
typedef LONG(WINAPI * rtl_ver_fn)(OSVERSIONINFOW *);
OSVERSIONINFOW vi;
memset(&vi, 0, sizeof(vi));
vi.dwOSVersionInfoSize = sizeof(vi);
HMODULE nt = GetModuleHandleW(L"ntdll.dll");
rtl_ver_fn fn = nt ? (rtl_ver_fn)(void (*)(void))GetProcAddress(nt, "RtlGetVersion")
: NULL;
SYSTEM_INFO si;
GetNativeSystemInfo(&si);
const char *arch = si.wProcessorArchitecture == PROCESSOR_ARCHITECTURE_ARM64
? "arm64"
: "x86_64";
if (fn && fn(&vi) == 0)
snprintf(out, n, "Windows %lu.%lu.%lu %s", vi.dwMajorVersion,
vi.dwMinorVersion, vi.dwBuildNumber, arch);
else
snprintf(out, n, "Windows %s", arch);
}
/* Mandatory label from an SDDL SACL such as "S:(ML;;NWNR;;;ME)". */
int os_win_set_label(const char *path, const char *sddl) {
wchar_t *wp = wpath(path), *ws = os_utf8_to_wide(sddl);
PSECURITY_DESCRIPTOR sd = NULL;
int rc = -1;
if (wp && ws &&
ConvertStringSecurityDescriptorToSecurityDescriptorW(ws, SDDL_REVISION_1,
&sd, NULL)) {
PACL sacl = NULL;
BOOL present = FALSE, defaulted = FALSE;
if (GetSecurityDescriptorSacl(sd, &present, &sacl, &defaulted) && present &&
SetNamedSecurityInfoW(wp, SE_FILE_OBJECT, LABEL_SECURITY_INFORMATION,
NULL, NULL, NULL, sacl) == ERROR_SUCCESS)
rc = 0;
LocalFree(sd);
}
free(wp);
free(ws);
return rc;
}
int os_win_get_label(const char *path, char *out, size_t n) {
wchar_t *wp = wpath(path);
PSECURITY_DESCRIPTOR sd = NULL;
wchar_t *ws = NULL;
int rc = -1;
if (wp && n > 2 &&
GetNamedSecurityInfoW(wp, SE_FILE_OBJECT, LABEL_SECURITY_INFORMATION,
NULL, NULL, NULL, NULL, &sd) == ERROR_SUCCESS &&
ConvertSecurityDescriptorToStringSecurityDescriptorW(
sd, SDDL_REVISION_1, LABEL_SECURITY_INFORMATION, &ws, NULL)) {
char *s = os_wide_to_utf8(ws);
if (s) {
/* keep "S:" and the explicit ACEs; inherited ones carry the ID flag */
size_t o = 0;
out[o++] = 'S';
out[o++] = ':';
rc = 0;
for (const char *p = strchr(s, '('); p; p = strchr(p + 1, '(')) {
const char *e = strchr(p, ')');
if (!e)
break;
/* (type;flags;rights;...): flags are two-letter codes */
const char *f = strchr(p, ';');
int inherited = 0;
for (f = f && f < e ? f + 1 : e; f + 1 < e && *f != ';'; f += 2)
inherited |= f[0] == 'I' && f[1] == 'D';
size_t len = (size_t)(e - p) + 1;
if (inherited)
continue;
if (o + len >= n) {
rc = -1;
break;
}
memcpy(out + o, p, len);
o += len;
}
out[o] = 0;
free(s);
}
}
if (ws)
LocalFree(ws);
if (sd)
LocalFree(sd);
free(wp);
return rc;
}
int os_chmod_private(const char *path) {
if (!path)
return -1;
return os_win_set_label(path, "S:(ML;;NWNRNX;;;ME)");
}
/* --- clipboard ----------------------------------------------------------- */
int os_clipboard_set(const char *utf8, size_t n) {
cst_buf_t b = {0}; /* CRLF is what Windows text consumers expect */
for (size_t i = 0; i < n; i++) {
if (utf8[i] == '\n' && (i == 0 || utf8[i - 1] != '\r'))
cst_buf_puts(&b, "\r");
cst_buf_append(&b, utf8 + i, 1);
}
if (!b.buf)
cst_buf_puts(&b, "");
int wn = MultiByteToWideChar(CP_UTF8, 0, b.buf, (int)b.len, NULL, 0);
HGLOBAL h = GlobalAlloc(GMEM_MOVEABLE, ((size_t)wn + 1) * sizeof(wchar_t));
wchar_t *w = h ? GlobalLock(h) : NULL;
if (!w) {
if (h)
GlobalFree(h);
cst_buf_free(&b);
return -1;
}
MultiByteToWideChar(CP_UTF8, 0, b.buf, (int)b.len, w, wn);
w[wn] = 0;
GlobalUnlock(h);
cst_buf_free(&b);
if (!OpenClipboard(NULL)) {
GlobalFree(h);
return -1;
}
EmptyClipboard();
int ok = SetClipboardData(CF_UNICODETEXT, h) != NULL;
if (!ok)
GlobalFree(h);
CloseClipboard();
return ok ? 0 : -1;
}
#endif /* _WIN32 */
+1901
View File
File diff suppressed because it is too large Load Diff
+69 -1
View File
@@ -7,6 +7,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include "cJSON.h"
#include "cst_perms.h"
@@ -33,7 +34,7 @@ static void load(cst_perms_t *p) {
while ((n = fread(tmp, 1, sizeof(tmp), fp)) > 0 && b.len < 256 * 1024)
cst_buf_append(&b, tmp, n);
fclose(fp);
cJSON *root = b.buf ? cJSON_Parse(b.buf) : NULL;
cJSON *root = b.buf ? cJSON_Parse(b.buf + cst_bom_len(b.buf, b.len)) : NULL;
cst_buf_free(&b);
const cJSON *it = NULL;
cJSON_ArrayForEach(it, cJSON_GetObjectItemCaseSensitive(root, "tools")) {
@@ -222,6 +223,73 @@ void cst_perms_suggest_prefix(const char *cmd, char *out, size_t out_sz) {
out[0] = 0;
}
/* --- network commands ------------------------------------------------------ */
static int net_word_eq(const char *w, size_t n, const char *name) {
size_t k = strlen(name);
if (n == k + 4 && strncasecmp(w + k, ".exe", 4) == 0)
n = k;
return n == k && strncasecmp(w, name, k) == 0;
}
int cst_shell_uses_network(const char *cmd) {
static const char *const TOOLS[] = {
"curl", "wget", "ssh", "scp", "sftp", "rsync",
"ftp", "tftp", "telnet", "nc", "ncat", "netcat",
"socat", "npx", "invoke-webrequest", "iwr", "invoke-restmethod",
"irm", "start-bitstransfer", "bitsadmin", "certutil"};
/* tool + sub-command that downloads or uploads */
static const char *const PAIRS[][2] = {
{"git", "clone"}, {"git", "fetch"}, {"git", "pull"},
{"git", "push"}, {"git", "ls-remote"}, {"git", "submodule"},
{"npm", "install"}, {"npm", "i"}, {"npm", "ci"},
{"npm", "update"}, {"npm", "publish"}, {"npm", "exec"},
{"pnpm", "install"}, {"pnpm", "add"}, {"pnpm", "dlx"},
{"yarn", "install"}, {"yarn", "add"}, {"bun", "install"},
{"bun", "add"}, {"pip", "install"}, {"pip", "download"},
{"pip3", "install"}, {"uv", "pip"}, {"uv", "add"},
{"uv", "sync"}, {"cargo", "install"}, {"cargo", "fetch"},
{"cargo", "update"}, {"cargo", "publish"}, {"go", "get"},
{"go", "install"}, {"go", "mod"}, {"gem", "install"},
{"dotnet", "restore"}, {"dotnet", "add"}, {"winget", "install"},
{"choco", "install"}, {"scoop", "install"}, {"apt", "install"},
{"apt-get", "install"}, {"pkg", "install"}, {"brew", "install"}};
if (!cmd)
return 0;
const char *prev = NULL;
size_t prev_n = 0;
for (const char *p = cmd; *p;) {
while (*p && strchr(" \t\r\n;|&()<>`'\"$=", *p))
p++;
const char *w = p;
while (*p && !strchr(" \t\r\n;|&()<>`'\"$=", *p))
p++;
size_t n = (size_t)(p - w);
if (!n)
break;
/* /usr/bin/curl, C:\tools\curl.exe */
for (size_t i = n; i > 0; i--) {
if (w[i - 1] == '/' || w[i - 1] == '\\') {
n -= i;
w += i;
break;
}
}
for (size_t i = 0; i < sizeof(TOOLS) / sizeof(TOOLS[0]); i++)
if (net_word_eq(w, n, TOOLS[i]))
return 1;
for (size_t i = 0; prev && i < sizeof(PAIRS) / sizeof(PAIRS[0]); i++)
if (net_word_eq(prev, prev_n, PAIRS[i][0]) &&
net_word_eq(w, n, PAIRS[i][1]))
return 1;
if (n && w[0] != '-') {
prev = w;
prev_n = n;
}
}
return 0;
}
/* --- read-only command rules ---------------------------------------------- */
#define SAFE_MAX_WORDS 64
+13 -33
View File
@@ -11,10 +11,7 @@
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#ifndef _WIN32
#include <dirent.h>
#include <unistd.h>
#endif
#include "cJSON.h"
#include "cst_mcp.h"
@@ -60,6 +57,7 @@ static char *slurp(const char *path, size_t max) {
fclose(fp);
if (!b.buf)
cst_buf_puts(&b, "");
cst_strip_bom(b.buf, &b.len);
return cst_buf_steal(&b);
}
@@ -101,14 +99,6 @@ static int plugin_run(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r, void *ud) {
const ptool_t *pt = ud;
r->exit_code = 1;
#ifdef _WIN32
(void)ctx;
(void)c;
(void)pt;
r->out = strdup("error: plugin tools are not supported on Windows yet");
r->out_len = r->out ? strlen(r->out) : 0;
return 0;
#else
const cst_tool_def_t *def = cst_tool_find(pt->tool);
char *args = cst_tool_args_json(def, c);
char tmp[OS_PATH_MAX];
@@ -180,7 +170,6 @@ static int plugin_run(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
r->out_len = out.len;
r->out = cst_buf_steal(&out);
return 0;
#endif
}
/* argv array -> quoted command line; relative entries found in the plugin
@@ -339,21 +328,20 @@ static void load_plugin(const char *dir, const char *dirname,
static int scan(const char *root, const char *source) {
int n = 0;
#ifndef _WIN32
DIR *d = opendir(root);
os_dir_t *d = os_dir_open(root);
if (!d)
return 0;
char *names[CST_PLUGINS_MAX * 2];
int nn = 0;
struct dirent *de;
while (nn < CST_PLUGINS_MAX * 2 && (de = readdir(d)) != NULL) {
if (de->d_name[0] == '.')
const os_dirent_t *de;
while (nn < CST_PLUGINS_MAX * 2 && (de = os_dir_next(d)) != NULL) {
if (de->name[0] == '.')
continue;
names[nn] = strdup(de->d_name);
names[nn] = strdup(de->name);
if (names[nn])
nn++;
}
closedir(d);
os_dir_close(d);
for (int i = 1; i < nn; i++)
for (int j = i; j > 0 && strcmp(names[j - 1], names[j]) > 0; j--) {
char *t = names[j];
@@ -371,32 +359,24 @@ static int scan(const char *root, const char *source) {
}
free(names[i]);
}
#else
(void)root;
(void)source;
#endif
return n;
}
static int count_plugin_dirs(const char *root) {
int n = 0;
#ifndef _WIN32
DIR *d = opendir(root);
os_dir_t *d = os_dir_open(root);
if (!d)
return 0;
struct dirent *de;
while ((de = readdir(d)) != NULL) {
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL) {
char pj[OS_PATH_MAX];
if (de->d_name[0] != '.' &&
snprintf(pj, sizeof(pj), "%s/%s/plugin.json", root, de->d_name) <
if (de->name[0] != '.' &&
snprintf(pj, sizeof(pj), "%s/%s/plugin.json", root, de->name) <
(int)sizeof(pj) &&
is_file(pj))
n++;
}
closedir(d);
#else
(void)root;
#endif
os_dir_close(d);
return n;
}
+193
View File
@@ -0,0 +1,193 @@
#include "cst_provider_switch.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int cst_ps_fetch(const char *provider, const char *api_base, const char *key,
char ids[][128], int max, char *err, size_t err_sz) {
cst_model_meta_t *m = calloc((size_t)(max > 0 ? max : 1), sizeof(*m));
if (!m) {
snprintf(err, err_sz, "out of memory");
return -1;
}
int r = cst_llm_list_models_meta(api_base, key, m, max, err, err_sz);
if (r >= 0)
cst_models_cache_save(provider, api_base, m, r);
for (int i = 0; i < r; i++)
snprintf(ids[i], 128, "%s", m[i].id);
free(m);
return r;
}
int cst_ps_cached(const char *provider, char ids[][128], int max, int *fresh) {
*fresh = 0;
cst_model_meta_t *m = calloc((size_t)(max > 0 ? max : 1), sizeof(*m));
if (!m)
return -1;
long age = 0;
int n = cst_models_cache_load(provider, m, max, &age);
for (int i = 0; i < n; i++)
snprintf(ids[i], 128, "%s", m[i].id);
free(m);
*fresh = n > 0 && age >= 0 && age < CST_MODELS_CACHE_TTL;
return n > 0 ? n : -1;
}
const char *cst_ps_base(const cst_agent_config_t *cfg, const char *provider) {
if (cfg && strcmp(cfg->provider, provider) == 0)
return cfg->api_base;
const cst_provider_t *p = cst_provider_find(provider);
return p ? p->api_base : "";
}
int cst_ps_models(const cst_agent_config_t *cfg, int refresh,
cst_ps_model_t *out, int max, int *api_rc, char *err,
size_t err_sz) {
err[0] = 0;
*api_rc = -1;
char(*ids)[128] = calloc(CST_PS_MAX_MODELS, sizeof(*ids));
if (!ids) {
snprintf(err, err_sz, "out of memory");
return 0;
}
int fresh = 0;
int na = cst_ps_cached(cfg->provider, ids, CST_PS_MAX_MODELS, &fresh);
*api_rc = na;
if ((refresh || !fresh) && cfg->api_key[0]) {
char(*got)[128] = calloc(CST_PS_MAX_MODELS, sizeof(*got));
if (got) {
int r = cst_ps_fetch(cfg->provider, cfg->api_base, cfg->api_key, got,
CST_PS_MAX_MODELS, err, err_sz);
*api_rc = r;
if (r >= 0) {
memcpy(ids, got, (size_t)r * sizeof(*ids));
na = r;
}
free(got);
}
}
int n = 0;
for (int i = 0; i < na && n < max; i++, n++) {
snprintf(out[n].id, sizeof(out[n].id), "%s", ids[i]);
out[n].api = 1;
cst_model_info(cfg->provider, ids[i], &out[n].info);
}
char builtin[32][128];
int nb = cst_models_list(cfg->model, builtin, 32);
for (int i = 0; i < nb && n < max; i++) {
int dup = 0;
for (int k = 0; k < na && !dup; k++)
dup = strcmp(ids[k], builtin[i]) == 0;
if (dup)
continue;
memcpy(out[n].id, builtin[i], sizeof(out[n].id));
out[n].id[sizeof(out[n].id) - 1] = 0;
out[n].api = 0;
cst_model_info(cfg->provider, builtin[i], &out[n].info);
n++;
}
free(ids);
return n;
}
int cst_ps_verify_save(const char *provider, const char *api_base,
const char *key, char ids[][128], int max, int *nids,
char *err, size_t err_sz) {
err[0] = 0;
char(*own)[128] = NULL;
if (!ids) {
own = calloc(CST_PS_MAX_MODELS, sizeof(*own));
if (!own) {
snprintf(err, err_sz, "out of memory");
return CST_PS_UNVERIFIED | CST_PS_UNSAVED;
}
ids = own;
max = CST_PS_MAX_MODELS;
}
int r = cst_ps_fetch(provider, api_base, key, ids, max, err, err_sz);
free(own);
if (nids)
*nids = r;
if (r == CST_MODELS_AUTH)
return CST_MODELS_AUTH;
int flags = r < 0 ? CST_PS_UNVERIFIED : 0;
if (r >= 0)
err[0] = 0;
if (cst_creds_save(provider, key) != 0)
flags |= CST_PS_UNSAVED;
return flags;
}
void cst_ps_apply_provider(cst_agent_config_t *cfg, const cst_provider_t *p,
const char *key) {
cst_settings_t s;
cst_settings_load(&s);
const char *model = cst_settings_model(&s, p->name);
snprintf(cfg->provider, sizeof(cfg->provider), "%s", p->name);
snprintf(cfg->api_base, sizeof(cfg->api_base), "%s", p->api_base);
snprintf(cfg->model, sizeof(cfg->model), "%s", model ? model : p->model);
snprintf(cfg->api_key, sizeof(cfg->api_key), "%s", key);
cst_settings_remember(cfg);
}
int cst_ps_switch_provider(cst_agent_config_t *cfg, const char *name,
const char *key, char *err, size_t err_sz) {
err[0] = 0;
const cst_provider_t *p = name ? cst_provider_find(name) : NULL;
if (!p) {
snprintf(err, err_sz, "unknown provider '%.64s'", name ? name : "");
return CST_PS_UNKNOWN;
}
char k[512];
int flags = 0;
char(*ids)[128] = calloc(CST_PS_MAX_MODELS, sizeof(*ids));
int nids = -1;
if (key && key[0]) {
snprintf(k, sizeof(k), "%s", key);
flags = cst_ps_verify_save(p->name, p->api_base, k, ids,
ids ? CST_PS_MAX_MODELS : 0, &nids, err,
err_sz);
if (flags < 0) {
memset(k, 0, sizeof(k));
free(ids);
return flags;
}
} else if (cst_creds_resolve(p->name, k, sizeof(k), NULL) != 0) {
free(ids);
snprintf(err, err_sz, "%s needs an API key", p->name);
return CST_PS_NEED_KEY;
}
cst_ps_apply_provider(cfg, p, k);
memset(k, 0, sizeof(k));
if (!cfg->model[0] && ids) {
/* a router preset: the first model the API lists */
int fresh = 0;
if (nids <= 0)
nids = cst_ps_cached(p->name, ids, CST_PS_MAX_MODELS, &fresh);
if (nids <= 0) {
char e2[256];
nids = cst_ps_fetch(p->name, cfg->api_base, cfg->api_key, ids,
CST_PS_MAX_MODELS, e2, sizeof(e2));
}
if (nids > 0) {
snprintf(cfg->model, sizeof(cfg->model), "%s", ids[0]);
cst_settings_remember(cfg);
}
}
free(ids);
return flags;
}
int cst_ps_set_key(cst_agent_config_t *cfg, const char *key, char *err,
size_t err_sz) {
if (!key || !key[0]) {
snprintf(err, err_sz, "empty key");
return CST_PS_NEED_KEY;
}
int r = cst_ps_verify_save(cfg->provider, cfg->api_base, key, NULL, 0, NULL,
err, err_sz);
if (r >= 0)
snprintf(cfg->api_key, sizeof(cfg->api_key), "%s", key);
return r;
}
+427 -22
View File
@@ -19,6 +19,9 @@
#elif defined(__APPLE__)
#define CST_HAVE_SEATBELT 1
#include <stdint.h>
#elif defined(_WIN32)
#include <windows.h>
#include <sddl.h>
#endif
#if defined(__linux__) && (defined(__x86_64__) || defined(__aarch64__))
@@ -83,8 +86,6 @@ static int g_abi;
#define LL_FS_MAKE_SYM (1ULL << 12)
#define LL_FS_REFER (1ULL << 13) /* ABI 2 */
#define LL_FS_TRUNCATE (1ULL << 14) /* ABI 3 */
#define LL_NET_BIND_TCP (1ULL << 0) /* ABI 4 */
#define LL_NET_CONNECT_TCP (1ULL << 1) /* ABI 4 */
struct ll_ruleset_attr {
uint64_t handled_access_fs;
@@ -134,9 +135,7 @@ static int ll_prepare(const char *ws, const char *tmp, cst_sandbox_plan_t *p) {
int abi = g_abi;
struct ll_ruleset_attr attr;
memset(&attr, 0, sizeof(attr));
attr.handled_access_fs = ll_write_rights(abi);
if (abi >= 4)
attr.handled_access_net = LL_NET_BIND_TCP | LL_NET_CONNECT_TCP;
attr.handled_access_fs = ll_write_rights(abi); /* network stays open */
int fd = (int)syscall(__NR_landlock_create_ruleset, &attr, sizeof(attr), 0);
if (fd < 0)
return -1;
@@ -190,8 +189,7 @@ static int sb_prepare(const char *ws, const char *tmp, cst_sandbox_plan_t *p) {
"(version 1)(allow default)(deny file-write*)"
"(allow file-write* (subpath \"%s\") (subpath \"%s\") "
"(literal \"/dev/null\") (literal \"/dev/tty\") "
"(literal \"/dev/dtracehelper\"))"
"(deny network*)",
"(literal \"/dev/dtracehelper\"))",
qws, qtmp);
return 0;
}
@@ -287,9 +285,8 @@ static const int SC_NOTIFY[] = {
#endif
};
/* Foreign ABIs are refused, IPv4/IPv6 sockets fail with EACCES, opens
* without write flags pass untouched, write-type calls go to the
* supervisor. */
/* Foreign ABIs are refused, opens without write flags pass untouched,
* write-type calls go to the supervisor. */
static void sc_build(sc_prog_t *p) {
p->n = 0;
SC_LD(offsetof(struct seccomp_data, arch));
@@ -300,12 +297,6 @@ static void sc_build(sc_prog_t *p) {
sc_emit(p, BPF_JMP | BPF_JGE | BPF_K, 0, 1, 0x40000000U); /* x32 */
SC_RET(SECCOMP_RET_ERRNO | EPERM);
#endif
SC_JEQ(__NR_socket, 0, 5);
SC_LD(SC_ARG(0));
SC_JEQ(AF_INET, 2, 0);
SC_JEQ(AF_INET6, 1, 0);
SC_RET(SECCOMP_RET_ALLOW);
SC_RET(SECCOMP_RET_ERRNO | EACCES);
SC_JEQ(__NR_io_uring_setup, 0, 1);
SC_RET(SECCOMP_RET_ERRNO | EPERM);
SC_JEQ(__NR_openat, 0, 4);
@@ -388,6 +379,47 @@ static ssize_t sc_peek(int pid, uint64_t addr, void *out, size_t n) {
return k;
}
/* Path decisions need sc_peek on the sandboxed child. Yama
* kernel.yama.ptrace_scope=2 (and no CAP_SYS_PTRACE) denies that, so a
* listener-only probe would report seccomp available while every write
* fails with EACCES. */
static int sc_can_peek(void) {
static const char want[] = "cst-peek-ok";
int sp[2];
if (pipe2(sp, O_CLOEXEC) != 0)
return 0;
pid_t pid = fork();
if (pid == 0) {
char marker[] = "cst-peek-ok";
uint64_t addr = (uint64_t)(uintptr_t)marker;
close(sp[0]);
if (write(sp[1], &addr, sizeof(addr)) != (ssize_t)sizeof(addr))
_exit(1);
close(sp[1]);
pause();
_exit(0);
}
close(sp[1]);
if (pid < 0) {
close(sp[0]);
return 0;
}
uint64_t addr = 0;
int ok = 0;
if (read(sp[0], &addr, sizeof(addr)) == (ssize_t)sizeof(addr) && addr) {
char got[sizeof(want)];
if (sc_peek((int)pid, addr, got, sizeof(got)) == (ssize_t)sizeof(got) &&
memcmp(got, want, sizeof(got)) == 0)
ok = 1;
}
close(sp[0]);
kill(pid, SIGKILL);
int st = 0;
while (waitpid(pid, &st, 0) < 0 && errno == EINTR)
;
return ok;
}
static int sc_read_mem(int pid, uint64_t addr, void *out, size_t n) {
size_t got = 0;
while (got < n) {
@@ -522,9 +554,10 @@ static void sc_rewrite(int pid, const char *in, char *out, size_t n) {
static int sc_abs_ok(const sc_state_t *s, const char *full, int follow,
int strict) {
char path[OS_PATH_MAX], real[OS_PATH_MAX], loc[2 * OS_PATH_MAX + 2];
if (strlen(full) >= sizeof(path))
size_t flen = strlen(full);
if (flen >= sizeof(path))
return 0;
snprintf(path, sizeof(path), "%s", full);
memcpy(path, full, flen + 1);
size_t n = strlen(path);
while (n > 1 && path[n - 1] == '/')
path[--n] = 0;
@@ -859,11 +892,346 @@ static int sc_probe(void) {
int st = 0;
while (waitpid(pid, &st, 0) < 0 && errno == EINTR)
;
return ok && WIFEXITED(st) && WEXITSTATUS(st) == 0;
return ok && WIFEXITED(st) && WEXITSTATUS(st) == 0 && sc_can_peek();
}
#endif /* CST_HAVE_SECCOMP */
/* --- Windows: Low integrity level ----------------------------------------- */
#ifdef _WIN32
#define LW_LABEL "S:(ML;OICI;NW;;;LW)"
#define LW_RECORD "sandbox-labels"
/* Same user, no admin privileges, Low IL. No LUA_TOKEN: a deny-only
* Administrators group cannot open an admin's (SSH) window station and the
* child fails in DLL init; the integrity check does not need it. */
static HANDLE lw_token(void) {
HANDLE self = NULL, tok = NULL;
if (!OpenProcessToken(GetCurrentProcess(),
TOKEN_DUPLICATE | TOKEN_QUERY | TOKEN_ADJUST_DEFAULT |
TOKEN_ASSIGN_PRIMARY,
&self))
return NULL;
BOOL ok = CreateRestrictedToken(self, DISABLE_MAX_PRIVILEGE, 0, NULL, 0,
NULL, 0, NULL, &tok);
CloseHandle(self);
if (!ok)
return NULL;
PSID low = NULL;
if (!ConvertStringSidToSidW(L"S-1-16-4096", &low)) {
CloseHandle(tok);
return NULL;
}
TOKEN_MANDATORY_LABEL tml;
memset(&tml, 0, sizeof(tml));
tml.Label.Sid = low;
tml.Label.Attributes = SE_GROUP_INTEGRITY;
ok = SetTokenInformation(tok, TokenIntegrityLevel, &tml,
(DWORD)(sizeof(tml) + GetLengthSid(low)));
LocalFree(low);
if (!ok) {
CloseHandle(tok);
return NULL;
}
return tok;
}
static int lw_record_path(char *out, size_t n) {
char dir[OS_PATH_MAX];
if (os_config_dir(dir, sizeof(dir)) != 0)
return -1;
return snprintf(out, n, "%s/%s", dir, LW_RECORD) < (int)n ? 0 : -1;
}
static char *lw_read_record(void) {
char path[OS_PATH_MAX];
if (lw_record_path(path, sizeof(path)) != 0)
return NULL;
FILE *f = fopen(path, "rb");
if (!f)
return NULL;
size_t cap = 4096, len = 0;
char *s = malloc(cap);
size_t k;
while (s && (k = fread(s + len, 1, cap - len - 1, f)) > 0) {
len += k;
if (cap - len < 2) {
char *g = realloc(s, cap * 2);
if (!g) {
free(s);
s = NULL;
break;
}
s = g;
cap *= 2;
}
}
fclose(f);
if (s)
s[len] = 0;
return s;
}
/* dir is a recorded directory or inside one (the label is inherited). */
static int lw_recorded(const char *dir) {
char *s = lw_read_record();
int found = 0;
for (char *line = s; line && *line && !found;) {
char *nl = strchr(line, '\n');
if (nl)
*nl = 0;
char *tab = strchr(line, '\t');
if (tab)
*tab = 0;
found = line[0] && os_path_under(line, dir);
line = nl ? nl + 1 : NULL;
}
free(s);
return found;
}
static int lw_record_add(const char *dir, const char *old_sddl) {
char path[OS_PATH_MAX];
if (lw_record_path(path, sizeof(path)) != 0)
return -1;
FILE *f = fopen(path, "ab");
if (!f)
return -1;
fprintf(f, "%s\t%s\n", dir, old_sddl);
return fclose(f) == 0 ? 0 : -1;
}
static int lw_same_dir(const char *a, const char *b) {
return a && b && a[0] && b[0] && os_path_under(a, b) && os_path_under(b, a);
}
/* An environment directory in the form os_realpath gives the workspace
* (long names, '/'), or NULL. */
static const char *lw_env_dir(const char *name, char *buf, size_t n) {
const char *v = getenv(name);
if (!v || !v[0] || os_realpath(v, buf, n) != 0)
return NULL;
os_path_norm_slashes(buf);
return buf;
}
/* Labeling a whole drive, the profile or system folders would open far more
* than a project to every sandboxed command. */
static int lw_refused(const char *dir) {
size_t n = strlen(dir);
while (n > 0 && (dir[n - 1] == '/' || dir[n - 1] == '\\'))
n--;
if (n <= 2)
return 1;
static const char *const SELF[] = {"USERPROFILE", "APPDATA",
"LOCALAPPDATA", "TEMP"};
static const char *const TREE[] = {"SystemRoot", "ProgramFiles",
"ProgramFiles(x86)", "ProgramData"};
char buf[OS_PATH_MAX];
for (size_t i = 0; i < sizeof(SELF) / sizeof(SELF[0]); i++)
if (lw_same_dir(lw_env_dir(SELF[i], buf, sizeof(buf)), dir))
return 1;
for (size_t i = 0; i < sizeof(TREE) / sizeof(TREE[0]); i++) {
const char *v = lw_env_dir(TREE[i], buf, sizeof(buf));
if (v && os_path_under(v, dir))
return 1;
}
return 0;
}
static int lw_prepare(const char *ws, cst_sandbox_plan_t *p) {
static int creds_done, tmp_done;
char buf[OS_PATH_MAX + 256];
if (!ws || !ws[0])
return -1;
if (!creds_done) {
/* files saved by older versions have no no-read-up label yet */
char dir[OS_PATH_MAX];
if (os_config_dir(dir, sizeof(dir)) == 0 &&
snprintf(buf, sizeof(buf), "%s/credentials", dir) < (int)sizeof(buf)) {
os_stat_t st;
if (os_lstat(buf, &st) != 0 || os_chmod_private(buf) == 0)
creds_done = 1;
}
}
char tmp[OS_PATH_MAX];
if (os_temp_dir(tmp, sizeof(tmp)) != 0 ||
snprintf(buf, sizeof(buf), "%s/cst-agent-sandbox", tmp) >=
(int)sizeof(tmp))
return -1;
memcpy(tmp, buf, strlen(buf) + 1);
if (!tmp_done) {
os_mkdir_p(tmp, 0700);
if (os_win_set_label(tmp, LW_LABEL) != 0)
return -1;
tmp_done = 1;
}
if (!lw_recorded(ws)) {
char old[512];
if (lw_refused(ws) || os_win_get_label(ws, old, sizeof(old)) != 0 ||
os_win_set_label(ws, LW_LABEL) != 0)
return -1;
lw_record_add(ws, old);
snprintf(buf, sizeof(buf),
"[cst: sandbox: labeled %s Low integrity so sandboxed commands "
"can write there; /sandbox reset-labels undoes it]\n",
ws);
p->note = strdup(buf);
}
HANDLE tok = lw_token();
if (!tok) {
free(p->note);
p->note = NULL;
return -1;
}
p->token = tok;
for (char *c = tmp; *c; c++)
if (*c == '/')
*c = '\\';
p->tmpdir = strdup(tmp);
return p->tmpdir ? 0 : -1;
}
static int lw_run(const char *ws, const char *cmd, size_t max_out,
const cst_sandbox_plan_t *p, os_run_result_t *r) {
size_t tl = strlen(p->tmpdir) + 8;
char *te = malloc(tl), *tm = malloc(tl);
int rc = -1;
if (te && tm) {
snprintf(te, tl, "TEMP=%s", p->tmpdir);
snprintf(tm, tl, "TMP=%s", p->tmpdir);
const char *env[] = {te, tm, NULL};
rc = os_run_shell_token(ws, cmd, max_out, r, p->token, env);
}
free(te);
free(tm);
if (rc == 0 && p->note) {
size_t nl = strlen(p->note);
char *o = malloc(nl + r->out_len + 1);
if (o) {
memcpy(o, p->note, nl);
if (r->out)
memcpy(o + nl, r->out, r->out_len);
o[nl + r->out_len] = 0;
free(r->out);
r->out = o;
r->out_len += nl;
}
}
return rc;
}
#endif /* _WIN32 */
char *cst_sandbox_labels(void) {
#ifdef _WIN32
char *s = lw_read_record();
if (!s)
return NULL;
size_t o = 0;
for (char *line = s; *line;) {
char *nl = strchr(line, '\n');
size_t len = nl ? (size_t)(nl - line) : strlen(line);
size_t path = strcspn(line, "\t\n");
if (path) {
memmove(s + o, line, path);
o += path;
s[o++] = '\n';
}
line += len + (nl ? 1 : 0);
}
s[o] = 0;
if (!o) {
free(s);
return NULL;
}
return s;
#else
return NULL;
#endif
}
int cst_sandbox_reset_labels(char **report) {
if (report)
*report = NULL;
#ifdef _WIN32
char *s = lw_read_record();
size_t cap = 256, len = 0;
char *rep = malloc(cap), *keep = NULL;
size_t kcap = 0, klen = 0;
int ok = 0, bad = 0;
if (rep)
rep[0] = 0;
for (char *line = s; line && *line;) {
char *nl = strchr(line, '\n');
if (nl)
*nl = 0;
char *tab = strchr(line, '\t');
const char *old = tab ? tab + 1 : "S:";
if (tab)
*tab = 0;
if (line[0]) {
os_stat_t st;
int gone = os_lstat(line, &st) != 0;
int done = gone || os_win_set_label(line, old[0] ? old : "S:") == 0;
char msg[OS_PATH_MAX + 64];
int m = snprintf(msg, sizeof(msg), "%s %s\n",
gone ? "gone " : done ? "restored" : "FAILED ",
line);
if (m > 0 && rep) {
while (len + (size_t)m + 1 > cap) {
char *g = realloc(rep, cap *= 2);
if (!g)
break;
rep = g;
}
if (len + (size_t)m + 1 <= cap) {
memcpy(rep + len, msg, (size_t)m + 1);
len += (size_t)m;
}
}
if (done) {
ok++;
} else {
bad++;
size_t need = strlen(line) + strlen(old) + 3;
if (klen + need + 1 > kcap) {
char *g = realloc(keep, kcap = (klen + need + 1) * 2);
if (g)
keep = g;
}
if (keep && klen + need + 1 <= kcap)
klen += (size_t)snprintf(keep + klen, kcap - klen, "%s\t%s\n",
line, old);
}
}
line = nl ? nl + 1 : NULL;
}
free(s);
char path[OS_PATH_MAX];
if (lw_record_path(path, sizeof(path)) == 0) {
if (klen) {
FILE *f = fopen(path, "wb");
if (f) {
fwrite(keep, 1, klen, f);
fclose(f);
}
} else {
remove(path);
}
}
free(keep);
if (report)
*report = rep;
else
free(rep);
return bad ? -1 : ok;
#else
return 0;
#endif
}
/* --- common --------------------------------------------------------------- */
int cst_sandbox_probe(void) {
@@ -891,6 +1259,15 @@ int cst_sandbox_probe(void) {
#endif
#ifdef CST_HAVE_SEATBELT
kind = CST_SANDBOX_SEATBELT;
#endif
#ifdef _WIN32
if (!e || strcmp(e, "lowil") == 0) {
HANDLE tok = lw_token();
if (tok) {
CloseHandle(tok);
kind = CST_SANDBOX_LOWIL;
}
}
#endif
(void)want_ll;
(void)want_sc;
@@ -917,6 +1294,8 @@ const char *cst_sandbox_name(int kind) {
return "seatbelt";
case CST_SANDBOX_SECCOMP:
return "seccomp";
case CST_SANDBOX_LOWIL:
return "lowil";
default:
return "none";
}
@@ -939,6 +1318,10 @@ int cst_sandbox_prepare(const char *workspace, const char *tmpdir,
#ifdef CST_HAVE_SECCOMP
if (kind == CST_SANDBOX_SECCOMP)
rc = sc_prepare(workspace, tmpdir, plan);
#endif
#ifdef _WIN32
if (kind == CST_SANDBOX_LOWIL && (rc = lw_prepare(workspace, plan)) != 0)
cst_sandbox_release(plan);
#endif
(void)workspace;
(void)tmpdir;
@@ -988,6 +1371,15 @@ void cst_sandbox_release(cst_sandbox_plan_t *plan) {
#ifdef CST_HAVE_SECCOMP
sc_free(plan->sc);
#endif
#ifdef _WIN32
if (plan->token)
CloseHandle(plan->token);
#endif
plan->token = NULL;
free(plan->tmpdir);
plan->tmpdir = NULL;
free(plan->note);
plan->note = NULL;
plan->sc = NULL;
free(plan->profile);
plan->profile = NULL;
@@ -1004,6 +1396,12 @@ int cst_sandbox_run_shell(const char *workspace, const char *cmd,
if (cst_sandbox_prepare(workspace, tmp, &plan) != 0) {
memset(r, 0, sizeof(*r));
const char *msg = "[cst: sandbox unavailable; command not run]\n";
#ifdef _WIN32
if (workspace && !lw_recorded(workspace) && lw_refused(workspace))
msg = "[cst: sandbox: will not label a drive root, the profile or a "
"system folder writable; command not run. Start cst-agent in a "
"project folder]\n";
#endif
r->out = malloc(strlen(msg) + 1);
if (r->out) {
memcpy(r->out, msg, strlen(msg) + 1);
@@ -1012,8 +1410,12 @@ int cst_sandbox_run_shell(const char *workspace, const char *cmd,
r->exit_code = 126;
return 0;
}
#ifdef _WIN32
int rc = lw_run(workspace, cmd, max_out, &plan, r);
#else
int rc = os_run_shell_ex(workspace, cmd, max_out, r, cst_sandbox_enter,
&plan);
#endif
cst_sandbox_release(&plan);
return rc;
}
@@ -1032,13 +1434,16 @@ int cst_sandbox_blocked(const char *out, size_t n, int exit_code) {
static const char *const SIGNS[] = {
"Permission denied", "Read-only file system", "Operation not permitted",
"EACCES", "EPERM", "EROFS",
"[cst: sandbox", "Network is unreachable",
/* curl when the socket itself was refused (seccomp) */
"Could not connect to server", "Failed to connect to",
"[cst: sandbox",
/* localized strerror(EACCES/EPERM/EROFS) */
"权限不够", "不允许的操作", "只读文件系统", "權限不足",
"拒絕不符權限的操作", "許可がありません", "Keine Berechtigung",
"Permission non accord", "Permiso denegado", "Отказано в доступе",
/* Windows ERROR_ACCESS_DENIED from cmd, PowerShell and Win32 tools */
"Access is denied", "Access to the path", "UnauthorizedAccess",
"拒绝访问", "存取被拒", "Zugriff verweigert", "Accès refusé",
/* Cygwin/MSYS2 programs cannot initialize at Low IL */
"DLL initialization routine failed", "0xc0000142",
};
for (size_t i = 0; i < sizeof(SIGNS) / sizeof(SIGNS[0]); i++)
if (has(out, n, SIGNS[i]))
+120 -4
View File
@@ -7,6 +7,8 @@
#include "cJSON.h"
#include "cst_agent.h"
#include "cst_attach.h"
#include "cst_util.h"
#include <pthread.h>
#include <stdio.h>
@@ -97,7 +99,11 @@ static void msg_free(cst_message_t *m) {
free(m->content);
free(m->tool_calls_json);
free(m->meta_json);
m->content = m->tool_calls_json = m->meta_json = NULL;
free(m->reasoning);
cst_attachments_free(m->attach, m->nattach);
m->content = m->tool_calls_json = m->meta_json = m->reasoning = NULL;
m->attach = NULL;
m->nattach = 0;
}
static int session_grow(cst_session_t *s) {
@@ -127,6 +133,16 @@ static int session_push_mem(cst_session_t *s, const cst_message_t *src) {
d->tool_calls_json = xstrdup(src->tool_calls_json);
if (src->meta_json && src->meta_json[0])
d->meta_json = xstrdup(src->meta_json);
if (src->reasoning && src->reasoning[0])
d->reasoning = xstrdup(src->reasoning);
if (src->attach && src->nattach > 0) {
d->attach = cst_attachments_dup(src->attach, src->nattach);
if (!d->attach) {
msg_free(d);
return -1;
}
d->nattach = src->nattach;
}
s->nmsgs++;
return 0;
}
@@ -137,7 +153,7 @@ static const char *jstr(const cJSON *o, const char *key) {
}
static void load_existing(cst_session_t *s) {
FILE *rp = fopen(s->path, "r");
FILE *rp = fopen(s->path, "rb");
if (!rp)
return;
char *line = NULL;
@@ -174,9 +190,14 @@ static void load_existing(cst_session_t *s) {
if (cJSON_IsObject(mo))
meta = cJSON_PrintUnformatted(mo);
m.meta_json = meta;
m.reasoning = (char *)jstr(o, "reasoning");
const cJSON *at = cJSON_GetObjectItemCaseSensitive(o, "attachments");
if (cJSON_IsArray(at))
m.attach = cst_attachments_from_json(at, &m.nattach);
session_push_mem(s, &m);
free(tcj);
free(meta);
cst_attachments_free(m.attach, m.nattach);
cJSON_Delete(o);
}
free(line);
@@ -196,7 +217,7 @@ cst_session_t *cst_session_open(const char *path) {
pthread_mutex_init(&s->mu, NULL);
snprintf(s->path, sizeof(s->path), "%s", path ? path : "session.jsonl");
load_existing(s);
s->fp = fopen(s->path, "a+");
s->fp = fopen(s->path, "a+b");
if (!s->fp || session_grow(s) != 0) {
if (s->fp)
fclose(s->fp);
@@ -249,6 +270,8 @@ static int write_line(FILE *fp, const cst_message_t *msg) {
cJSON_AddStringToObject(o, "tool_calls_json", msg->tool_calls_json);
}
}
if (msg->reasoning && msg->reasoning[0])
cJSON_AddStringToObject(o, "reasoning", msg->reasoning);
if (msg->meta_json && msg->meta_json[0]) {
cJSON *mo = cJSON_Parse(msg->meta_json);
if (cJSON_IsObject(mo))
@@ -256,6 +279,28 @@ static int write_line(FILE *fp, const cst_message_t *msg) {
else
cJSON_Delete(mo);
}
if (msg->attach && msg->nattach > 0) {
cJSON *arr = cJSON_CreateArray();
for (int i = 0; i < msg->nattach; i++) {
const cst_attachment_t *a = &msg->attach[i];
cJSON *it = cJSON_CreateObject();
cJSON_AddStringToObject(it, "id", a->id);
cJSON_AddStringToObject(it, "name", a->name);
cJSON_AddStringToObject(it, "mime", a->mime);
cJSON_AddStringToObject(it, "rel", a->rel);
cJSON_AddNumberToObject(it, "bytes", (double)a->bytes);
if (a->width > 0 && a->height > 0) {
cJSON_AddNumberToObject(it, "width", a->width);
cJSON_AddNumberToObject(it, "height", a->height);
}
if (a->detail[0])
cJSON_AddStringToObject(it, "detail", a->detail);
if (a->origin[0])
cJSON_AddStringToObject(it, "origin", a->origin);
cJSON_AddItemToArray(arr, it);
}
cJSON_AddItemToObject(o, "attachments", arr);
}
char *txt = cJSON_PrintUnformatted(o);
cJSON_Delete(o);
if (!txt)
@@ -295,7 +340,7 @@ static int reopen_truncate(cst_session_t *s) {
fclose(s->fp);
s->fp = NULL;
}
s->fp = fopen(s->path, "w+");
s->fp = fopen(s->path, "w+b");
return s->fp ? 0 : -1;
}
@@ -310,6 +355,7 @@ int cst_session_clear(cst_session_t *s, int keep_system) {
if (s->msgs[i].role == CST_ROLE_SYSTEM) {
sys.role = CST_ROLE_SYSTEM;
sys.content = xstrdup(s->msgs[i].content ? s->msgs[i].content : "");
sys.meta_json = xstrdup(s->msgs[i].meta_json);
have_sys = sys.content != NULL;
break;
}
@@ -322,6 +368,7 @@ int cst_session_clear(cst_session_t *s, int keep_system) {
write_line(s->fp, &sys);
}
free(sys.content);
free(sys.meta_json);
pthread_mutex_unlock(&s->mu);
return rc;
}
@@ -339,6 +386,11 @@ int cst_session_rewrite(cst_session_t *s, const cst_message_t *msgs, int n) {
copy[i].tool_calls_json =
msgs[i].tool_calls_json ? strdup(msgs[i].tool_calls_json) : NULL;
copy[i].meta_json = msgs[i].meta_json ? strdup(msgs[i].meta_json) : NULL;
copy[i].reasoning = msgs[i].reasoning ? strdup(msgs[i].reasoning) : NULL;
copy[i].attach = msgs[i].attach && msgs[i].nattach > 0
? cst_attachments_dup(msgs[i].attach, msgs[i].nattach)
: NULL;
copy[i].nattach = copy[i].attach ? msgs[i].nattach : 0;
}
pthread_mutex_lock(&s->mu);
free_all(s);
@@ -352,7 +404,71 @@ int cst_session_rewrite(cst_session_t *s, const cst_message_t *msgs, int n) {
free(copy[i].content);
free(copy[i].tool_calls_json);
free(copy[i].meta_json);
free(copy[i].reasoning);
cst_attachments_free(copy[i].attach, copy[i].nattach);
}
free(copy);
return rc;
}
int cst_message_is_tool_result(const cst_message_t *m) {
return m && (m->role == CST_ROLE_TOOL ||
(m->role == CST_ROLE_USER && m->tool_call_id[0]));
}
static void md_fence(cst_buf_t *b, const char *s, size_t max) {
size_t n = strlen(s);
if (n > max)
n = cst_utf8_floor(s, max);
cst_buf_puts(b, "````\n");
cst_buf_append(b, s, n);
cst_buf_puts(b, n < strlen(s) ? "\n…(truncated)\n````\n\n" : "\n````\n\n");
}
char *cst_session_export_md(cst_session_t *s, const char *title,
const char *sid, const char *model) {
if (!s)
return NULL;
cst_buf_t b = {0};
cst_buf_printf(&b, "# %s\n\nsession `%s` · model `%s`\n\n",
title ? title : "", sid ? sid : "", model ? model : "");
cst_session_lock(s);
for (int i = 0; i < cst_session_count(s); i++) {
const cst_message_t *m = cst_session_get(s, i);
const char *c = m->content ? m->content : "";
if (m->role == CST_ROLE_SYSTEM)
continue;
if (cst_message_is_tool_result(m)) {
cst_buf_printf(&b, "**Tool result** `%s`\n\n", m->tool_name);
md_fence(&b, c, 4000);
} else if (cst_message_is_note(m)) {
cst_buf_puts(&b, "> ");
cst_buf_append(&b, c, strcspn(c, "\n"));
cst_buf_puts(&b, "\n\n");
} else if (m->role == CST_ROLE_USER) {
cst_buf_puts(&b, "## User\n\n");
cst_buf_puts(&b, c);
cst_buf_puts(&b, "\n\n");
for (int k = 0; k < m->nattach; k++) {
char d[128];
cst_attach_describe(&m->attach[k], d, sizeof(d));
cst_buf_printf(&b, "- %s `%s` (%s)\n",
cst_attach_is_image(&m->attach[k]) ? "image"
: "document",
m->attach[k].name, d);
}
if (m->nattach)
cst_buf_puts(&b, "\n");
} else {
cst_buf_puts(&b, "## Assistant\n\n");
cst_buf_puts(&b, c);
cst_buf_puts(&b, "\n\n");
if (m->tool_calls_json && m->tool_calls_json[0]) {
cst_buf_puts(&b, "**Tool calls**\n\n");
md_fence(&b, m->tool_calls_json, 4000);
}
}
}
cst_session_unlock(s);
return cst_buf_steal(&b);
}
+38 -3
View File
@@ -1,4 +1,5 @@
#include "cst_agent.h"
#include "cst_util.h"
#include "os_compat.h"
#include "cJSON.h"
@@ -27,11 +28,17 @@ int cst_settings_load(cst_settings_t *s) {
FILE *fp = fopen(path, "rb");
if (!fp)
return -1;
static _Thread_local char buf[65536];
size_t n = fread(buf, 1, sizeof(buf) - 1, fp);
enum { SETTINGS_MAX = 65536 };
char *buf = malloc(SETTINGS_MAX);
if (!buf) {
fclose(fp);
return -1;
}
size_t n = fread(buf, 1, SETTINGS_MAX - 1, fp);
fclose(fp);
buf[n] = 0;
cJSON *root = cJSON_Parse(buf);
cJSON *root = cJSON_Parse(buf + cst_bom_len(buf, n));
free(buf);
if (!cJSON_IsObject(root)) {
cJSON_Delete(root);
return -1;
@@ -40,6 +47,8 @@ int cst_settings_load(cst_settings_t *s) {
cJSON_GetObjectItemCaseSensitive(root, "provider"));
copy_str(s->effort, sizeof(s->effort),
cJSON_GetObjectItemCaseSensitive(root, "effort"));
copy_str(s->mode, sizeof(s->mode),
cJSON_GetObjectItemCaseSensitive(root, "mode"));
copy_str(s->approval, sizeof(s->approval),
cJSON_GetObjectItemCaseSensitive(root, "approval"));
s->show_thinking =
@@ -79,6 +88,8 @@ int cst_settings_save(const cst_settings_t *s) {
cJSON_AddStringToObject(root, "provider", s->provider);
if (s->effort[0])
cJSON_AddStringToObject(root, "effort", s->effort);
if (s->mode[0])
cJSON_AddStringToObject(root, "mode", s->mode);
if (s->approval[0])
cJSON_AddStringToObject(root, "approval", s->approval);
if (s->show_thinking)
@@ -128,6 +139,21 @@ int cst_approval_parse(const char *s, int *out) {
return -1;
}
static const char *const MODE_NAMES[] = {"coding", "general"};
int cst_mode_valid(const char *m) {
for (size_t i = 0; m && m[0] && i < sizeof(MODE_NAMES) / sizeof(*MODE_NAMES);
i++) {
if (strcmp(m, MODE_NAMES[i]) == 0)
return 1;
}
return 0;
}
const char *cst_mode_name(const char *m) {
return cst_mode_valid(m) ? m : CST_DEFAULT_MODE;
}
const char *cst_settings_model(const cst_settings_t *s, const char *provider) {
for (int i = 0; provider && i < s->nmodels; i++) {
if (strcmp(s->models[i].provider, provider) == 0 && s->models[i].model[0])
@@ -141,6 +167,8 @@ int cst_settings_remember(const cst_agent_config_t *cfg) {
cst_settings_load(&s);
snprintf(s.provider, sizeof(s.provider), "%s", cfg->provider);
snprintf(s.effort, sizeof(s.effort), "%s", cfg->effort);
if (cfg->mode[0])
snprintf(s.mode, sizeof(s.mode), "%s", cfg->mode);
int i = 0;
while (i < s.nmodels && strcmp(s.models[i].provider, cfg->provider) != 0)
i++;
@@ -169,6 +197,13 @@ int cst_settings_remember_thinking(int show_thinking) {
return cst_settings_save(&s);
}
int cst_settings_remember_mode(const char *mode) {
cst_settings_t s;
cst_settings_load(&s);
snprintf(s.mode, sizeof(s.mode), "%s", cst_mode_name(mode));
return cst_settings_save(&s);
}
static void real_ws(const char *ws, char *out, size_t n) {
if (!ws || os_realpath(ws, out, n) != 0)
snprintf(out, n, "%s", ws ? ws : "");
+24 -36
View File
@@ -13,10 +13,7 @@
#include <string.h>
#include <sys/stat.h>
#ifndef _WIN32
#include <dirent.h>
#include <pthread.h>
#endif
#include "cst_tools.h"
#include "cst_util.h"
@@ -166,14 +163,9 @@ static struct {
char source[48];
} g_roots[EXTRA_ROOTS];
static int g_nroots;
#ifndef _WIN32
static pthread_mutex_t g_mu = PTHREAD_MUTEX_INITIALIZER;
#define LOCK() pthread_mutex_lock(&g_mu)
#define UNLOCK() pthread_mutex_unlock(&g_mu)
#else
#define LOCK() ((void)0)
#define UNLOCK() ((void)0)
#endif
void cst_skills_add_root(const char *dir, const char *source) {
if (!dir || !dir[0])
@@ -225,6 +217,7 @@ static char *slurp(const char *path, size_t max, size_t *len) {
}
if (!b.buf)
cst_buf_puts(&b, "");
cst_strip_bom(b.buf, &b.len);
if (len)
*len = b.len;
return cst_buf_steal(&b);
@@ -243,16 +236,15 @@ static int have(const cst_skill_t *v, int n, const char *name) {
static int scan_root(const char *root, const char *source, cst_skill_t *out,
int n, int max) {
#ifndef _WIN32
DIR *d = opendir(root);
os_dir_t *d = os_dir_open(root);
if (!d)
return n;
struct dirent *de;
while (n < max && (de = readdir(d)) != NULL) {
if (de->d_name[0] == '.')
const os_dirent_t *de;
while (n < max && (de = os_dir_next(d)) != NULL) {
if (de->name[0] == '.')
continue;
char dir[OS_PATH_MAX], md[OS_PATH_MAX + 16];
if (snprintf(dir, sizeof(dir), "%s/%s", root, de->d_name) >=
if (snprintf(dir, sizeof(dir), "%s/%s", root, de->name) >=
(int)sizeof(dir))
continue;
snprintf(md, sizeof(md), "%s/SKILL.md", dir);
@@ -262,7 +254,7 @@ static int scan_root(const char *root, const char *source, cst_skill_t *out,
cst_skill_t s;
memset(&s, 0, sizeof(s));
if (!cst_fm_get(text, "name", s.name, sizeof(s.name)) || !s.name[0])
snprintf(s.name, sizeof(s.name), "%.63s", de->d_name);
snprintf(s.name, sizeof(s.name), "%.63s", de->name);
cst_fm_get(text, "description", s.desc, sizeof(s.desc));
if (!s.desc[0]) {
/* first prose line of the body */
@@ -282,13 +274,7 @@ static int scan_root(const char *root, const char *source, cst_skill_t *out,
if (!have(out, n, s.name))
out[n++] = s;
}
closedir(d);
#else
(void)root;
(void)source;
(void)out;
(void)max;
#endif
os_dir_close(d);
return n;
}
@@ -371,25 +357,24 @@ static int find_skill(const char *ws, const char *name, cst_skill_t *out) {
static void list_files(const char *dir, const char *rel, int depth,
cst_buf_t *b, int *count) {
#ifndef _WIN32
char full[OS_PATH_MAX];
if (snprintf(full, sizeof(full), "%s%s%s", dir, rel[0] ? "/" : "", rel) >=
(int)sizeof(full))
return;
DIR *d = opendir(full);
os_dir_t *d = os_dir_open(full);
if (!d)
return;
char *names[256];
int nn = 0;
struct dirent *de;
while (nn < 256 && (de = readdir(d)) != NULL) {
if (de->d_name[0] == '.')
const os_dirent_t *de;
while (nn < 256 && (de = os_dir_next(d)) != NULL) {
if (de->name[0] == '.')
continue;
names[nn] = strdup(de->d_name);
names[nn] = strdup(de->name);
if (names[nn])
nn++;
}
closedir(d);
os_dir_close(d);
for (int i = 1; i < nn; i++) {
for (int j = i; j > 0 && strcmp(names[j - 1], names[j]) > 0; j--) {
char *tmp = names[j];
@@ -416,13 +401,6 @@ static void list_files(const char *dir, const char *rel, int depth,
}
for (int i = 0; i < nn; i++)
free(names[i]);
#else
(void)dir;
(void)rel;
(void)depth;
(void)b;
(void)count;
#endif
}
static char *errf(int *code, const char *fmt, const char *a, const char *b) {
@@ -510,6 +488,16 @@ char *cst_skill_read(const char *ws, const char *name, const char *file,
return cst_buf_steal(&b);
}
const char *cst_init_prompt(void) {
return "Analyze this repository and write an AGENTS.md file at the "
"workspace root for future coding agents. Cover: what the project "
"is, how to build, test and run it (exact commands), the directory "
"layout, coding conventions you can infer from the code, and "
"anything surprising. Explore with list_dir/glob/grep/read_file "
"before writing. Keep it under 150 lines. If AGENTS.md already "
"exists, improve it rather than starting over.";
}
char *cst_skill_prompt(const char *ws, const char *name, const char *task) {
int code = 0;
char *body = cst_skill_read(ws, name, NULL, &code);
+28 -45
View File
@@ -14,9 +14,6 @@
#include <string.h>
#include <sys/stat.h>
#include <time.h>
#ifndef _WIN32
#include <dirent.h>
#endif
#include "cJSON.h"
#include "cst_agent.h"
@@ -43,7 +40,7 @@ static const struct {
"runs commands as needed.",
"", 0},
{"explore",
"Fast read-only explorer: searches and reads the codebase to answer a "
"Fast read-only explorer: searches and reads the workspace to answer a "
"question (where is X, how does Y work, which files do Z). Several can "
"run at once.",
"You only read. Search broadly first (grep, glob, list_dir), then read "
@@ -52,11 +49,11 @@ static const struct {
"you looked for and did not find.",
1},
{"plan",
"Read-only planner: investigates the code involved and returns a "
"Read-only planner: investigates the material involved and returns a "
"step-by-step implementation plan.",
"You only read. Investigate the code involved, then return a concrete "
"plan: the files to change and what to change in each, risks and edge "
"cases, and how to verify. Sketch code only where it removes ambiguity.",
"You only read. Investigate what is involved, then return a concrete "
"plan: what to produce or change and where, risks and edge cases, and "
"how to check the result. Sketch code only where it removes ambiguity.",
1},
};
#define NBUILTIN ((int)(sizeof(BUILTIN) / sizeof(BUILTIN[0])))
@@ -79,6 +76,7 @@ static char *slurp(const char *path, size_t max) {
fclose(fp);
if (!b.buf)
cst_buf_puts(&b, "");
cst_strip_bom(b.buf, &b.len);
return cst_buf_steal(&b);
}
@@ -118,25 +116,24 @@ static int truthy(const char *v) {
static void load_dir(const char *dir, const char *source, cst_agent_role_t *v,
int *n, int max) {
#ifndef _WIN32
DIR *d = opendir(dir);
os_dir_t *d = os_dir_open(dir);
if (!d)
return;
struct dirent *de;
while ((de = readdir(d)) != NULL && *n < max) {
size_t len = strlen(de->d_name);
if (de->d_name[0] == '.' || len < 4 ||
strcmp(de->d_name + len - 3, ".md") != 0)
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL && *n < max) {
size_t len = strlen(de->name);
if (de->name[0] == '.' || len < 4 ||
strcmp(de->name + len - 3, ".md") != 0)
continue;
char p[OS_PATH_MAX];
if (snprintf(p, sizeof(p), "%s/%s", dir, de->d_name) >= (int)sizeof(p))
if (snprintf(p, sizeof(p), "%s/%s", dir, de->name) >= (int)sizeof(p))
continue;
char *text = slurp(p, ROLE_FILE_MAX);
if (!text)
continue;
char name[64], val[1024];
if (!cst_fm_get(text, "name", name, sizeof(name)) || !name[0])
snprintf(name, sizeof(name), "%.*s", (int)(len - 3), de->d_name);
snprintf(name, sizeof(name), "%.*s", (int)(len - 3), de->name);
char clean[64];
cst_tool_sanitize_name(name, clean, sizeof(clean));
int at = find_role(v, *n, clean);
@@ -166,14 +163,7 @@ static void load_dir(const char *dir, const char *source, cst_agent_role_t *v,
r->prompt = strdup(body);
free(text);
}
closedir(d);
#else
(void)dir;
(void)source;
(void)v;
(void)n;
(void)max;
#endif
os_dir_close(d);
}
int cst_roles_list(const char *ws, cst_agent_role_t *out, int max) {
@@ -255,18 +245,16 @@ int cst_subagent_path(const char *ws, const char *parent_id, const char *file,
static int next_file(const char *dir, char *file, size_t fsz) {
pthread_mutex_lock(&g_seq_mu);
int best = 0;
#ifndef _WIN32
DIR *d = opendir(dir);
os_dir_t *d = os_dir_open(dir);
if (d) {
struct dirent *de;
while ((de = readdir(d)) != NULL) {
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL) {
int k = 0;
if (sscanf(de->d_name, "sub-%d.jsonl", &k) == 1 && k > best)
if (sscanf(de->name, "sub-%d.jsonl", &k) == 1 && k > best)
best = k;
}
closedir(d);
os_dir_close(d);
}
#endif
int rc = -1;
for (int k = best + 1; k < best + 100; k++) {
char p[OS_PATH_MAX];
@@ -276,7 +264,7 @@ static int next_file(const char *dir, char *file, size_t fsz) {
struct stat st;
if (stat(p, &st) == 0)
continue;
FILE *fp = fopen(p, "a");
FILE *fp = fopen(p, "ab");
if (fp) {
fclose(fp);
rc = 0;
@@ -302,7 +290,7 @@ static void write_meta(const char *dir, const char *file, const char *role,
cJSON_AddNumberToObject(o, "ms", ms);
char *txt = cJSON_PrintUnformatted(o);
cJSON_Delete(o);
FILE *fp = txt ? fopen(p, "w") : NULL;
FILE *fp = txt ? fopen(p, "wb") : NULL;
if (fp) {
fputs(txt, fp);
fputc('\n', fp);
@@ -317,19 +305,18 @@ int cst_subagent_list(const char *ws, const char *parent_id,
if (sub_dir(ws, parent_id, dir, sizeof(dir)) != 0)
return 0;
int n = 0;
#ifndef _WIN32
DIR *d = opendir(dir);
os_dir_t *d = os_dir_open(dir);
if (!d)
return 0;
struct dirent *de;
while ((de = readdir(d)) != NULL && n < max) {
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL && n < max) {
int k = 0;
char tail[8] = "";
if (sscanf(de->d_name, "sub-%d.%7s", &k, tail) != 2 ||
if (sscanf(de->name, "sub-%d.%7s", &k, tail) != 2 ||
strcmp(tail, "json") != 0)
continue;
char p[OS_PATH_MAX];
if (snprintf(p, sizeof(p), "%s/%s", dir, de->d_name) >= (int)sizeof(p))
if (snprintf(p, sizeof(p), "%s/%s", dir, de->name) >= (int)sizeof(p))
continue;
char *txt = slurp(p, 64 * 1024);
cJSON *o = txt ? cJSON_Parse(txt) : NULL;
@@ -358,11 +345,7 @@ int cst_subagent_list(const char *ws, const char *parent_id,
r->mtime = (long long)st.st_mtime;
n++;
}
closedir(d);
#else
(void)out;
(void)max;
#endif
os_dir_close(d);
/* newest first: higher number */
for (int i = 1; i < n; i++)
for (int j = i; j > 0 && atoi(out[j - 1].file + 4) < atoi(out[j].file + 4);
+12 -6
View File
@@ -3,23 +3,29 @@
#endif
#include "cst_textsel.h"
#include "os_compat.h"
#include <stdlib.h>
#include <string.h>
#include <wctype.h>
int cst_utf8_decode(const char *s, size_t avail, wchar_t *wc, size_t *eaten) {
unsigned long cp = 0;
int n;
if (avail == 0)
return -1;
mbstate_t st;
memset(&st, 0, sizeof(st));
size_t n = mbrtowc(wc, s, avail, &st);
if (n == (size_t)-1 || n == (size_t)-2 || n == 0) {
*wc = (unsigned char)s[0];
/* os_utf8_decode(), not mbrtowc(): the latter follows setlocale(), which on
* Windows means the ANSI code page, so UTF-8 text would be decoded as GBK
* and rendered as mojibake. */
n = os_utf8_decode(s, avail, &cp);
if (n <= 0) {
/* Not a valid sequence: fall back to one raw byte, as before. */
*wc = (wchar_t)(unsigned char)s[0];
*eaten = 1;
return 0;
}
*eaten = n;
*wc = (wchar_t)cp;
*eaten = (size_t)n;
return 0;
}
+68
View File
@@ -7,6 +7,7 @@
#include <string.h>
#include "cst_tools.h"
#include "cst_clipboard.h"
#include "cst_util.h"
#include "tools_internal.h"
@@ -106,3 +107,70 @@ int cst_tool_ask_user(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
1);
return tool_set_resultf(r, 0, "user answered: %s", ans);
}
/* The system clipboard: what the user copied elsewhere. Reading needs no
* permission (READ); writing replaces what the user has copied (WRITE, so
* the confirmation frontier applies). */
int cst_tool_clipboard_read(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r) {
(void)ctx;
char err[192];
char *text = NULL;
size_t n = 0;
if (cst_clipboard_text(&text, &n, err, sizeof(err)) != 0)
return tool_set_result(r, err, 1);
long cap = tool_arg_long(c, "max_bytes", 65536);
if (cap > 0 && (size_t)cap < n)
n = (size_t)cap;
cst_buf_t b = {0};
cst_buf_printf(&b, "clipboard text (%zu bytes%s):\n", (size_t)n,
(size_t)cap < (size_t)n ? ", truncated" : "");
cst_buf_append(&b, text, n);
if (b.len && b.buf[b.len - 1] != '\n')
cst_buf_puts(&b, "\n");
free(text);
return tool_take_buf(r, &b, 0);
}
int cst_tool_clipboard_write(const cst_tool_ctx_t *ctx,
const ctl_tool_call_t *c, cst_tool_result_t *r) {
(void)ctx;
size_t n = 0;
const char *text = tool_arg_n(c, "text", &n);
if (!text)
return tool_set_result(r, "error: missing text", 1);
char err[192];
if (cst_clipboard_set_text(text, n, err, sizeof(err)) != 0)
return tool_set_result(r, err, 1);
return tool_set_resultf(r, 0, "copied %zu bytes to the clipboard", n);
}
/* Confirmation preview: what the clipboard would hold afterwards. Clipboard
* text has no diff, so this is the text itself, cut to a readable size. */
#define CLIP_PREVIEW_MAX 2000
int cst_preview_clipboard_write(const cst_tool_ctx_t *ctx,
const ctl_tool_call_t *c, char **out) {
(void)ctx;
size_t n = 0;
const char *text = tool_arg_n(c, "text", &n);
if (!text) {
*out = strdup("error: missing text");
return -1;
}
int lines = 0;
for (size_t i = 0; i < n; i++)
lines += text[i] == '\n';
size_t cut = n;
if (cut > CLIP_PREVIEW_MAX)
cut = cst_utf8_floor(text, CLIP_PREVIEW_MAX);
cst_buf_t b = {0};
cst_buf_printf(&b, "clipboard: replace with %zu bytes, %d line%s\n", n,
lines + (n && text[n - 1] != '\n'), lines == 0 ? "" : "s");
cst_buf_append(&b, text, cut);
if (b.len && b.buf[b.len - 1] != '\n')
cst_buf_puts(&b, "\n");
if (cut < n)
cst_buf_printf(&b, "… (%zu more bytes)\n", n - cut);
*out = cst_buf_steal(&b);
return 0;
}
+98 -77
View File
@@ -12,13 +12,9 @@
#include <string.h>
#include <sys/stat.h>
#ifndef _WIN32
#include <dirent.h>
#include <fnmatch.h>
#include <regex.h>
#include <unistd.h>
#endif
#include "cst_attach.h"
#include "cst_tools.h"
#include "cst_util.h"
#include "os_compat.h"
@@ -154,6 +150,9 @@ int cst_tool_read_file2(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
FILE *fp = fopen(full, "rb");
if (!fp)
return tool_set_resultf(r, 1, "error: cannot open file: %.200s", path);
char bom[3];
if (fread(bom, 1, 3, fp) != 3 || !cst_bom_len(bom, 3))
rewind(fp);
long offset = tool_arg_long(c, "offset", 0);
long limit = tool_arg_long(c, "limit", 0);
@@ -178,6 +177,8 @@ int cst_tool_read_file2(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
return tool_set_resultf(r, 1, "error: %.200s is a binary file (%zu bytes)",
path, n);
}
cst_strip_bom(buf, &n);
n = cst_crlf_to_lf(buf, n); /* edits match LF text and keep CRLF */
cst_tool_result_free(r);
r->out = buf;
r->out_len = n;
@@ -217,6 +218,9 @@ int cst_tool_read_file2(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
}
if (numbers)
cst_buf_printf(&b, "%6ld|", lineno);
if (line.len >= 2 && line.buf[line.len - 2] == '\r' &&
line.buf[line.len - 1] == '\n')
line.buf[--line.len - 1] = '\n';
cst_buf_append(&b, line.buf, line.len);
shown++;
}
@@ -238,6 +242,55 @@ int cst_tool_read_file2(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
return tool_take_buf(r, &b, 0);
}
/* --- view_image ---------------------------------------------------------- */
/* Show an image to the model: the bytes are copied into the session's
* attachment store and travel as a multimodal content part of the result. */
int cst_tool_view_image(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r) {
const char *path = tool_arg(c, "path");
if (!path || !path[0])
return tool_set_result(r, "error: missing path", 1);
const char *session = ctx->agent ? cst_agent_session_path(ctx->agent) : NULL;
if (!session || !session[0])
return tool_set_result(r, "error: no session to store the image in", 1);
char full[OS_PATH_MAX];
if (resolve(ctx, path, full, sizeof(full)) != 0)
return tool_set_resultf(r, 1,
"error: path outside workspace or missing parent: "
"%.200s",
path);
if (is_dir(full))
return tool_set_resultf(r, 1, "error: %.200s is a directory", path);
char err[256];
cst_attachment_t *a = cst_attach_import_file(session, full, err, sizeof(err));
if (!a)
return tool_set_resultf(r, 1, "error: cannot show %.200s: %s", path, err);
if (!cst_attach_is_image(a)) {
cst_attach_free(a);
return tool_set_resultf(
r, 1,
"error: %.200s is not an image; a text file or PDF reaches the model "
"through /attach <path> (a PDF becomes its text layer)",
path);
}
snprintf(a->origin, sizeof(a->origin), "view_image");
const char *d = tool_arg(c, "detail");
if (d && (strcmp(d, "low") == 0 || strcmp(d, "high") == 0))
snprintf(a->detail, sizeof(a->detail), "%s", d);
char desc[128];
cst_attach_describe(a, desc, sizeof(desc));
cst_buf_t out = {0};
cst_buf_printf(&out, "view_image %.200s: %s\n(shown to you as an image)\n",
path, desc);
int rc = tool_take_buf(r, &out, 0);
r->attach = a; /* one entry; freed as a block by cst_tool_result_free */
r->nattach = 1;
return rc;
}
/* --- edit_file ----------------------------------------------------------- */
static size_t count_occ(const char *hay, size_t hn, const char *needle,
@@ -442,6 +495,14 @@ static int plan_edit(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
nwn = n2n;
}
}
} else if (cnt && memchr(nw, '\n', nwn) &&
cst_text_is_crlf(pl->oldc, pl->olen)) {
size_t n2n;
n2 = to_crlf(nw, nwn, &n2n);
if (n2) {
nw = n2;
nwn = n2n;
}
}
int rc = 0;
if (cnt == 0) {
@@ -539,7 +600,6 @@ int cst_preview_edit_file(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
/* --- delete_file / move_file --------------------------------------------- */
static int reject_symlink(const cst_tool_ctx_t *ctx, const char *path) {
#ifndef _WIN32
char root[OS_PATH_MAX], joined[OS_PATH_MAX];
if (os_realpath(ctx->workspace_root, root, sizeof(root)) != 0)
return 1;
@@ -547,13 +607,9 @@ static int reject_symlink(const cst_tool_ctx_t *ctx, const char *path) {
snprintf(joined, sizeof(joined), "%s", path);
else if (os_path_join(joined, sizeof(joined), root, path) != 0)
return 1;
struct stat st;
if (lstat(joined, &st) == 0 && S_ISLNK(st.st_mode))
os_stat_t st;
if (os_lstat(joined, &st) == 0 && st.is_link)
return 1;
#else
(void)ctx;
(void)path;
#endif
return 0;
}
@@ -672,8 +728,6 @@ int cst_tool_move_file(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
/* --- directory walking (list_dir / glob / grep) -------------------------- */
#ifndef _WIN32
static const char *const SKIP_DIRS[] = {
".git", ".hg", ".svn", "node_modules", ".agent", "__pycache__",
".venv", ".cache", ".idea", ".vscode", ".mypy_cache", ".pytest_cache"};
@@ -703,7 +757,7 @@ static void ignore_load(const char *root, ignore_t *ig) {
char p[OS_PATH_MAX];
if (os_path_join(p, sizeof(p), root, ".gitignore") != 0)
return;
FILE *fp = fopen(p, "r");
FILE *fp = fopen(p, "rb");
if (!fp)
return;
char line[512];
@@ -749,7 +803,7 @@ static int ignored(const ignore_t *ig, const char *rel, const char *name,
const ign_rule_t *r = &ig->v[i];
if (r->dir_only && !isdir)
continue;
if (fnmatch(r->pat, r->anchored ? rel : name, 0) == 0)
if (cst_fnmatch(r->pat, r->anchored ? rel : name, 0) == 0)
return 1;
}
return 0;
@@ -758,7 +812,7 @@ static int ignored(const ignore_t *ig, const char *rel, const char *name,
typedef struct walker walker_t;
/* Return non-zero to stop the walk. */
typedef int (*walk_fn)(walker_t *w, const char *rel, const char *full,
const struct stat *st, int depth);
const os_stat_t *st, int depth);
struct walker {
const char *root; /* workspace realpath */
@@ -781,15 +835,13 @@ static void walk_dir(walker_t *w, const char *rel, int depth) {
snprintf(dirfull, sizeof(dirfull), "%s/%s", w->root, rel);
else
snprintf(dirfull, sizeof(dirfull), "%s", w->root);
DIR *d = opendir(dirfull);
os_dir_t *d = os_dir_open(dirfull);
if (!d)
return;
char **names = NULL;
int n = 0, cap = 0;
struct dirent *de;
while ((de = readdir(d)) != NULL) {
if (strcmp(de->d_name, ".") == 0 || strcmp(de->d_name, "..") == 0)
continue;
const os_dirent_t *de;
while ((de = os_dir_next(d)) != NULL) {
if (n == cap) {
cap = cap ? cap * 2 : 32;
char **nv = realloc(names, (size_t)cap * sizeof(char *));
@@ -797,11 +849,11 @@ static void walk_dir(walker_t *w, const char *rel, int depth) {
break;
names = nv;
}
names[n] = strdup(de->d_name);
names[n] = strdup(de->name);
if (names[n])
n++;
}
closedir(d);
os_dir_close(d);
qsort(names, (size_t)n, sizeof(char *), name_cmp);
for (int i = 0; i < n && !w->stop; i++) {
char crel[OS_PATH_MAX], cfull[OS_PATH_MAX];
@@ -810,10 +862,10 @@ static void walk_dir(walker_t *w, const char *rel, int depth) {
int k2 = snprintf(cfull, sizeof(cfull), "%s/%s", w->root, crel);
if (k1 >= (int)sizeof(crel) || k2 >= (int)sizeof(cfull))
continue; /* path too long */
struct stat st;
if (lstat(cfull, &st) != 0)
os_stat_t st;
if (os_lstat(cfull, &st) != 0)
continue;
int isdir = S_ISDIR(st.st_mode);
int isdir = st.is_dir;
if (ignored(&w->ig, crel, names[i], isdir))
continue;
if (w->fn(w, crel, cfull, &st, depth))
@@ -861,21 +913,18 @@ typedef struct {
/* Entries directly inside a directory (for the depth boundary). */
static int dir_entries(const char *full) {
DIR *d = opendir(full);
os_dir_t *d = os_dir_open(full);
if (!d)
return -1;
int n = 0;
struct dirent *de;
while ((de = readdir(d)) != NULL) {
if (strcmp(de->d_name, ".") != 0 && strcmp(de->d_name, "..") != 0)
n++;
}
closedir(d);
while (os_dir_next(d))
n++;
os_dir_close(d);
return n;
}
static int list_cb(walker_t *w, const char *rel, const char *full,
const struct stat *st, int depth) {
const os_stat_t *st, int depth) {
list_ud_t *u = w->ud;
if (u->count >= LIST_MAX) {
if (++u->more >= LIST_COUNT_MAX) {
@@ -888,7 +937,7 @@ static int list_cb(walker_t *w, const char *rel, const char *full,
name = name ? name + 1 : rel;
for (int i = 0; i < depth; i++)
cst_buf_puts(&u->b, " ");
if (S_ISDIR(st->st_mode)) {
if (st->is_dir) {
int k = -1;
if (skip_dir_name(name))
cst_buf_printf(&u->b, "%s/ (skipped)\n", name);
@@ -897,10 +946,10 @@ static int list_cb(walker_t *w, const char *rel, const char *full,
k == 1 ? "y" : "ies");
else
cst_buf_printf(&u->b, "%s/%s\n", name, k == 0 ? " (empty)" : "");
} else if (S_ISLNK(st->st_mode))
} else if (st->is_link)
cst_buf_printf(&u->b, "%s@\n", name);
else
cst_buf_printf(&u->b, "%s (%lld B)\n", name, (long long)st->st_size);
cst_buf_printf(&u->b, "%s (%lld B)\n", name, st->size);
u->count++;
return 0;
}
@@ -974,7 +1023,7 @@ static int seg_match(const char *pat, const char *path) {
pseg[pl] = 0;
memcpy(sseg, path, sl);
sseg[sl] = 0;
if (fnmatch(pseg, sseg, FNM_PERIOD) != 0)
if (cst_fnmatch(pseg, sseg, CST_FNM_PERIOD) != 0)
return 0;
if (!ps)
return !ss;
@@ -994,11 +1043,11 @@ typedef struct {
} glob_ud_t;
static int glob_cb(walker_t *w, const char *rel, const char *full,
const struct stat *st, int depth) {
const os_stat_t *st, int depth) {
(void)full;
(void)depth;
glob_ud_t *u = w->ud;
if (!S_ISREG(st->st_mode))
if (!st->is_reg)
return 0;
const char *sub = rel;
size_t bl = strlen(u->base);
@@ -1009,7 +1058,7 @@ static int glob_cb(walker_t *w, const char *rel, const char *full,
}
const char *name = strrchr(sub, '/');
name = name ? name + 1 : sub;
int ok = u->basename_only ? fnmatch(u->pattern, name, FNM_PERIOD) == 0
int ok = u->basename_only ? cst_fnmatch(u->pattern, name, CST_FNM_PERIOD) == 0
: seg_match(u->pattern, sub);
if (!ok)
return 0;
@@ -1025,7 +1074,7 @@ static int glob_cb(walker_t *w, const char *rel, const char *full,
return 1;
}
u->hits[u->n] = strdup(rel);
u->mtimes[u->n] = (double)st->st_mtime;
u->mtimes[u->n] = st->mtime;
if (u->hits[u->n])
u->n++;
if (u->n >= GLOB_MAX * 20) {
@@ -1109,11 +1158,11 @@ typedef struct {
} files_ud_t;
static int files_cb(walker_t *w, const char *rel, const char *full,
const struct stat *st, int depth) {
const os_stat_t *st, int depth) {
(void)full;
(void)depth;
files_ud_t *u = w->ud;
if (!S_ISREG(st->st_mode))
if (!st->is_reg)
return 0;
if (u->n >= u->max)
return 1;
@@ -1275,16 +1324,16 @@ static void grep_file(grep_ud_t *u, const char *rel, const char *full) {
}
static int grep_cb(walker_t *w, const char *rel, const char *full,
const struct stat *st, int depth) {
const os_stat_t *st, int depth) {
(void)depth;
grep_ud_t *u = w->ud;
if (!S_ISREG(st->st_mode) || st->st_size > READ_FILE_MAX)
if (!st->is_reg || st->size > READ_FILE_MAX)
return 0;
if (u->glob) {
const char *name = strrchr(rel, '/');
name = name ? name + 1 : rel;
int ok = strchr(u->glob, '/') ? fnmatch(u->glob, rel, 0) == 0
: fnmatch(u->glob, name, 0) == 0;
int ok = strchr(u->glob, '/') ? cst_fnmatch(u->glob, rel, 0) == 0
: cst_fnmatch(u->glob, name, 0) == 0;
if (!ok)
return 0;
}
@@ -1391,32 +1440,4 @@ int cst_tool_grep(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
return tool_take_buf(r, &out, 0);
}
#else /* _WIN32: directory tools are not implemented natively */
int cst_tool_list_dir(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r) {
(void)ctx;
(void)c;
return tool_set_result(r, "error: list_dir is not available on Windows; "
"use run_shell", 1);
}
int cst_workspace_files(const char *ws, char ***out, int max) {
(void)ws;
(void)max;
*out = NULL;
return 0;
}
int cst_tool_glob(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r) {
(void)ctx;
(void)c;
return tool_set_result(r, "error: glob is not available on Windows", 1);
}
int cst_tool_grep(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
cst_tool_result_t *r) {
(void)ctx;
(void)c;
return tool_set_result(r, "error: grep is not available on Windows", 1);
}
#endif
+48 -3
View File
@@ -1,4 +1,5 @@
#include "cst_agent.h"
#include "cst_util.h"
#include <stdio.h>
#include <stdlib.h>
@@ -129,9 +130,9 @@ static int parse_hunk_header(const char *p, size_t n, long *a, long *b,
return -1;
}
int cst_patch_apply_buffer(const char *orig, size_t olen, const char *hunks,
size_t hlen, char **out, size_t *out_len,
int *nhunks, char *err, size_t err_sz) {
static int patch_core(const char *orig, size_t olen, const char *hunks,
size_t hlen, char **out, size_t *out_len, int *nhunks,
char *err, size_t err_sz) {
lines_t file = {0}, hl = {0};
lines_t old_seq = {0}, new_seq = {0};
int rc = -1;
@@ -315,3 +316,47 @@ done:
free(new_seq.v);
return rc;
}
/* A CRLF file is patched as LF (the model's hunks usually are) and written
* back as CRLF; a BOM is kept out of the first line's context. */
int cst_patch_apply_buffer(const char *orig, size_t olen, const char *hunks,
size_t hlen, char **out, size_t *out_len,
int *nhunks, char *err, size_t err_sz) {
size_t bom = orig ? cst_bom_len(orig, olen) : 0;
int crlf = orig && cst_text_is_crlf(orig, olen);
if (!bom && !crlf)
return patch_core(orig, olen, hunks, hlen, out, out_len, nhunks, err,
err_sz);
if (out)
*out = NULL;
char *o = malloc(olen - bom + 1), *h = malloc(hlen + 1), *res = NULL;
size_t rlen = 0;
int rc = -1;
if (!o || !h) {
snprintf(err, err_sz, "oom");
goto done;
}
memcpy(o, orig + bom, olen - bom);
o[olen - bom] = 0;
size_t on = olen - bom, hn = hlen;
if (hlen)
memcpy(h, hunks, hlen);
h[hlen] = 0;
if (crlf) {
on = cst_crlf_to_lf(o, on);
hn = cst_crlf_to_lf(h, hn);
}
rc = patch_core(o, on, h, hn, &res, &rlen, nhunks, err, err_sz);
if (rc == 0) {
*out = cst_text_restyle(res, rlen, crlf, bom != 0, out_len);
if (!*out) {
snprintf(err, err_sz, "oom");
rc = -1;
}
}
done:
free(res);
free(o);
free(h);
return rc;
}

Some files were not shown because too many files have changed in this diff Show More