v0.9.6: documents, downloads, PDFs and the phone's own files
Four tools and a document writer, plus the storage half of the phone: until now the picker's answer (a content:// URI) reached the engine as a path, and it said "cannot read" -- the app had no way to hand a phone's file over. - md_to_html (new, src/tools/web.c + cst_web_md_document()): markdown in, one self-contained HTML document out -- doctype, utf-8, viewport, escaped title, a small stylesheet; no script, no external request. It is written into the workspace and delivered as a file card (for_user, so it never enters a request), the way deliver_file does. The default name comes from the source file's stem or, for pasted markdown, from the title with what a path cannot carry turned into '-'. It has a preview, so an approval prompt says "render x -> y.html" before anything is written. - download (src/tools/web.c): an http(s) URL into a file in the workspace, streamed with a 128 MB ceiling; a redirect chain, the same CA handling and the same refusals as web_fetch, but the bytes stay on disk (a half file is removed on any failure, including HTTP >= 400). The name comes from the URL when the caller names only a directory. - read_pdf (src/tools/fs_tools.c): the text layer and page count of a PDF, which read_file refuses because the bytes are binary -- and a PDF is exactly what arrives from a phone. Same extractor an attachment uses, its own 4 MiB input ceiling, 256 KiB of text by default with a note when cut. - session_search (new, src/tools/session_tools.c): every stored session -- this one and all the ones before it -- searched for a substring, reporting the session's id, message count and title, then the matching lines with their number and role. Case-insensitive by default, capped at 20 matches. - device_info (same file, perm_name "device"): version, system/arch/CPU count, workspace, sandbox, model and permissions, config/temp/program directories, pid -- so the model stops guessing where it runs. - Tools 3 and 4 come with tests/test_tools_ext.c (also in both build files): the document is a whole page, a script tag and a javascript: link do not survive, a hostile title stays text and cannot write outside the workspace, the preview names the file, read_pdf/download refuse what they must (no network in a test), session_search finds a fixture session and says so when nothing matches, device_info names the workspace. README's tool table grew five rows. - Android storage: the picker (SAF) answers with a content:// URI, and the engine opens files by path, so the app now copies what it points at into its own directory first -- app/android/src/xyz/whcself/cstagent/ AttachFile.java, called through a new app::materializeAttachment() (ToolProvider.h: one API for every platform; a desktop strips file:// and refuses content:// with a reason). The copy needs no permission at all: the grant travels with the URI. - "All files access" (MANAGE_EXTERNAL_STORAGE) is the other half: it is what lets the engine read a path in shared storage that a task names rather than picks. Android has no runtime dialog for it, so main.cpp opens the one Settings screen once (AttachFile.java's requestAllFilesAccess, with the all-apps screen as a fallback) and never insists: without it everything works, one picked file at a time. On the NX789J the screen came up, the app-op reads back "allow", and a relaunch no longer asks. - Composer.qml hands a content:// URI through untouched (it is not a path, and decoding it would corrupt it); AgentCore::attachFile() turns whatever the picker gave into a path first. Desktop: build clean, make check 36/36 (the new test included, also under ASan), --uitest all green, --selftest and cst-app-selftest OK. The APK builds and installs (86 MB, release-signed) with the new Java class in its dex. Version 0.9.6.
This commit is contained in:
WHC
@@ -15,6 +15,7 @@
|
||||
#include <regex.h>
|
||||
|
||||
#include "cst_attach.h"
|
||||
#include "cst_pdf.h"
|
||||
#include "cst_tools.h"
|
||||
#include "cst_util.h"
|
||||
#include "os_compat.h"
|
||||
@@ -1490,4 +1491,58 @@ int cst_tool_grep(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
return tool_take_buf(r, &out, 0);
|
||||
}
|
||||
|
||||
/* --- read_pdf ------------------------------------------------------------ */
|
||||
|
||||
/* A PDF is not text, so read_file refuses it (it sniffs NUL bytes) -- and a
|
||||
* PDF is exactly the file a phone gets handed. This reads its text layer the
|
||||
* way an attachment does, with the extractor's own 4 MiB ceiling. */
|
||||
#define PDF_MAX_IN (4 * 1024 * 1024 + 1024)
|
||||
#define PDF_DEFAULT_OUT (256 * 1024)
|
||||
|
||||
int cst_tool_read_pdf(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r) {
|
||||
const char *path = tool_arg(c, "path");
|
||||
if (!path || !path[0])
|
||||
return tool_set_result(r, "error: missing path", 1);
|
||||
char full[OS_PATH_MAX];
|
||||
if (resolve(ctx, path, full, sizeof(full)) != 0)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: path outside workspace or missing parent: "
|
||||
"%.200s",
|
||||
path);
|
||||
if (is_dir(full))
|
||||
return tool_set_resultf(r, 1, "error: %.200s is a directory", path);
|
||||
size_t n = 0;
|
||||
char *data = tool_slurp(full, &n, PDF_MAX_IN);
|
||||
if (!data)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: cannot read %.200s (or it is larger than "
|
||||
"%d MB)",
|
||||
path, PDF_MAX_IN >> 20);
|
||||
int pages = 0;
|
||||
char err[256] = "";
|
||||
char *text = cst_pdf_text(data, n, &pages, err, sizeof(err));
|
||||
free(data);
|
||||
if (!text)
|
||||
return tool_set_resultf(r, 1, "error: %.200s is not a readable PDF: %.150s",
|
||||
path, err);
|
||||
long max_out = tool_arg_long(c, "max_bytes", PDF_DEFAULT_OUT);
|
||||
if (max_out <= 0)
|
||||
max_out = PDF_DEFAULT_OUT;
|
||||
size_t tlen = strlen(text);
|
||||
size_t keep = cst_utf8_floor(text, (size_t)max_out);
|
||||
if (keep > tlen)
|
||||
keep = tlen;
|
||||
cst_buf_t out = {0};
|
||||
cst_buf_printf(&out, "%s · %d page%s · %zu bytes of text\n\n", path, pages,
|
||||
pages == 1 ? "" : "s", tlen);
|
||||
cst_buf_append(&out, text, keep);
|
||||
if (keep < tlen)
|
||||
cst_buf_printf(&out, "\n...(text cut at %zu of %zu bytes; a PDF's own "
|
||||
"layout is not reproduced)\n",
|
||||
keep, tlen);
|
||||
free(text);
|
||||
return tool_take_buf(r, &out, 0);
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -166,6 +166,53 @@ static const cst_tool_def_t TOOLS[] = {
|
||||
{{"url", S, 1, "http:// or https:// URL"},
|
||||
{"max_bytes", I, 0, "maximum bytes to return (default 65536)"}},
|
||||
CST_TP_NET, 0, cst_tool_web_fetch, NULL, NO_DYN, NO_META},
|
||||
{"download",
|
||||
"Fetch an http(s) URL into the workspace as a file (up to 128 MB) — a "
|
||||
"release, a dataset, an image, a PDF. The bytes stay on disk: this "
|
||||
"answer only says what was saved. Use web_fetch when you want to read "
|
||||
"the content yourself.",
|
||||
"Download an http(s) URL to a file in the workspace.",
|
||||
{{"url", S, 1, "http:// or https:// URL"},
|
||||
{"path", S, 0, "file path (default: the URL's own file name)"},
|
||||
{"max_bytes", I, 0, "give up above this size (default 128 MB)"}},
|
||||
CST_TP_NET, 0, cst_tool_download, NULL, NO_DYN, NO_META},
|
||||
{"read_pdf",
|
||||
"Read a PDF's text layer, page count and all — what read_file refuses "
|
||||
"because the bytes are binary. Layout, tables and images are not "
|
||||
"reproduced.",
|
||||
"Read the text of a PDF file in the workspace.",
|
||||
{{"path", S, 1, "PDF file path relative to the workspace"},
|
||||
{"max_bytes", I, 0, "cap on the returned text (default 256 KiB)"}},
|
||||
CST_TP_READ, 0, cst_tool_read_pdf, NULL, NO_DYN, NO_META},
|
||||
{"md_to_html",
|
||||
"Write a document: markdown in, one self-contained HTML file out "
|
||||
"(the engine's own renderer — no script, safe to open anywhere), saved "
|
||||
"in the workspace and handed to the person as a card they can open or "
|
||||
"share. Use it when the answer should be a document to keep.",
|
||||
"Render markdown into a standalone HTML document and hand it over.",
|
||||
{{"markdown", S, 0, "markdown text to render"},
|
||||
{"file", S, 0, "workspace .md file to render instead"},
|
||||
{"out", S, 0, "output path (default: the source's name + .html)"},
|
||||
{"title", S, 0, "document title (default: the file name)"}},
|
||||
CST_TP_WRITE, 0, cst_tool_md_to_html, cst_preview_md_to_html, NO_DYN,
|
||||
NO_META},
|
||||
{"session_search",
|
||||
"Search the workspace's stored sessions (this one and every earlier "
|
||||
"conversation) for a substring and get the matching lines with their "
|
||||
"session, message number and context. Answers \"what did we decide "
|
||||
"before\" without a shell.",
|
||||
"Search earlier and current sessions for text.",
|
||||
{{"query", S, 1, "text to look for"},
|
||||
{"max_matches", I, 0, "stop after this many matches (default 20)"},
|
||||
{"ignore_case", B, 0, "case-insensitive (default true)"}},
|
||||
CST_TP_READ, 0, cst_tool_session_search, NULL, NO_DYN, NO_META},
|
||||
{"device_info",
|
||||
"Where this agent is running: system and CPU count, workspace, sandbox, "
|
||||
"model and permissions, config/temp/program directories, pid. Call it "
|
||||
"instead of guessing the environment.",
|
||||
"Report the machine and session the agent runs on.",
|
||||
{{"section", S, 0, "unused; the whole report is always returned"}},
|
||||
CST_TP_READ, 0, cst_tool_device_info, NULL, NO_DYN, "device", NULL, NULL},
|
||||
{"ask_user",
|
||||
"Ask the user and wait for the answer. Only for decisions that are "
|
||||
"truly theirs (preferences, credentials, destructive choices).",
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
#ifndef _DEFAULT_SOURCE
|
||||
#define _DEFAULT_SOURCE
|
||||
#endif
|
||||
#ifndef _XOPEN_SOURCE
|
||||
#define _XOPEN_SOURCE 700
|
||||
#endif
|
||||
|
||||
#include <ctype.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#include "cJSON.h"
|
||||
#include "cst_agent.h"
|
||||
#include "cst_sandbox.h"
|
||||
#include "cst_tools.h"
|
||||
#include "cst_util.h"
|
||||
#include "cst_version.h"
|
||||
#include "os_compat.h"
|
||||
#include "tools_internal.h"
|
||||
|
||||
#ifndef _WIN32
|
||||
#include <sys/utsname.h>
|
||||
#include <unistd.h>
|
||||
#else
|
||||
#include <windows.h>
|
||||
#endif
|
||||
|
||||
/* A search that does not care about case: the model asks "where did we talk
|
||||
* about the store", and nobody remembers how it was capitalised. */
|
||||
static const char *find_in(const char *hay, const char *needle, int icase) {
|
||||
size_t nn = strlen(needle);
|
||||
if (!nn)
|
||||
return NULL;
|
||||
for (const char *p = hay; *p; p++) {
|
||||
if (!icase) {
|
||||
if (strncmp(p, needle, nn) == 0)
|
||||
return p;
|
||||
continue;
|
||||
}
|
||||
size_t i = 0;
|
||||
while (i < nn && p[i] &&
|
||||
tolower((unsigned char)p[i]) == tolower((unsigned char)needle[i]))
|
||||
i++;
|
||||
if (i == nn)
|
||||
return p;
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
/* A line of context around a match: control characters become spaces (one
|
||||
* transcript line is one JSON object anyway) and the tail is marked. */
|
||||
static void say_context(cst_buf_t *b, const char *from, size_t max) {
|
||||
size_t i = 0;
|
||||
while (from[i] && i < max) {
|
||||
char c = from[i];
|
||||
if (c == '\n' || c == '\r' || c == '\t')
|
||||
c = ' ';
|
||||
cst_buf_append(b, &c, 1);
|
||||
i++;
|
||||
}
|
||||
if (from[i])
|
||||
cst_buf_puts(b, " ...");
|
||||
}
|
||||
|
||||
/* --- session_search ------------------------------------------------------ */
|
||||
|
||||
/* What was said before: every stored session's transcript, line by line. The
|
||||
* store is the same one the CLI, the TUI, the WebUI and the app use, so this
|
||||
* is how "we solved this last week" is answered without a shell. */
|
||||
int cst_tool_session_search(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r) {
|
||||
const char *query = tool_arg(c, "query");
|
||||
if (!query || !query[0])
|
||||
return tool_set_result(r, "error: missing query", 1);
|
||||
while (*query == ' ')
|
||||
query++;
|
||||
if (!query[0])
|
||||
return tool_set_result(r, "error: missing query", 1);
|
||||
long want = tool_arg_long(c, "max_matches", 20);
|
||||
if (want <= 0)
|
||||
want = 20;
|
||||
if (want > 100)
|
||||
want = 100;
|
||||
int icase = tool_arg_bool(c, "ignore_case", 1);
|
||||
|
||||
static char ids[200][128];
|
||||
int n = cst_agent_store_list(ctx->workspace_root, ids, 200);
|
||||
cst_buf_t hits = {0};
|
||||
int matches = 0, in_sessions = 0, scanned_sessions = 0;
|
||||
for (int i = 0; i < n && matches < (int)want; i++) {
|
||||
char path[OS_PATH_MAX];
|
||||
if (cst_agent_store_path_for_id(ctx->workspace_root, ids[i], path,
|
||||
sizeof(path)) != 0)
|
||||
continue;
|
||||
FILE *fp = fopen(path, "rb");
|
||||
if (!fp)
|
||||
continue;
|
||||
scanned_sessions++;
|
||||
cst_session_info_t in;
|
||||
int have_info = cst_agent_store_info(ctx->workspace_root, ids[i], &in) == 0;
|
||||
int in_this = 0, lineno = 0;
|
||||
char *line = NULL;
|
||||
size_t cap = 0;
|
||||
cst_buf_t one = {0};
|
||||
ssize_t got;
|
||||
while ((got = getline(&line, &cap, fp)) > 0) {
|
||||
lineno++;
|
||||
if (got <= 1)
|
||||
continue;
|
||||
cJSON *o = cJSON_Parse(line);
|
||||
if (!o)
|
||||
continue;
|
||||
const cJSON *role = cJSON_GetObjectItemCaseSensitive(o, "role");
|
||||
const cJSON *content = cJSON_GetObjectItemCaseSensitive(o, "content");
|
||||
const char *txt = cJSON_IsString(content) ? content->valuestring : NULL;
|
||||
const char *at = txt ? find_in(txt, query, icase) : NULL;
|
||||
int stop = 0;
|
||||
if (at) {
|
||||
if (!in_this) {
|
||||
in_this = 1;
|
||||
in_sessions++;
|
||||
cst_buf_printf(&one, "\n%s", ids[i]);
|
||||
if (have_info && in.messages)
|
||||
cst_buf_printf(&one, " · %d msgs", in.messages);
|
||||
if (have_info && in.title[0])
|
||||
cst_buf_printf(&one, " · %s", in.title);
|
||||
else if (have_info && in.first_user[0])
|
||||
cst_buf_printf(&one, " · %s", in.first_user);
|
||||
cst_buf_puts(&one, "\n");
|
||||
}
|
||||
const char *who = cJSON_IsString(role) ? role->valuestring : "?";
|
||||
const char *meta = NULL;
|
||||
const cJSON *kind =
|
||||
cJSON_GetObjectItemCaseSensitive(
|
||||
cJSON_GetObjectItemCaseSensitive(o, "meta"), "kind");
|
||||
if (cJSON_IsString(kind))
|
||||
meta = kind->valuestring;
|
||||
/* a little before the match, a line's worth after it */
|
||||
const char *start = at;
|
||||
for (int back = 0; start > txt && back < 60; back++)
|
||||
start--;
|
||||
cst_buf_printf(&one, " #%d %s%s: ", lineno, who,
|
||||
meta && strcmp(meta, "note") == 0 ? " (note)" : "");
|
||||
say_context(&one, start, (size_t)(at - start) + strlen(query) + 160);
|
||||
cst_buf_puts(&one, "\n");
|
||||
matches++;
|
||||
stop = matches >= (int)want;
|
||||
}
|
||||
cJSON_Delete(o);
|
||||
if (stop)
|
||||
break;
|
||||
}
|
||||
cst_buf_append(&hits, one.buf, one.buf ? one.len : 0);
|
||||
cst_buf_free(&one);
|
||||
free(line);
|
||||
fclose(fp);
|
||||
}
|
||||
if (matches == 0) {
|
||||
cst_buf_free(&hits);
|
||||
return tool_set_resultf(r, 0,
|
||||
"no match for \"%.80s\" in %d stored session%s "
|
||||
"(the store holds %d)\n",
|
||||
query, scanned_sessions,
|
||||
scanned_sessions == 1 ? "" : "s", n);
|
||||
}
|
||||
cst_buf_t out = {0};
|
||||
cst_buf_printf(&out, "%d match%s for \"%.80s\" in %d of %d stored session%s\n",
|
||||
matches, matches == 1 ? "" : "es", query, in_sessions,
|
||||
scanned_sessions, scanned_sessions == 1 ? "" : "s");
|
||||
cst_buf_append(&out, hits.buf, hits.len);
|
||||
if (matches >= (int)want)
|
||||
cst_buf_puts(&out, "\n(more may match; raise max_matches or narrow the "
|
||||
"query)\n");
|
||||
cst_buf_free(&hits);
|
||||
return tool_take_buf(r, &out, 0);
|
||||
}
|
||||
|
||||
/* --- device_info --------------------------------------------------------- */
|
||||
|
||||
/* Where this agent is running: enough for the model to stop guessing (a
|
||||
* phone in a pocket and a desktop in a terminal answer differently), and no
|
||||
* more than that -- no serial numbers, no accounts. */
|
||||
int cst_tool_device_info(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r) {
|
||||
(void)c;
|
||||
cst_buf_t b = {0};
|
||||
cst_buf_printf(&b, "cst-agent %s\n", CST_VERSION);
|
||||
#if defined(__ANDROID__)
|
||||
const char *plat = "android";
|
||||
#elif defined(_WIN32)
|
||||
const char *plat = "windows";
|
||||
#elif defined(__APPLE__)
|
||||
const char *plat = "macos";
|
||||
#elif defined(__linux__)
|
||||
const char *plat = "linux";
|
||||
#else
|
||||
const char *plat = "unknown";
|
||||
#endif
|
||||
#ifdef _WIN32
|
||||
char host[160] = "";
|
||||
os_windows_version(host, sizeof(host));
|
||||
SYSTEM_INFO si;
|
||||
memset(&si, 0, sizeof(si));
|
||||
GetSystemInfo(&si);
|
||||
cst_buf_printf(&b, "system: %s (%d-bit, %lu CPUs)\n",
|
||||
host[0] ? host : plat, (int)(sizeof(void *) * 8),
|
||||
(unsigned long)si.dwNumberOfProcessors);
|
||||
#else
|
||||
struct utsname u;
|
||||
long cpus = sysconf(_SC_NPROCESSORS_ONLN);
|
||||
if (uname(&u) == 0)
|
||||
cst_buf_printf(&b, "system: %s %.80s %.60s (%d-bit, %ld CPUs)\n",
|
||||
u.sysname, u.release, u.machine, (int)(sizeof(void *) * 8),
|
||||
cpus > 0 ? cpus : 0L);
|
||||
else
|
||||
cst_buf_printf(&b, "system: %s (%d-bit)\n", plat,
|
||||
(int)(sizeof(void *) * 8));
|
||||
#endif
|
||||
cst_buf_printf(&b, "workspace: %s\n",
|
||||
ctx->workspace_root ? ctx->workspace_root : "?");
|
||||
cst_buf_printf(&b, "sandbox: %s\n", cst_sandbox_name(cst_sandbox_probe()));
|
||||
const cst_agent_config_t *cfg = ctx->agent ? cst_agent_config(ctx->agent) : NULL;
|
||||
if (cfg) {
|
||||
char sid[128] = "";
|
||||
cst_agent_session_id(ctx->agent, sid, sizeof(sid));
|
||||
cst_buf_printf(&b, "session: %.120s\n", sid[0] ? sid : "(none)");
|
||||
cst_buf_printf(&b, "model: %.60s · %.30s · effort %.12s · mode %.12s\n",
|
||||
cfg->model, cfg->provider, cfg->effort, cfg->mode);
|
||||
cst_buf_printf(&b, "permissions: approval %s%s%s\n",
|
||||
cst_approval_name(cfg->approval),
|
||||
cfg->plan_mode ? ", plan mode (read-only)" : "",
|
||||
cfg->auto_approve ? ", unattended" : "");
|
||||
cst_buf_printf(&b, "steps: %d model calls per turn max\n", cfg->max_steps);
|
||||
}
|
||||
char dir[OS_PATH_MAX];
|
||||
if (os_config_dir(dir, sizeof(dir)) == 0)
|
||||
cst_buf_printf(&b, "config: %s\n", dir);
|
||||
if (os_temp_dir(dir, sizeof(dir)) == 0)
|
||||
cst_buf_printf(&b, "temp: %s\n", dir);
|
||||
if (os_exe_dir(dir, sizeof(dir)) == 0)
|
||||
cst_buf_printf(&b, "program: %s\n", dir);
|
||||
cst_buf_printf(&b, "process: pid %d\n", (int)os_getpid());
|
||||
return tool_take_buf(r, &b, 0);
|
||||
}
|
||||
@@ -55,6 +55,18 @@ int cst_preview_clipboard_write(const cst_tool_ctx_t *ctx,
|
||||
const ctl_tool_call_t *c, char **out);
|
||||
int cst_tool_web_fetch(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_tool_download(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_tool_md_to_html(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_preview_md_to_html(const cst_tool_ctx_t *ctx,
|
||||
const ctl_tool_call_t *c, char **out);
|
||||
int cst_tool_read_pdf(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_tool_session_search(const cst_tool_ctx_t *ctx,
|
||||
const ctl_tool_call_t *c, cst_tool_result_t *r);
|
||||
int cst_tool_device_info(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_tool_view_image(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r);
|
||||
int cst_tool_deliver_file(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
|
||||
+329
@@ -6,8 +6,12 @@
|
||||
|
||||
#include <curl/curl.h>
|
||||
|
||||
#include "cst_agent.h"
|
||||
#include "cst_attach.h"
|
||||
#include "cst_tools.h"
|
||||
#include "cst_util.h"
|
||||
#include "cst_web_md.h"
|
||||
#include "os_compat.h"
|
||||
#include "tools_internal.h"
|
||||
|
||||
#define FETCH_DOWNLOAD_MAX (5 * 1024 * 1024)
|
||||
@@ -284,3 +288,328 @@ int cst_tool_web_fetch(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_buf_free(&f.body);
|
||||
return tool_take_buf(r, &b, code);
|
||||
}
|
||||
|
||||
/* --- md_to_html ---------------------------------------------------------- */
|
||||
|
||||
#define MD_TO_HTML_MAX (8 * 1024 * 1024)
|
||||
|
||||
/* The file name a URL ends in; "" when it names nothing (a bare host, a
|
||||
* directory). Kept to plain characters: it becomes a file name. */
|
||||
static void name_from_url(const char *url, char *out, size_t n) {
|
||||
const char *p = strstr(url, "://");
|
||||
p = p ? p + 3 : url;
|
||||
const char *q = strpbrk(p, "?#");
|
||||
const char *end = q ? q : p + strlen(p);
|
||||
const char *slash = NULL;
|
||||
for (const char *s = p; s < end; s++)
|
||||
if (*s == '/')
|
||||
slash = s;
|
||||
const char *leaf = slash ? slash + 1 : p;
|
||||
size_t len = (size_t)(end - leaf);
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
unsigned char ch = (unsigned char)leaf[i];
|
||||
if (!(isalnum(ch) || ch == '.' || ch == '-' || ch == '_' || ch == '+')) {
|
||||
len = i; /* cut at the first byte that is not a file name's */
|
||||
break;
|
||||
}
|
||||
}
|
||||
while (len && leaf[len - 1] == '.')
|
||||
len--;
|
||||
if (!len)
|
||||
snprintf(out, n, "download.bin");
|
||||
else
|
||||
snprintf(out, n, "%.*s", (int)len, leaf);
|
||||
}
|
||||
|
||||
/* "docs/notes.md" -> "notes" (the title a document falls back to). */
|
||||
static void stem_of(const char *rel, char *out, size_t n) {
|
||||
char leaf[128];
|
||||
snprintf(leaf, sizeof(leaf), "%.127s", rel && rel[0] ? rel : "");
|
||||
char *slash = NULL;
|
||||
for (char *p = leaf; *p; p++)
|
||||
if (*p == '/' || *p == '\\')
|
||||
slash = p;
|
||||
char *base = slash ? slash + 1 : leaf;
|
||||
char *dot = strrchr(base, '.');
|
||||
if (dot && dot != base)
|
||||
*dot = 0;
|
||||
snprintf(out, n, "%s", base[0] ? base : "document");
|
||||
}
|
||||
|
||||
/* A title turned into a file name: what a path cannot carry becomes '-',
|
||||
* control bytes go, and a name that is only dots or spaces is refused. */
|
||||
static void stem_of_title(const char *title, char *out, size_t n) {
|
||||
size_t i = 0;
|
||||
for (const char *p = title; *p && i + 1 < n && i < 80; p++) {
|
||||
unsigned char ch = (unsigned char)*p;
|
||||
if (ch == '/' || ch == '\\' || ch == ':' || ch == '*' || ch == '?' ||
|
||||
ch == '"' || ch == '<' || ch == '>' || ch == '|')
|
||||
ch = '-';
|
||||
else if (ch < 0x20)
|
||||
continue;
|
||||
out[i++] = (char)ch;
|
||||
}
|
||||
while (i && (out[i - 1] == ' ' || out[i - 1] == '.'))
|
||||
i--;
|
||||
size_t start = 0;
|
||||
while (start < i && (out[start] == ' ' || out[start] == '.'))
|
||||
start++;
|
||||
i -= start;
|
||||
if (i)
|
||||
memmove(out, out + start, i);
|
||||
out[i] = 0;
|
||||
if (!i)
|
||||
snprintf(out, n, "document");
|
||||
}
|
||||
|
||||
/* The file md_to_html writes: `out` when given (a directory takes the
|
||||
* source's name inside it), otherwise the source file's stem, or the title. */
|
||||
static void md_out_name(const ctl_tool_call_t *c, char *out_name, size_t n) {
|
||||
const char *file = tool_arg(c, "file");
|
||||
const char *out_arg = tool_arg(c, "out");
|
||||
const char *title = tool_arg(c, "title");
|
||||
char stem[200];
|
||||
if (file && file[0])
|
||||
stem_of(file, stem, sizeof(stem));
|
||||
else if (title && title[0])
|
||||
stem_of_title(title, stem, sizeof(stem));
|
||||
else
|
||||
snprintf(stem, sizeof(stem), "document");
|
||||
if (out_arg && out_arg[0]) {
|
||||
snprintf(out_name, n, "%.300s", out_arg);
|
||||
size_t ol = strlen(out_name);
|
||||
if (ol && (out_name[ol - 1] == '/' || out_name[ol - 1] == '\\') &&
|
||||
ol + strlen(stem) + 6 < n)
|
||||
snprintf(out_name + ol, n - ol, "%s.html", stem);
|
||||
} else {
|
||||
snprintf(out_name, n, "%s.html", stem);
|
||||
}
|
||||
}
|
||||
|
||||
/* What the window shows before the write is approved. */
|
||||
int cst_preview_md_to_html(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
char **out) {
|
||||
(void)ctx;
|
||||
const char *md = tool_arg(c, "markdown");
|
||||
const char *file = tool_arg(c, "file");
|
||||
if ((!md || !md[0]) && (!file || !file[0])) {
|
||||
*out = strdup("error: give markdown, or a .md file to render");
|
||||
return -1;
|
||||
}
|
||||
char out_name[400];
|
||||
md_out_name(c, out_name, sizeof(out_name));
|
||||
cst_buf_t b = {0};
|
||||
if (file && file[0])
|
||||
cst_buf_printf(&b, "render %.200s → %.200s\n", file, out_name);
|
||||
else
|
||||
cst_buf_printf(&b, "render %zu bytes of markdown → %.200s\n",
|
||||
md ? strlen(md) : 0, out_name);
|
||||
*out = cst_buf_steal(&b);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int cst_tool_md_to_html(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r) {
|
||||
const char *md = tool_arg(c, "markdown");
|
||||
const char *file = tool_arg(c, "file");
|
||||
if ((!md || !md[0]) && (!file || !file[0]))
|
||||
return tool_set_result(r, "error: give markdown, or a .md file to render",
|
||||
1);
|
||||
const char *text = md;
|
||||
size_t len = md ? strlen(md) : 0;
|
||||
char *owned = NULL;
|
||||
if (file && file[0]) {
|
||||
char src[OS_PATH_MAX];
|
||||
if (cst_sandbox_resolve(ctx->workspace_root, file, src, sizeof(src)) != 0)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: path outside workspace or missing "
|
||||
"parent: %.200s",
|
||||
file);
|
||||
size_t n = 0;
|
||||
owned = tool_slurp(src, &n, MD_TO_HTML_MAX);
|
||||
if (!owned)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: cannot read %.200s (or it is larger than "
|
||||
"%d MB)",
|
||||
file, MD_TO_HTML_MAX >> 20);
|
||||
text = owned;
|
||||
len = n;
|
||||
}
|
||||
|
||||
char title[256], out_name[400];
|
||||
const char *given = tool_arg(c, "title");
|
||||
snprintf(title, sizeof(title), "%.200s", given && given[0] ? given : "Document");
|
||||
md_out_name(c, out_name, sizeof(out_name));
|
||||
|
||||
char *html = cst_web_md_document(text, len, title);
|
||||
free(owned);
|
||||
if (!html)
|
||||
return tool_set_result(r, "error: cannot render the markdown", 1);
|
||||
tool_ensure_parent_dirs(ctx->workspace_root, out_name);
|
||||
char full[OS_PATH_MAX];
|
||||
if (cst_sandbox_resolve(ctx->workspace_root, out_name, full, sizeof(full)) !=
|
||||
0) {
|
||||
free(html);
|
||||
return tool_set_resultf(r, 1, "error: path outside workspace: %.200s",
|
||||
out_name);
|
||||
}
|
||||
size_t bytes = strlen(html);
|
||||
int wrote = tool_write_whole(full, html, bytes);
|
||||
free(html);
|
||||
if (wrote != 0)
|
||||
return tool_set_resultf(r, 1, "error: cannot write %.200s", out_name);
|
||||
|
||||
const char *session = ctx->agent ? cst_agent_session_path(ctx->agent) : NULL;
|
||||
cst_attachment_t *a = NULL;
|
||||
char err[256] = "";
|
||||
if (session && session[0])
|
||||
a = cst_attach_import_artifact(session, full, NULL, err, sizeof(err));
|
||||
|
||||
cst_buf_t out = {0};
|
||||
cst_buf_printf(&out,
|
||||
"wrote %.200s (%zu bytes): one HTML file with everything "
|
||||
"inside it, ready to open in a browser\n",
|
||||
out_name, bytes);
|
||||
if (a)
|
||||
cst_buf_puts(&out,
|
||||
"(the user sees it as a file card they can open or share; "
|
||||
"you will not be shown its bytes again)\n");
|
||||
else if (err[0])
|
||||
cst_buf_printf(&out, "(the file is in the workspace; handing it over "
|
||||
"failed: %.150s)\n",
|
||||
err);
|
||||
int rc = tool_take_buf(r, &out, 0);
|
||||
if (a) {
|
||||
r->attach = a;
|
||||
r->nattach = 1;
|
||||
}
|
||||
return rc;
|
||||
}
|
||||
|
||||
/* --- download ------------------------------------------------------------ */
|
||||
|
||||
#define DOWNLOAD_MAX (128 * 1024 * 1024)
|
||||
#define DOWNLOAD_TIMEOUT 300L
|
||||
|
||||
typedef struct {
|
||||
FILE *fp;
|
||||
size_t written;
|
||||
size_t cap;
|
||||
int overflow;
|
||||
int failed;
|
||||
} dl_t;
|
||||
|
||||
static size_t on_file(char *p, size_t sz, size_t n, void *ud) {
|
||||
dl_t *d = ud;
|
||||
size_t len = sz * n;
|
||||
if (d->written + len > d->cap) {
|
||||
d->overflow = 1;
|
||||
return 0; /* abort: the cap is what keeps a wrong URL from filling a disk */
|
||||
}
|
||||
if (fwrite(p, 1, len, d->fp) != len) {
|
||||
d->failed = 1;
|
||||
return 0;
|
||||
}
|
||||
d->written += len;
|
||||
return len;
|
||||
}
|
||||
|
||||
int cst_tool_download(const cst_tool_ctx_t *ctx, const ctl_tool_call_t *c,
|
||||
cst_tool_result_t *r) {
|
||||
const char *url = tool_arg(c, "url");
|
||||
if (!url || !url[0])
|
||||
return tool_set_result(r, "error: missing url", 1);
|
||||
while (*url == ' ')
|
||||
url++;
|
||||
if (strncasecmp(url, "http://", 7) != 0 &&
|
||||
strncasecmp(url, "https://", 8) != 0)
|
||||
return tool_set_result(r, "error: only http:// and https:// URLs", 1);
|
||||
long cap = tool_arg_long(c, "max_bytes", DOWNLOAD_MAX);
|
||||
if (cap <= 0 || cap > DOWNLOAD_MAX)
|
||||
cap = DOWNLOAD_MAX;
|
||||
|
||||
const char *path = tool_arg(c, "path");
|
||||
char rel[400];
|
||||
if (path && path[0])
|
||||
snprintf(rel, sizeof(rel), "%s", path);
|
||||
else
|
||||
name_from_url(url, rel, sizeof(rel));
|
||||
size_t rl = strlen(rel);
|
||||
if (rl && (rel[rl - 1] == '/' || rel[rl - 1] == '\\')) {
|
||||
/* the caller named the directory: the URL names the file */
|
||||
char name[200];
|
||||
name_from_url(url, name, sizeof(name));
|
||||
snprintf(rel + rl, sizeof(rel) - rl, "%s", name);
|
||||
}
|
||||
tool_ensure_parent_dirs(ctx->workspace_root, rel);
|
||||
char full[OS_PATH_MAX];
|
||||
if (cst_sandbox_resolve(ctx->workspace_root, rel, full, sizeof(full)) != 0)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: path outside workspace or missing parent: "
|
||||
"%.200s",
|
||||
rel);
|
||||
FILE *fp = fopen(full, "wb");
|
||||
if (!fp)
|
||||
return tool_set_resultf(r, 1, "error: cannot write %.200s", rel);
|
||||
|
||||
CURL *curl = curl_easy_init();
|
||||
if (!curl) {
|
||||
fclose(fp);
|
||||
remove(full);
|
||||
return tool_set_result(r, "error: curl init failed", 1);
|
||||
}
|
||||
dl_t d;
|
||||
memset(&d, 0, sizeof(d));
|
||||
d.fp = fp;
|
||||
d.cap = (size_t)cap;
|
||||
char errbuf[CURL_ERROR_SIZE];
|
||||
errbuf[0] = 0;
|
||||
cst_curl_apply_ca(curl);
|
||||
curl_easy_setopt(curl, CURLOPT_URL, url);
|
||||
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
|
||||
curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 5L);
|
||||
#if LIBCURL_VERSION_NUM >= 0x075500
|
||||
curl_easy_setopt(curl, CURLOPT_PROTOCOLS_STR, "http,https");
|
||||
curl_easy_setopt(curl, CURLOPT_REDIR_PROTOCOLS_STR, "http,https");
|
||||
#else
|
||||
curl_easy_setopt(curl, CURLOPT_PROTOCOLS,
|
||||
(long)(CURLPROTO_HTTP | CURLPROTO_HTTPS));
|
||||
curl_easy_setopt(curl, CURLOPT_REDIR_PROTOCOLS,
|
||||
(long)(CURLPROTO_HTTP | CURLPROTO_HTTPS));
|
||||
#endif
|
||||
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 15L);
|
||||
curl_easy_setopt(curl, CURLOPT_TIMEOUT, DOWNLOAD_TIMEOUT);
|
||||
curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1L);
|
||||
curl_easy_setopt(curl, CURLOPT_USERAGENT, "cst-agent/1.0 (+download)");
|
||||
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, on_file);
|
||||
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &d);
|
||||
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
|
||||
CURLcode rc = curl_easy_perform(curl);
|
||||
long http = 0;
|
||||
char *ctype = NULL;
|
||||
char *final_url = NULL;
|
||||
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &http);
|
||||
curl_easy_getinfo(curl, CURLINFO_CONTENT_TYPE, &ctype);
|
||||
curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &final_url);
|
||||
int closed = fclose(fp) != 0;
|
||||
curl_easy_cleanup(curl);
|
||||
if (rc != CURLE_OK || d.failed || closed || http >= 400) {
|
||||
remove(full); /* half a file is worse than none */
|
||||
if (d.overflow)
|
||||
return tool_set_resultf(r, 1,
|
||||
"error: %.200s is larger than %ld bytes and "
|
||||
"nothing was saved (raise max_bytes, up to %d "
|
||||
"MB)",
|
||||
url, cap, DOWNLOAD_MAX >> 20);
|
||||
return tool_set_resultf(r, 1, "error: download failed: HTTP %ld %.200s",
|
||||
http, errbuf[0] ? errbuf : curl_easy_strerror(rc));
|
||||
}
|
||||
cst_buf_t b = {0};
|
||||
cst_buf_printf(&b, "saved %.200s (%s, %zu bytes) from HTTP %ld %.200s\n", rel,
|
||||
ctype ? ctype : "application/octet-stream", d.written, http,
|
||||
final_url ? final_url : url);
|
||||
cst_buf_puts(&b,
|
||||
"(the bytes are on disk, not in this answer; read_file or "
|
||||
"read_pdf it when you need to look inside)\n");
|
||||
return tool_take_buf(r, &b, 0);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user