v0.9.0: the Qt front end as an Android APK, images in the transcript

Two strands land together: the window draws pictures now, and the whole front
end builds, installs and runs on the phone with the engine linked in-process.

Images (the "make a picture actually show up" part)
- Engine, additively: cst_agent_hooks_t.on_tool_images(call, list, n, ud) fires
  when a call left attachments behind, and cst_agent_pending_rel() /
  cst_agent_pending_is_image() read that list without the app touching the
  store. tests/test_attach.c::test_agent_pending pins the three cases (an
  image, a stored name that passes cst_attach_rel_ok, out-of-range NULL).
- App: image://cstattach/<stored name> (app/src/ui/SessionImages.cpp) serves
  only the current session's attachment directory, checked with
  cst_attach_rel_ok, decodes down to the requested sourceSize and never
  upscales, and holds a mutex because requestImage() runs on the render
  thread while the directory can change underneath it. AgentCore gained
  attachmentDir(), pendingImages(), the toolImages()/promptImages() signals
  and hookToolImages(); history() gives replayed rows their images too.
- QML: ImageStrip.qml (a thumb strip, <=150 px tall, shaped by the picture,
  tooltip with name/size/origin, the file's name when it went missing),
  wired into MessageCard and ToolCard. Qt 6's ListModel cannot hold a
  list-valued role (append() reads back undefined, setProperty drops it), so
  a row carries one image per line as <stored>|<name>|<origin>|<w>x<h> and
  ImageStrip decodes that.

Android (the engine is already in the APK; this is the front end)
- app/src/main.cpp: on Android the workspace, config, tmp and HOME go under
  the app's own data directory before the engine opens anything, because
  QDir::currentPath() is "/" there and /tmp is not writable.
- app/android/: the manifest (QtApplication/QtActivity, INTERNET and
  POST_NOTIFICATIONS, adjustResize, no storage permission -- the picker will
  be the system one), strings, a vector icon; minSdk 24 and targetSdk 35 come
  from app/CMakeLists.txt through QT_ANDROID_MIN/TARGET_SDK_VERSION.
- app/src/platform/android/AndroidTools.cpp: the Android file behind
  PlatformInit(), owner "app:android" (notification, clipboard, share sheet,
  file picker are stubs for now, so the tool list does not lie about them).
- app/tools/selftest.c decides its scratch workspace with os_temp_dir(), so
  the C smoke test runs on the device too.
- scripts/build_apk_android.sh: sources to installed APK in one command
  (environment checks, qt-cmake configure, --target apk, adb install), and
  the host-side plumbing the NixOS + qt.io combination needs -- the NDK
  sysroot include dirs for Qt's FindGLESv2/FindEGL (they only look under
  <sysroot>/include), -L$PREFIX/lib on the shared/module link lines as well
  as the executables', the Android debug keystore created on first use with
  QT_ANDROID_SIGN_APK + QT_ANDROID_KEYSTORE_* (androiddeployqt copies the
  unsigned gradle output and signs it itself), and the #!/bin/bash shebang of
  the build-tools wrappers, which NixOS does not have.
- scripts/gradle-android-init.gradle (installed into ~/.gradle/init.d/): the
  release lint task is switched off -- lintVitalAnalyzeRelease resolves its
  own tool jars straight from dl.google.com, bypassing every repository the
  build configures, and fails the build after all the real work is done.
- Icons: app/icons/*.svg (Breeze's 24@3x files, the 19 names the window asks
  for) are compiled in as :/cst-icons and ThemeIcons.cpp falls back to them
  when the platform has no icon theme. On the phone the Qt theme lookup found
  nothing and the QML glyph fallback was an empty box in the phone's fonts.
- docs/android.md: "The APK (Qt front end)" -- the SDK assembled without
  sdkmanager, the host Qt's missing libraries on NixOS, the Gradle mirrors,
  lint, signing -- and a Status that says what runs where. app/README.md
  gains the images and APK lines, and stops claiming Android platform tools
  that are still stubs.

Verified: engine `make check` green; desktop `cst-app-selftest: OK` and
--uitest all checks pass; phone (NX789J, Android 16, arm64-v8a) -- the APK
installs, starts on the workspace under /data/user/0/xyz.whcself.cstagent/
files/, a turn reaches a configured provider over HTTPS from inside the ART
process and renders the reply, rotation keeps the session and docks the
sidebar, and the toolbar/card icons draw.

Version 0.9.0.

mode = coding
This commit is contained in:
WHC
2026-10-11 15:22:10 +08:00
parent 82dee65942
commit c2320f946b
46 changed files with 1370 additions and 26 deletions
+96 -4
View File
@@ -17,6 +17,7 @@
#include <cstdlib>
#include "cst_agent.h"
#include "cst_attach.h"
#include "cst_models.h"
#include "cst_provider_switch.h"
#include "cst_web_md.h"
@@ -53,6 +54,7 @@ AgentCore::AgentCore(QObject *parent) : QObject(parent) {
m_hooks.on_think_delta = hookThink;
m_hooks.on_tool_event = hookToolEvent;
m_hooks.on_tool_result = hookToolResult;
m_hooks.on_tool_images = hookToolImages;
m_hooks.on_todo = hookTodo;
m_hooks.on_permission = hookPermission;
m_hooks.on_ask_user = hookAskUser;
@@ -172,7 +174,12 @@ void AgentCore::close() {
void AgentCore::send(const QString &prompt) {
if (!m_agent || m_running || prompt.trimmed().isEmpty())
return;
/* The prompt takes the pending images with it: the view shows them on the
* message the composer just appended, before the turn can say anything. */
const QString sending = pendingImages();
setRunning(true);
if (!sending.isEmpty())
emit promptImages(sending);
m_turn = new Turn(this, prompt);
m_turn->start();
}
@@ -314,9 +321,41 @@ QVariantMap sessionRow(const QString &kind, const QString &role) {
r["detail"] = QString();
r["output"] = QString();
r["exitCode"] = 0;
r["images"] = QString();
return r;
}
/* One field of one picture line: names come from the user and from tool
* output, so a separator may not split a line in two. */
QString field(const char *s) {
QString out = utf8(s);
out.replace(QLatin1Char('|'), QLatin1Char('/'));
out.replace(QLatin1Char('\n'), QLatin1Char(' '));
return out;
}
/* The images of one message or tool result, one line per picture:
*
* <stored name>|<name>|<origin>|<w>x<h>
*
* A list role is not an option: a ListModel keeps what fits in a QVariant of
* a simple type -- a list reads back as undefined there, and setProperty drops
* the role -- so the rows carry this text and ImageStrip.qml decodes it. A
* document (PDF, text) is not a picture and stays out of it. */
QString imagesText(const cst_attachment_t *list, int n) {
QStringList lines;
for (int i = 0; list && i < n; i++) {
const cst_attachment_t *a = &list[i];
if (!cst_attach_is_image(a))
continue;
lines << QStringLiteral("%1|%2|%3|%4x%5")
.arg(field(a->rel), field(a->name), field(a->origin))
.arg(a->width)
.arg(a->height);
}
return lines.join(QLatin1Char('\n'));
}
/* The open tool row this result belongs to, -1 when none. Rows are paired by
* call id first: the same tool may run twice in a turn, and pairing by name
* would hang the first card on "running" forever. Only a call without an id
@@ -354,6 +393,20 @@ QString AgentCore::sessionId() const {
return utf8(id);
}
/* Where the session keeps its images: what image://cstattach/ serves, and
* what the view asks for when the session changes. */
QString AgentCore::attachmentDir() const {
if (!m_agent)
return QString();
const char *sess = cst_agent_session_path(m_agent);
if (!sess || !sess[0])
return QString();
char dir[4096] = "";
if (cst_attach_dir(sess, dir, sizeof(dir)) != 0)
return QString();
return utf8(dir);
}
QVariantList AgentCore::sessions() const {
QVariantList out;
if (m_workspace.isEmpty())
@@ -404,12 +457,14 @@ QVariantList AgentCore::history() const {
r["name"] = name;
r["callId"] = callId;
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows.append(r);
} else {
QVariantMap r = rows.at(at).toMap();
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows[at] = r;
@@ -426,13 +481,16 @@ QVariantList AgentCore::history() const {
if (m->role == CST_ROLE_USER) {
QVariantMap r = sessionRow(QStringLiteral("message"), QStringLiteral("user"));
QString text = utf8(m->content);
if (m->nattach > 0) {
QStringList names;
for (int a = 0; a < m->nattach; a++)
/* A document keeps its name in the text; a picture is shown instead
* (the images below), so only the rest is worth naming. */
QStringList names;
for (int a = 0; a < m->nattach; a++)
if (!cst_attach_is_image(&m->attach[a]))
names << utf8(m->attach[a].name);
if (!names.isEmpty())
text += QStringLiteral("\n\n[attached: %1]").arg(names.join(", "));
}
r["text"] = text;
r["images"] = imagesText(m->attach, m->nattach);
r["complete"] = true;
rows.append(r);
continue;
@@ -802,6 +860,30 @@ QVariantMap AgentCore::usage() const {
/* --- attachments ---------------------------------------------------------- */
/* What the prompt about to be sent carries, ready for the view. The stored
* name is the engine's (the file already lives next to the session), the
* display name is the one the window kept when it copied the file in. */
QString AgentCore::pendingImages() const {
QStringList lines;
if (!m_agent)
return QString();
const int n = cst_agent_pending_count(m_agent);
for (int i = 0; i < n; i++) {
if (!cst_agent_pending_is_image(m_agent, i))
continue;
const char *rel = cst_agent_pending_rel(m_agent, i);
if (!rel || !rel[0])
continue;
const QString name =
i < m_attachNames.size() ? m_attachNames.at(i) : utf8(rel);
/* The size is not known before the engine stores the file; the strip
* falls back to a picture's usual shape until it is. */
lines << QStringLiteral("%1|%2|user|0x0")
.arg(field(rel), field(name.toUtf8().constData()));
}
return lines.join(QLatin1Char('\n'));
}
QStringList AgentCore::pendingAttachments() const {
QStringList out;
if (!m_agent)
@@ -937,6 +1019,16 @@ void AgentCore::hookToolResult(const ctl_tool_call_t *call, const char *out,
[self, id, n, exit_code, o] { emit self->toolResult(id, n, exit_code, o); });
}
void AgentCore::hookToolImages(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString images = imagesText(list, n);
if (images.isEmpty())
return; /* a call that made no picture needs no signal */
const QString id = QString::fromUtf8(call && call->id ? call->id : "");
self->post([self, id, images] { emit self->toolImages(id, images); });
}
void AgentCore::hookTodo(const char *text, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString t = QString::fromUtf8(text ? text : "");