v0.9.0: the Qt front end as an Android APK, images in the transcript

Two strands land together: the window draws pictures now, and the whole front
end builds, installs and runs on the phone with the engine linked in-process.

Images (the "make a picture actually show up" part)
- Engine, additively: cst_agent_hooks_t.on_tool_images(call, list, n, ud) fires
  when a call left attachments behind, and cst_agent_pending_rel() /
  cst_agent_pending_is_image() read that list without the app touching the
  store. tests/test_attach.c::test_agent_pending pins the three cases (an
  image, a stored name that passes cst_attach_rel_ok, out-of-range NULL).
- App: image://cstattach/<stored name> (app/src/ui/SessionImages.cpp) serves
  only the current session's attachment directory, checked with
  cst_attach_rel_ok, decodes down to the requested sourceSize and never
  upscales, and holds a mutex because requestImage() runs on the render
  thread while the directory can change underneath it. AgentCore gained
  attachmentDir(), pendingImages(), the toolImages()/promptImages() signals
  and hookToolImages(); history() gives replayed rows their images too.
- QML: ImageStrip.qml (a thumb strip, <=150 px tall, shaped by the picture,
  tooltip with name/size/origin, the file's name when it went missing),
  wired into MessageCard and ToolCard. Qt 6's ListModel cannot hold a
  list-valued role (append() reads back undefined, setProperty drops it), so
  a row carries one image per line as <stored>|<name>|<origin>|<w>x<h> and
  ImageStrip decodes that.

Android (the engine is already in the APK; this is the front end)
- app/src/main.cpp: on Android the workspace, config, tmp and HOME go under
  the app's own data directory before the engine opens anything, because
  QDir::currentPath() is "/" there and /tmp is not writable.
- app/android/: the manifest (QtApplication/QtActivity, INTERNET and
  POST_NOTIFICATIONS, adjustResize, no storage permission -- the picker will
  be the system one), strings, a vector icon; minSdk 24 and targetSdk 35 come
  from app/CMakeLists.txt through QT_ANDROID_MIN/TARGET_SDK_VERSION.
- app/src/platform/android/AndroidTools.cpp: the Android file behind
  PlatformInit(), owner "app:android" (notification, clipboard, share sheet,
  file picker are stubs for now, so the tool list does not lie about them).
- app/tools/selftest.c decides its scratch workspace with os_temp_dir(), so
  the C smoke test runs on the device too.
- scripts/build_apk_android.sh: sources to installed APK in one command
  (environment checks, qt-cmake configure, --target apk, adb install), and
  the host-side plumbing the NixOS + qt.io combination needs -- the NDK
  sysroot include dirs for Qt's FindGLESv2/FindEGL (they only look under
  <sysroot>/include), -L$PREFIX/lib on the shared/module link lines as well
  as the executables', the Android debug keystore created on first use with
  QT_ANDROID_SIGN_APK + QT_ANDROID_KEYSTORE_* (androiddeployqt copies the
  unsigned gradle output and signs it itself), and the #!/bin/bash shebang of
  the build-tools wrappers, which NixOS does not have.
- scripts/gradle-android-init.gradle (installed into ~/.gradle/init.d/): the
  release lint task is switched off -- lintVitalAnalyzeRelease resolves its
  own tool jars straight from dl.google.com, bypassing every repository the
  build configures, and fails the build after all the real work is done.
- Icons: app/icons/*.svg (Breeze's 24@3x files, the 19 names the window asks
  for) are compiled in as :/cst-icons and ThemeIcons.cpp falls back to them
  when the platform has no icon theme. On the phone the Qt theme lookup found
  nothing and the QML glyph fallback was an empty box in the phone's fonts.
- docs/android.md: "The APK (Qt front end)" -- the SDK assembled without
  sdkmanager, the host Qt's missing libraries on NixOS, the Gradle mirrors,
  lint, signing -- and a Status that says what runs where. app/README.md
  gains the images and APK lines, and stops claiming Android platform tools
  that are still stubs.

Verified: engine `make check` green; desktop `cst-app-selftest: OK` and
--uitest all checks pass; phone (NX789J, Android 16, arm64-v8a) -- the APK
installs, starts on the workspace under /data/user/0/xyz.whcself.cstagent/
files/, a turn reaches a configured provider over HTTPS from inside the ART
process and renders the reply, rotation keeps the session and docks the
sidebar, and the toolbar/card icons draw.

Version 0.9.0.

mode = coding
This commit is contained in:
WHC
2026-10-11 15:22:10 +08:00
parent 82dee65942
commit c2320f946b
46 changed files with 1370 additions and 26 deletions
+78 -2
View File
@@ -76,13 +76,23 @@ if(CST_APP_GUI)
# main.cpp loads) and let CMake generate the qmldir of qml/ for us.
qt_policy(SET QTP0001 NEW)
qt_policy(SET QTP0004 NEW)
# The front end's platform tools: one file per platform, the same split
# core/ToolProvider.cpp makes. Android brings its own (JNI, owner
# "app:android"); the other platforms still use the Linux file, whose
# registration entry point is simply not called there.
if(ANDROID)
set(CST_APP_PLATFORM_SRC src/platform/android/AndroidTools.cpp)
else()
set(CST_APP_PLATFORM_SRC src/platform/linux/NotifyTool.cpp)
endif()
qt_add_executable(cst-agent-app
src/main.cpp
src/core/AgentCore.cpp
src/core/ToolProvider.cpp
src/ui/UiTest.cpp
src/ui/ThemeIcons.cpp
src/platform/linux/NotifyTool.cpp)
src/ui/SessionImages.cpp
${CST_APP_PLATFORM_SRC})
target_include_directories(cst-agent-app PRIVATE
src "${CMAKE_CURRENT_SOURCE_DIR}/../include")
target_compile_definitions(cst-agent-app PRIVATE CST_APP_VERSION="${PROJECT_VERSION}")
@@ -96,6 +106,52 @@ if(CST_APP_GUI)
set_target_properties(cst-agent-app PROPERTIES WIN32_EXECUTABLE TRUE)
endif()
# --- Android (APK) -----------------------------------------------------
# The engine is compiled in (top-level CMakeLists: -DCST_BUILD_TUI=OFF,
# -DCST_BUILD_WEBUI=OFF, so src/webui_stub.c stands in for the server and
# nothing listens on a port). The manifest, icons and strings live in
# app/android/, which androiddeployqt copies next to its own gradle
# project. minSdk 24 is the engine's own API_LEVEL (scripts/
# build_third_party_android.sh builds curl/mbedTLS for android-24).
if(ANDROID)
file(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/../include/cst_version.h"
_cst_ver_lines REGEX "#define CST_VERSION_(MAJOR|MINOR|PATCH) ")
foreach(_line ${_cst_ver_lines})
if(_line MATCHES "CST_VERSION_MAJOR ([0-9]+)")
set(_cst_vmaj "${CMAKE_MATCH_1}")
elseif(_line MATCHES "CST_VERSION_MINOR ([0-9]+)")
set(_cst_vmin "${CMAKE_MATCH_1}")
elseif(_line MATCHES "CST_VERSION_PATCH ([0-9]+)")
set(_cst_vpat "${CMAKE_MATCH_1}")
endif()
endforeach()
set_target_properties(cst-agent-app PROPERTIES
QT_ANDROID_PACKAGE_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/android"
QT_ANDROID_MIN_SDK_VERSION 24
QT_ANDROID_TARGET_SDK_VERSION 35
QT_ANDROID_VERSION_NAME "${_cst_vmaj}.${_cst_vmin}.${_cst_vpat}"
QT_ANDROID_VERSION_CODE 1
QT_ANDROID_ABIS "${ANDROID_ABI}")
message(STATUS "app: Android package "
"${_cst_vmaj}.${_cst_vmin}.${_cst_vpat}, ABI ${ANDROID_ABI}, "
"minSdk 24, targetSdk 35")
# Signing: androiddeployqt signs the APK itself and refuses a signed
# gradle output ("Could not copy unsigned APK"), so the keystore goes
# here, not into gradle. The path, alias and passwords arrive through
# the QT_ANDROID_KEYSTORE_* environment variables;
# scripts/build_apk_android.sh points them at the standard Android debug
# key it creates on first use, and CST_ANDROID_KEYSTORE overrides it.
if(DEFINED ENV{QT_ANDROID_KEYSTORE_PATH} AND NOT "$ENV{QT_ANDROID_KEYSTORE_PATH}" STREQUAL "")
set(QT_ANDROID_SIGN_APK TRUE)
message(STATUS "app: signing the APK with "
"$ENV{QT_ANDROID_KEYSTORE_PATH} ($ENV{QT_ANDROID_KEYSTORE_ALIAS})")
else()
message(STATUS "app: QT_ANDROID_KEYSTORE_PATH unset -- the APK stays "
"unsigned and 'adb install' will refuse it (INSTALL_PARSE_FAILED_NO_CERTIFICATES)")
endif()
endif()
# Theme is read by every component, so it is a QML singleton (one instance,
# no object needed in the tree).
set_source_files_properties(qml/Theme.qml PROPERTIES QT_QML_SINGLETON_TYPE TRUE)
@@ -105,7 +161,7 @@ if(CST_APP_GUI)
# requires absolute URLs" on every start).
foreach(_qml Main ChatView Sidebar SessionRow MessageCard ToolCard ThinkBlock
NoteLine TodoPanel Composer ConsentDialog AskDialog SettingsDialog
ThemeIcon Theme)
ThemeIcon Theme ImageStrip)
set_source_files_properties(qml/${_qml}.qml PROPERTIES QT_RESOURCE_ALIAS ${_qml}.qml)
endforeach()
qt_add_qml_module(cst-agent-app
@@ -126,6 +182,26 @@ if(CST_APP_GUI)
qml/AskDialog.qml
qml/SettingsDialog.qml
qml/ThemeIcon.qml
qml/ImageStrip.qml
qml/Theme.qml)
# The icons the window asks for, compiled in (app/icons, Breeze's scalable
# names copied at 24@3x). The desktop finds them in the platform's icon
# theme and only falls back to these (ThemeIcons.cpp); a machine without
# any theme -- Android -- has nothing else, and before this the QML glyph
# fallback was all that showed, which the phone's fonts cannot render.
set(CST_APP_ICONS
go-down mail-attachment application-menu settings-configure edit-copy
edit-delete arrow-down arrow-right list-add task-complete box
document-edit text-x-generic folder-open edit-find code-context
internet-services format-list-unordered help-hint)
set(_cst_icon_files "")
foreach(_icon ${CST_APP_ICONS})
list(APPEND _cst_icon_files "${CMAKE_CURRENT_SOURCE_DIR}/icons/${_icon}.svg")
endforeach()
qt_add_resources(cst-agent-app "cst_icons"
PREFIX "/cst-icons"
BASE "${CMAKE_CURRENT_SOURCE_DIR}/icons"
FILES ${_cst_icon_files})
endif()
endif()
+28 -3
View File
@@ -212,6 +212,27 @@ is refused with a reason, the composer shows one chip, "clear all" empties the
list, and — with an 8×8 image put on the clipboard — `clipboardHasImage()` and
`attachClipboard()` behave.
## Images
What a session keeps as a picture is shown, not only stored: a file
`view_image` returned, an image an MCP server sent, a screenshot the user
attached or pasted. Each one appears as a thumbnail in the transcript -- a
prompt's pictures under the message, a tool's under its card -- with the name,
size and origin in the tooltip. The height is capped so a 4K screenshot cannot
push the conversation off the screen, and a file the store no longer has says
which image it was instead of leaving a hole.
The view never touches the filesystem on its own authority: the thumbnails
come from `image://cstattach/<stored name>` (`app/src/ui/SessionImages.cpp`),
where the name is the one the engine wrote next to the session and is checked
with the engine's own `cst_attach_rel_ok()`, so only the session's attachment
directory can be read. The picture is decoded at the size of its thumbnail
rather than at its full resolution, and the provider's directory follows the
session: `AgentCore::attachmentDir()` is re-read on every switch.
A document (PDF, UTF-8 text) still arrives as its name in the message text --
those are sent to the model as text, not as a picture.
## Transcript, tool calls, streaming text
- A tool call is a card of its own, and the call **id** is what pairs it with
@@ -307,7 +328,7 @@ GUI only through queued signals.
|---|---|---|
| Linux | `cmake -S app -B build-app` (libcurl via pkg-config) | clipboard (shell helpers), notification (`notify-send` / D-Bus) |
| Windows | Qt's MinGW kit, `cmake -G Ninja -DCMAKE_PREFIX_PATH=<Qt>/lib/cmake`; the engine needs libcurl for Windows (see `docs/README-windows.md`) | notification via the WinRT/tray API, clipboard through the core's Win32 path |
| Android | engine compiled with the NDK inside the APK (`-DCST_TUI=0`, `CST_NO_WEBUI=1`); the app talks to Android APIs through Qt (JNI) | notification (Android), share sheet, clipboard, file picker |
| Android | engine compiled with the NDK inside the APK (`scripts/build_apk_android.sh`, `-DCST_TUI=0`, `CST_NO_WEBUI=1`); no port, QML talks to `AgentCore` in-process | none yet: `app/src/platform/android/AndroidTools.cpp` is a stub, so notification, share sheet, clipboard and the file picker are still missing |
| macOS | Qt from the installer, libcurl from the system | notification (`osascript`), clipboard (`pbpaste`/`pbcopy`) |
Android's long-term shape: the core lives in the APK (no Termux needed), the
@@ -328,8 +349,12 @@ The build flag that keeps Unix helpers out of the picture is
- [x] attachments: picker, drop on the window, Ctrl+V image paste
- [x] icons from the desktop's icon theme (Breeze), with a glyph fallback
- [x] contrast pass: faint text and selected rows are readable
- [x] images in the transcript: tool results and attachments are drawn as
thumbnails (`image://cstattach/<stored name>`, see the Images section)
- [x] Android APK: the engine is linked in (no port), builds, installs under
`xyz.whcself.cstagent` and runs on the device (see `docs/android.md`)
- [ ] a real turn driven through the window (needs a provider key)
- [ ] tool results that carry images (they are stored, not drawn yet)
- [ ] Android platform tools (JNI: notification, share sheet, clipboard, file
picker), the back gesture, the icon glyphs on the phone
- [ ] Kirigami (the rest of the Breeze look: colors, controls), tray,
global shortcuts
- [ ] Android APK (engine in-process, JNI platform tools)
+73
View File
@@ -0,0 +1,73 @@
<?xml version="1.0"?>
<!-- The APK's manifest (QT_ANDROID_PACKAGE_SOURCE_DIR in app/CMakeLists.txt):
androiddeployqt copies this file and res/ into its gradle project and
fills in the INSERT_* placeholders below (version code and name, library
name, app arguments). An XML comment may not contain a double hyphen, so
the placeholder syntax is only spelled out in the attributes themselves.
The SDK levels are not here on purpose: they come from
QT_ANDROID_MIN_SDK_VERSION and QT_ANDROID_TARGET_SDK_VERSION in
app/CMakeLists.txt, and androiddeployqt writes them into the generated
gradle.properties. The app name and icon come from res/ next to this
file (values/strings.xml, drawable/ic_launcher.xml).
WHAT THIS APP IS: the engine runs in this process (no server, no port) and
writes everything (workspace, session store, settings, credentials) under
the app's own data directory, which main.cpp points the engine at (see
androidRoot() there).
Permissions: INTERNET for the providers the user configures,
POST_NOTIFICATIONS for the app's notification_send tool (Android 13+
asks at runtime; the tool stays hidden until it is granted). No storage
permission: the file picker is the system one (SAF) and its input is
copied into the app's directory. No cleartext traffic: every provider URL
is https. -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="xyz.whcself.cstagent"
android:installLocation="auto"
android:versionCode="-- %%INSERT_VERSION_CODE%% --"
android:versionName="-- %%INSERT_VERSION_NAME%% --">
<!-- %%INSERT_PERMISSIONS -->
<!-- %%INSERT_FEATURES -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<supports-screens
android:anyDensity="true"
android:largeScreens="true"
android:normalScreens="true"
android:smallScreens="true" />
<application
android:name="org.qtproject.qt.android.bindings.QtApplication"
android:label="@string/app_name"
android:icon="@drawable/ic_launcher"
android:allowBackup="true"
android:hardwareAccelerated="true"
android:enableOnBackInvokedCallback="false">
<activity
android:name="org.qtproject.qt.android.bindings.QtActivity"
android:label="@string/app_name"
android:configChanges="orientation|uiMode|screenLayout|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|keyboard|keyboardHidden|navigation|density|mcc|mnc"
android:launchMode="singleTop"
android:screenOrientation="unspecified"
android:exported="true"
android:windowSoftInputMode="adjustResize">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<meta-data
android:name="android.app.lib_name"
android:value="-- %%INSERT_APP_LIB_NAME%% --" />
<meta-data
android:name="android.app.arguments"
android:value="-- %%INSERT_APP_ARGUMENTS%% --" />
</activity>
</application>
</manifest>
+30
View File
@@ -0,0 +1,30 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Launcher icon: the terminal prompt the app is about ("&gt;_"), the window's
own colors (bg #12161a, accent #5cc0f0, faint #8d959c). A vector drawable
works from API 24 up; a proper adaptive icon (mipmap-anydpi-v26 + a PNG
fallback) is polish for later. -->
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path
android:fillColor="#12161a"
android:pathData="M0,0h108v108h-108z" />
<path
android:fillColor="#00000000"
android:strokeColor="#5cc0f0"
android:strokeWidth="7"
android:strokeLineCap="round"
android:strokeLineJoin="round"
android:pathData="M32,38 L52,54 L32,70" />
<path
android:fillColor="#00000000"
android:strokeColor="#8d959c"
android:strokeWidth="7"
android:strokeLineCap="round"
android:pathData="M58,74 L78,74" />
</vector>
+5
View File
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- The name under the launcher icon. -->
<resources>
<string name="app_name">CST-agent</string>
</resources>
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3 5v2h16v-2h-16m0 5v2h16v-2h-16m0 5v2h16v-2h-16" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 446 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3.707031 7l-.707031.707031 6.125 6.125 1.875 1.875 1.875-1.875 6.125-6.125-.707031-.707031-6.125 6.125-1.167969 1.167969-1.167969-1.167969-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 549 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m7.707031 3l-.707031.707031 6.125 6.125 1.167969 1.167969-1.167969 1.167969-6.125 6.125.707031.707031 6.125-6.125 1.875-1.875-1.875-1.875-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 547 B

+10
View File
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 24 24" width="24" height="24">
<defs>
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" transform="translate(-.99999-.99999)" d="m4 4v16h16v-16h-16m1 1h14v14h-14v-14" fill="currentColor"/>
<path d="m3.5056345 10.977883v-7.5052069h14.878582v15.010415h-14.878582z" class="ColorScheme-Text" fill="currentColor" fill-opacity=".50196078"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 589 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m10.04492 3c-1.77683 0-2.521484.706446-2.521484 2.509766v2.441406c0 1.11382-.034195 2.404297-1.015625 2.404297h-.507813v1.289062h.507813c.98148 0 1.015625 1.290477 1.015625 2.404297v2.441406c0 1.80332.744654 2.509766 2.521484 2.509766v-1.183594c-1.18455 0-1.201172-.478327-1.201172-2.210937v-2.01563c0-.83093-.204705-2.067378-1.203125-2.580078.99842-.51271 1.203125-1.751092 1.203125-2.582032v-2.033203c0-1.7326.016592-2.210937 1.201172-2.210937v-1.183594m1.894531 0v1.183594c1.18456 0 1.216797.478337 1.216797 2.210937v2.033203c0 .83094.204705 2.069322 1.203125 2.582032-.99842.5127-1.203125 1.749148-1.203125 2.580078v2.01563c0 1.73261-.032187 2.210937-1.216797 2.210937v1.183594c1.77684 0 2.537109-.706446 2.537109-2.509766v-2.441406c0-1.11382.068644-2.404297 1.033204-2.404297h.490234v-1.289062h-.490234c-.96457 0-1.033204-1.290477-1.033204-2.404297v-2.441406c0-1.80332-.760269-2.509766-2.537109-2.509766" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m14.996094 3l-11.992188 11.992188h-.003906v4.00781h1 2 1.00781v-.003906l11.992188-11.992188-.001953-.001953.001953-.001953-4-4-.001953.001953-.001953-.001953m-1.998047 3.412109l2.589844 2.589844-7.587891 7.587891v-1.589844h-1-1v-1-.589844l6.998047-6.998047m-7.998047 7.998047v1.589844h1 1v1 .589844l-.410156.410156h-1.589844l-1-1v-1.589844l1-1" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 741 B

+10
View File
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" version="1.1" width="24" height="24">
<defs>
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" style="fill:currentColor; fill-opacity:1; stroke:none" d="M 3 3 L 3 17 L 7 17 L 7 19 L 17 19 L 17 10 L 13 6 L 12 6 L 9 3 L 3 3 Z M 4 4 L 8 4 L 8 6 L 7 6 L 7 16 L 4 16 L 4 4 Z M 8 7 L 12 7 L 12 11 L 16 11 L 16 18 L 8 18 L 8 7 Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 583 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-NegativeText { color: #da4453; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" class="ColorScheme-NegativeText" d="M8 3v2h1V4h4v1h1V3H8M4 6v1h14V6H4m2 2v11h10V8h-1v10H7V8H6"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 471 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 9 3 C 5.6759952 3 3 5.6759952 3 9 C 3 12.324005 5.6759952 15 9 15 C 10.481205 15 11.830584 14.465318 12.875 13.582031 L 18.292969 19 L 19 18.292969 L 13.582031 12.875 C 14.465318 11.830584 15 10.481205 15 9 C 15 5.6759952 12.324005 3 9 3 z M 9 4 C 11.770005 4 14 6.2299952 14 9 C 14 11.770005 11.770005 14 9 14 C 6.2299952 14 4 11.770005 4 9 C 4 6.2299952 6.2299952 4 9 4 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 774 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor" d="M 3 3 L 3 4 L 3 19 L 4 19 L 19 19 L 19 18 L 19 5 L 12. 5 L 10. 3 L 10 3 L 10 3 L 4 3 L 3 3 z M 9 8 L 18 8 L 18 18 L 4 18 L 4 10 L 7 10 L 7 10 L 7 10 L 9 8 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 528 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 4 3 C 3.4477153 3 3 3.4477153 3 4 C 3 4.5522847 3.4477153 5 4 5 C 4.5522847 5 5 4.5522847 5 4 C 5 3.4477153 4.5522847 3 4 3 z M 10 3 L 10 4 L 19 4 L 19 3 L 10 3 z M 10 5 L 10 6 L 16 6 L 16 5 L 10 5 z M 4 9 C 3.4477153 9 3 9.4477153 3 10 C 3 10.552285 3.4477153 11 4 11 C 4.5522847 11 5 10.552285 5 10 C 5 9.4477153 4.5522847 9 4 9 z M 10 9 L 10 10 L 19 10 L 19 9 L 10 9 z M 10 11 L 10 12 L 12 12 L 12 11 L 10 11 z M 4 15 C 3.4477153 15 3 15.447715 3 16 C 3 16.552285 3.4477153 17 4 17 C 4.5522847 17 5 16.552285 5 16 C 5 15.447715 4.5522847 15 4 15 z M 10 15 L 10 16 L 19 16 L 19 15 L 10 15 z M 10 17 L 10 18 L 16 18 L 16 17 L 10 17 z M 17 18 L 17 19 L 19 19 L 19 18 L 17 18 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.1 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3.707031 7l-.707031.707031 6.125 6.125 1.875 1.875 1.875-1.875 6.125-6.125-.707031-.707031-6.125 6.125-1.167969 1.167969-1.167969-1.167969-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 549 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 11 3 C 6.568 3 3 6.568 3 11 C 3 15.432 6.568 19 11 19 C 15.432 19 19 15.432 19 11 C 19 6.568 15.432 3 11 3 z M 11 4 C 14.878 4 18 7.122 18 11 C 18 14.878 14.878 18 11 18 C 7.122 18 4 14.878 4 11 C 4 7.122 7.122 4 11 4 z M 10 6 L 10 8 L 12 8 L 12 6 L 10 6 z M 10 9 L 10 16 L 12 16 L 12 9 L 10 9 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 696 B

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 24 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 10 4 L 10 11 L 3 11 L 3 12 L 10 12 L 10 19 L 11 19 L 11 12 L 18 12 L 18 11 L 11 11 L 11 4 L 10 4 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 499 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 12.902344 3 L 3.734375 12.167969 C 2.167415 13.734929 2.167415 16.257279 3.734375 17.824219 C 5.301315 19.391169 7.823675 19.391169 9.390625 17.824219 L 17.851562 9.3632812 L 18.558594 8.65625 C 19.733804 7.48104 19.733794 5.5892625 18.558594 4.4140625 C 17.383384 3.2388525 15.491616 3.2388525 14.316406 4.4140625 L 13.609375 5.1210938 L 5.1484375 13.582031 C 4.3649575 14.365511 4.3649675 15.626686 5.1484375 16.410156 C 5.9319175 17.193636 7.1930825 17.193646 7.9765625 16.410156 L 16.4375 7.9492188 L 15.730469 7.2421875 L 7.9765625 14.996094 L 7.2695312 15.703125 C 6.8777912 16.094865 6.2471888 16.094865 5.8554688 15.703125 C 5.4637287 15.311405 5.4637187 14.680813 5.8554688 14.289062 L 14.316406 5.828125 L 15.023438 5.1210938 C 15.806917 4.3376037 17.068092 4.3376037 17.851562 5.1210938 C 18.635042 5.9045538 18.635042 7.1657387 17.851562 7.9492188 L 8.6835938 17.117188 C 7.5083838 18.292397 5.6166162 18.292397 4.4414062 17.117188 C 3.2661862 15.941977 3.2661862 14.05021 4.4414062 12.875 L 13.609375 3.7070312 L 12.902344 3 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 11.5 3 C 10.286139 3 9.2809778 3.8559279 9.0507812 5 L 3 5 L 3 6 L 9.0507812 6 C 9.2809778 7.1440721 10.286139 8 11.5 8 C 12.713861 8 13.719022 7.1440721 13.949219 6 L 19 6 L 19 5 L 13.949219 5 C 13.719022 3.8559279 12.713861 3 11.5 3 z M 5.5 14 C 4.1149999 14 3 15.115 3 16.5 C 3 17.885 4.1149999 19 5.5 19 C 6.7138604 19 7.7190223 18.144072 7.9492188 17 L 19 17 L 19 16 L 7.9492188 16 C 7.7190223 14.855928 6.7138604 14 5.5 14 z M 5.5 15 C 6.3310001 15 7 15.669 7 16.5 C 7 17.331 6.3310001 18 5.5 18 C 4.6689999 18 4 17.331 4 16.5 C 4 15.669 4.6689999 15 5.5 15 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 966 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style id="current-color-scheme" type="text/css">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" d="M 19,9 12,16 8,12 9.4896719,10.543135 12,13 17.482382,7.5492102 Z M 4,3 V 5 H 3 V 6 H 4 V 7 H 3 v 1 h 1 v 6 H 3 v 1 h 1 v 1 H 3 v 1 h 1 v 2 h 3 12 v -1 -7 h -1 v 1 1 2 1 2 H 10 9 7 V 16 15 13 12 10 9 7 6 4 h 2 1 8 v 2 h 1 V 3 H 7 Z m 4,3 v 2 h 2 V 6 Z m 3,0 v 1 h 6 V 6 Z" style="fill:currentColor;fill-opacity:1;stroke:none"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 669 B

+14
View File
@@ -0,0 +1,14 @@
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg xmlns="http://www.w3.org/2000/svg" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" width="24" version="1.1" height="24" viewBox="0 0 24 24">
<defs id="defs3871"/>
<metadata id="metadata3874"/>
<g transform="translate(1,1)">
<g inkscape:label="Capa 1" inkscape:groupmode="layer" id="layer1" transform="matrix(1 0 0 1 -326 -534.3622)">
<rect width="8" x="329" y="537.36218" height="1.00002" style="fill:#6c7a89" id="rect4279"/>
<rect width="16" x="329" y="541.36218" height="1" style="fill:#6c7a89" id="rect4281"/>
<rect width="16" x="329" y="548.36218" height="1" style="fill:#6c7a89" id="rect4283"/>
<rect width="8" x="329" y="545.36218" height="1" style="fill:#6c7a89" id="rect4285"/>
<rect width="12" x="329" y="552.36218" height="0.999983" style="fill:#6c7a89" id="rect4287"/>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 919 B

+2
View File
@@ -40,6 +40,7 @@ Item {
role: model.role
text: model.text
complete: model.complete
images: model.images !== undefined ? model.images : ""
/* html comes from the streaming renderer (Main.qml) or,
* for a replayed session, from here once. */
html: model.html.length > 0 ? model.html
@@ -65,6 +66,7 @@ Item {
output: model.output
exitCode: model.exitCode
hasResult: model.complete
images: model.images !== undefined ? model.images : ""
}
}
Component {
+110
View File
@@ -0,0 +1,110 @@
import QtQuick
import QtQuick.Controls
import QtQuick.Layouts
// The images a message or a tool call carries: thumbnails from the session's
// own store (image://cstattach/, app/src/ui/SessionImages.cpp). The names are
// the ones the engine wrote next to the session -- view_image, an MCP image, a
// picture the user attached -- never a path the view invented.
//
// One row of thumbnails: the height is capped so a screenshot cannot push the
// conversation off the screen, the width follows the picture's shape.
Flow {
id: strip
objectName: "imageStrip"
/* One line per picture, written by the engine side (AgentCore.cpp:
* "<stored name>|<name>|<origin>|<w>x<h>"). A row carries this as text
* because a ListModel role cannot hold a list. */
property string text: ""
property var images: decode(text)
spacing: 6
function decode(t) {
const out = []
const lines = String(t).split("\n")
for (let i = 0; i < lines.length; i++) {
if (!lines[i].length)
continue
const f = lines[i].split("|")
if (!f[0].length)
continue
let w = 0, h = 0
const dim = String(f[3] || "").split("x")
if (dim.length === 2) {
w = parseInt(dim[0], 10) || 0
h = parseInt(dim[1], 10) || 0
}
out.push({ url: "image://cstattach/" + f[0], rel: f[0],
name: f[1] || f[0], origin: f[2] || "", w: w, h: h })
}
return out
}
Repeater {
model: strip.images
delegate: Rectangle {
id: thumb
required property var modelData
required property int index
readonly property real ratio: {
const w = Number(modelData.w), h = Number(modelData.h)
return (w > 0 && h > 0) ? w / h : 1.5
}
readonly property int cap: 150 // the tallest a thumbnail gets
objectName: "imageThumb" + index
width: Math.max(56, Math.min(Math.round(cap * ratio), 320))
height: Math.max(40, Math.min(cap, Math.round(width / ratio)))
radius: Theme.radiusSm
color: Theme.bg
border.width: 1
border.color: mouse.containsMouse ? Theme.accentDim : Theme.border
clip: true
Image {
id: pic
anchors.fill: parent
anchors.margins: 2
source: String(thumb.modelData.url)
sourceSize.width: 512
sourceSize.height: 512
fillMode: Image.PreserveAspectFit
/* Not asynchronous: the provider already hands back a
* thumbnail-sized picture, so there is nothing to wait for. */
asynchronous: false
visible: status === Image.Ready
}
// A file the store no longer has is not worth shouting about: the
// frame says which image it was.
Text {
anchors.centerIn: parent
width: parent.width - 8
visible: pic.status === Image.Error
text: String(thumb.modelData.name)
elide: Text.ElideMiddle
horizontalAlignment: Text.AlignHCenter
color: Theme.textFaint
font.pixelSize: Theme.smallSize
}
MouseArea {
id: mouse
anchors.fill: parent
hoverEnabled: true
acceptedButtons: Qt.NoButton
}
ToolTip.visible: mouse.containsMouse
ToolTip.delay: 400
ToolTip.text: {
const m = thumb.modelData
let s = String(m.name)
if (m.w > 0 && m.h > 0)
s += " · " + m.w + "×" + m.h
if (m.origin && m.origin !== "user")
s += " · " + m.origin
return s
}
}
}
}
+33 -2
View File
@@ -61,7 +61,7 @@ ApplicationWindow {
function addRow(kind, role, state) {
transcript.append({ kind: kind, role: role, text: "", html: "", complete: false,
name: "", callId: "", state: state, detail: "", output: "",
exitCode: 0 })
exitCode: 0, images: "" })
}
// The engine streams one kind at a time: close whatever was open before.
@@ -112,7 +112,7 @@ ApplicationWindow {
function submit(text) {
transcript.append({ kind: "message", role: "user", text: text, html: "", complete: true,
name: "", callId: "", state: "", detail: "", output: "",
exitCode: 0 })
exitCode: 0, images: "" })
agent.send(text)
}
@@ -545,6 +545,29 @@ ApplicationWindow {
win.statusText = ""
}
/* The images the prompt just carried (AgentCore::promptImages): the
* composer has appended the user row, so they attach to it. */
function enginePromptImages(images) {
if (!images.length)
return
for (let i = transcript.count - 1; i >= 0; i--) {
const r = transcript.get(i)
if (r.kind === "message" && r.role === "user") {
transcript.setProperty(i, "images", images)
return
}
}
}
/* Images a tool produced, on the card of the call that made them. */
function engineToolImages(callId, images) {
if (!images.length)
return
const i = win.findTool(callId, "")
if (i >= 0)
transcript.setProperty(i, "images", images)
}
/* For --uitest: one transcript row, or null. */
function rowAt(i) {
return (i >= 0 && i < transcript.count) ? transcript.get(i) : null
@@ -584,6 +607,14 @@ ApplicationWindow {
win.engineToolResult(id, name, exitCode, output)
}
function onToolImages(callId, images) {
win.engineToolImages(callId, images)
}
function onPromptImages(images) {
win.enginePromptImages(images)
}
function onPermissionRequested(tool, detail, preview) {
consent.tool = tool
consent.detail = detail
+8
View File
@@ -11,6 +11,7 @@ Rectangle {
property string text: ""
property string html: ""
property bool complete: true
property string images: ""
radius: Theme.radius
color: role === "user" ? Theme.surfaceAlt : Theme.surface
@@ -105,5 +106,12 @@ Rectangle {
wrapMode: Text.Wrap
onLinkActivated: function (link) { Qt.openUrlExternally(link) }
}
// What the prompt carried: pictures from the store, not their names.
ImageStrip {
Layout.fillWidth: true
visible: card.images.length > 0
text: card.images
}
}
}
+9
View File
@@ -14,6 +14,7 @@ Rectangle {
property int exitCode: 0
property bool hasResult: false
property bool expanded: false
property string images: ""
readonly property bool running: state === "pending" || state === "ready" || state === "running"
readonly property color stateColor: state === "denied" ? Theme.err
@@ -125,6 +126,14 @@ Rectangle {
}
}
// What the tool made (view_image, an MCP image): shown as it is, with
// the output left folded -- the picture is the result.
ImageStrip {
Layout.fillWidth: true
visible: card.images.length > 0
text: card.images
}
// Folded view: the first line of the result, so a done card still says
// something without being opened.
Text {
+96 -4
View File
@@ -17,6 +17,7 @@
#include <cstdlib>
#include "cst_agent.h"
#include "cst_attach.h"
#include "cst_models.h"
#include "cst_provider_switch.h"
#include "cst_web_md.h"
@@ -53,6 +54,7 @@ AgentCore::AgentCore(QObject *parent) : QObject(parent) {
m_hooks.on_think_delta = hookThink;
m_hooks.on_tool_event = hookToolEvent;
m_hooks.on_tool_result = hookToolResult;
m_hooks.on_tool_images = hookToolImages;
m_hooks.on_todo = hookTodo;
m_hooks.on_permission = hookPermission;
m_hooks.on_ask_user = hookAskUser;
@@ -172,7 +174,12 @@ void AgentCore::close() {
void AgentCore::send(const QString &prompt) {
if (!m_agent || m_running || prompt.trimmed().isEmpty())
return;
/* The prompt takes the pending images with it: the view shows them on the
* message the composer just appended, before the turn can say anything. */
const QString sending = pendingImages();
setRunning(true);
if (!sending.isEmpty())
emit promptImages(sending);
m_turn = new Turn(this, prompt);
m_turn->start();
}
@@ -314,9 +321,41 @@ QVariantMap sessionRow(const QString &kind, const QString &role) {
r["detail"] = QString();
r["output"] = QString();
r["exitCode"] = 0;
r["images"] = QString();
return r;
}
/* One field of one picture line: names come from the user and from tool
* output, so a separator may not split a line in two. */
QString field(const char *s) {
QString out = utf8(s);
out.replace(QLatin1Char('|'), QLatin1Char('/'));
out.replace(QLatin1Char('\n'), QLatin1Char(' '));
return out;
}
/* The images of one message or tool result, one line per picture:
*
* <stored name>|<name>|<origin>|<w>x<h>
*
* A list role is not an option: a ListModel keeps what fits in a QVariant of
* a simple type -- a list reads back as undefined there, and setProperty drops
* the role -- so the rows carry this text and ImageStrip.qml decodes it. A
* document (PDF, text) is not a picture and stays out of it. */
QString imagesText(const cst_attachment_t *list, int n) {
QStringList lines;
for (int i = 0; list && i < n; i++) {
const cst_attachment_t *a = &list[i];
if (!cst_attach_is_image(a))
continue;
lines << QStringLiteral("%1|%2|%3|%4x%5")
.arg(field(a->rel), field(a->name), field(a->origin))
.arg(a->width)
.arg(a->height);
}
return lines.join(QLatin1Char('\n'));
}
/* The open tool row this result belongs to, -1 when none. Rows are paired by
* call id first: the same tool may run twice in a turn, and pairing by name
* would hang the first card on "running" forever. Only a call without an id
@@ -354,6 +393,20 @@ QString AgentCore::sessionId() const {
return utf8(id);
}
/* Where the session keeps its images: what image://cstattach/ serves, and
* what the view asks for when the session changes. */
QString AgentCore::attachmentDir() const {
if (!m_agent)
return QString();
const char *sess = cst_agent_session_path(m_agent);
if (!sess || !sess[0])
return QString();
char dir[4096] = "";
if (cst_attach_dir(sess, dir, sizeof(dir)) != 0)
return QString();
return utf8(dir);
}
QVariantList AgentCore::sessions() const {
QVariantList out;
if (m_workspace.isEmpty())
@@ -404,12 +457,14 @@ QVariantList AgentCore::history() const {
r["name"] = name;
r["callId"] = callId;
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows.append(r);
} else {
QVariantMap r = rows.at(at).toMap();
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows[at] = r;
@@ -426,13 +481,16 @@ QVariantList AgentCore::history() const {
if (m->role == CST_ROLE_USER) {
QVariantMap r = sessionRow(QStringLiteral("message"), QStringLiteral("user"));
QString text = utf8(m->content);
if (m->nattach > 0) {
QStringList names;
for (int a = 0; a < m->nattach; a++)
/* A document keeps its name in the text; a picture is shown instead
* (the images below), so only the rest is worth naming. */
QStringList names;
for (int a = 0; a < m->nattach; a++)
if (!cst_attach_is_image(&m->attach[a]))
names << utf8(m->attach[a].name);
if (!names.isEmpty())
text += QStringLiteral("\n\n[attached: %1]").arg(names.join(", "));
}
r["text"] = text;
r["images"] = imagesText(m->attach, m->nattach);
r["complete"] = true;
rows.append(r);
continue;
@@ -802,6 +860,30 @@ QVariantMap AgentCore::usage() const {
/* --- attachments ---------------------------------------------------------- */
/* What the prompt about to be sent carries, ready for the view. The stored
* name is the engine's (the file already lives next to the session), the
* display name is the one the window kept when it copied the file in. */
QString AgentCore::pendingImages() const {
QStringList lines;
if (!m_agent)
return QString();
const int n = cst_agent_pending_count(m_agent);
for (int i = 0; i < n; i++) {
if (!cst_agent_pending_is_image(m_agent, i))
continue;
const char *rel = cst_agent_pending_rel(m_agent, i);
if (!rel || !rel[0])
continue;
const QString name =
i < m_attachNames.size() ? m_attachNames.at(i) : utf8(rel);
/* The size is not known before the engine stores the file; the strip
* falls back to a picture's usual shape until it is. */
lines << QStringLiteral("%1|%2|user|0x0")
.arg(field(rel), field(name.toUtf8().constData()));
}
return lines.join(QLatin1Char('\n'));
}
QStringList AgentCore::pendingAttachments() const {
QStringList out;
if (!m_agent)
@@ -937,6 +1019,16 @@ void AgentCore::hookToolResult(const ctl_tool_call_t *call, const char *out,
[self, id, n, exit_code, o] { emit self->toolResult(id, n, exit_code, o); });
}
void AgentCore::hookToolImages(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString images = imagesText(list, n);
if (images.isEmpty())
return; /* a call that made no picture needs no signal */
const QString id = QString::fromUtf8(call && call->id ? call->id : "");
self->post([self, id, images] { emit self->toolImages(id, images); });
}
void AgentCore::hookTodo(const char *text, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString t = QString::fromUtf8(text ? text : "");
+20 -1
View File
@@ -63,7 +63,8 @@ class AgentCore : public QObject {
Q_INVOKABLE QVariantList sessions() const;
/* The open session as transcript rows, in exactly the shape the live
* signals build (kind, role, text, name, state, detail, output, exitCode,
* complete), so a reopened session renders like the turn that wrote it. */
* complete, images), so a reopened session renders like the turn that wrote
* it. */
Q_INVOKABLE QVariantList history() const;
/* None of these run during a turn; all return false when one does. */
Q_INVOKABLE bool newSession();
@@ -128,6 +129,16 @@ class AgentCore : public QObject {
* clipboard) or stay with the editor (text). */
Q_INVOKABLE bool clipboardHasImage() const;
Q_INVOKABLE void clearPending();
/* The images among the pending attachments, in the engine's order, in the
* same text history() and toolImages() carry (see imagesText()): one line
* per picture, "<stored name>|<name>|<origin>|<w>x<h>". Empty for a PDF or
* a text file, and for a picture whose stored name the engine did not
* report. */
Q_INVOKABLE QString pendingImages() const;
/* Where the current session keeps its stored images: the directory
* image://cstattach/ serves (SessionImages.cpp). "" until the engine is
* open. It follows the session. */
Q_INVOKABLE QString attachmentDir() const;
/* 1 the model is shown images, 0 it is not (they go as notes), -1 unknown. */
Q_INVOKABLE int seesImages() const;
@@ -176,6 +187,12 @@ class AgentCore : public QObject {
const QString &detail);
void toolResult(const QString &id, const QString &name, int exitCode,
const QString &output);
/* Images a tool produced (view_image, an MCP image), right after its
* result, as the text ImageStrip.qml decodes. */
void toolImages(const QString &callId, const QString &images);
/* The images the prompt that was just sent carries, in the same shape,
* fired before the turn's first delta. */
void promptImages(const QString &images);
void todoChanged(const QString &text);
void permissionRequested(const QString &tool, const QString &detail,
const QString &preview);
@@ -203,6 +220,8 @@ class AgentCore : public QObject {
const char *detail, void *ud);
static void hookToolResult(const ctl_tool_call_t *call, const char *out,
size_t len, int exit_code, void *ud);
static void hookToolImages(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud);
static void hookTodo(const char *text, void *ud);
static int hookPermission(const char *tool, const char *detail,
const char *preview, void *ud);
+39 -1
View File
@@ -27,9 +27,24 @@
#include "core/AgentCore.h"
#include "core/ToolProvider.h"
#include "ui/UiTest.h"
#include "ui/SessionImages.h"
#include "ui/ThemeIcons.h"
#include "cst_version.h"
#if defined(Q_OS_ANDROID)
/* Where an Android app is allowed to write: its own data directory. There is
* no usable $HOME, no writable "/" (QDir::currentPath()) and no /tmp, while
* the engine wants a config dir ($XDG_CONFIG_HOME/cst-agent, os_config_dir())
* and a scratch dir (the first writable entry of os_temp_dir()'s list). All
* of them are pointed here -- once, before the engine exists. */
QString androidRoot() {
const QString root =
QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);
return root.isEmpty() ? QDir::tempPath() + QStringLiteral("/cst-agent")
: root;
}
#endif
namespace {
int runSelftest() {
@@ -141,8 +156,13 @@ int main(int argc, char **argv) {
parser.process(app);
QString ws = parser.value(wsOpt);
if (ws.isEmpty())
if (ws.isEmpty()) {
#if defined(Q_OS_ANDROID)
ws = androidRoot() + QStringLiteral("/workspace");
#else
ws = QDir::currentPath();
#endif
}
ws = QFileInfo(ws).absoluteFilePath();
/* --uitest drives a scratch workspace when none was given: the session
@@ -160,6 +180,17 @@ int main(int argc, char **argv) {
}
}
#if defined(Q_OS_ANDROID)
/* Before the engine is created: its settings, credentials and session store
* all resolve their paths from the environment. */
const QString root = androidRoot();
for (const QString &sub : {QStringLiteral("config"), QStringLiteral("tmp")})
QDir().mkpath(root + QLatin1Char('/') + sub);
qputenv("XDG_CONFIG_HOME", (root + QStringLiteral("/config")).toUtf8());
qputenv("TMPDIR", (root + QStringLiteral("/tmp")).toUtf8());
qputenv("HOME", root.toUtf8());
#endif
AgentCore core;
QString err;
if (!core.open(ws, parser.value(providerOpt), parser.value(modelOpt),
@@ -195,6 +226,13 @@ int main(int argc, char **argv) {
QQmlApplicationEngine engine;
/* image://csttheme/ plus the desktop's icon theme behind it. */
app::installThemeIcons(&engine);
/* image://cstattach/: the session's own store, so the pictures a tool made
* (view_image, an MCP image) and the ones the user attached are shown, not
* only named. The directory follows the session. */
app::SessionImages *images = app::installSessionImages(&engine);
images->setDir(core.attachmentDir());
QObject::connect(&core, &AgentCore::sessionChanged, images,
[images, &core] { images->setDir(core.attachmentDir()); });
engine.rootContext()->setContextProperty(QStringLiteral("agent"), &core);
engine.rootContext()->setContextProperty(QStringLiteral("initialWidth"), winW);
engine.rootContext()->setContextProperty(QStringLiteral("initialHeight"), winH);
+43
View File
@@ -0,0 +1,43 @@
/* Platform tools on Android, implemented by the app: the same contract as
* app/src/platform/linux/NotifyTool.cpp, with Android APIs behind it.
*
* The engine only ever sees the capability: `platforms = kAndroidOnly` (the
* engine reports "android" when it was compiled with the NDK, see
* src/tools/registry.c: cst_platform_name()) plus an `available` probe decide
* whether a tool reaches the prompt and tools[] at all -- a Termux build on
* the same device reports "termux" and gets the android tools too, an APK
* never gets the desktop ones. `owner = "app:android"` is what
* unregisterPlatformTools() drops again.
*
* WHERE THE CODE RUNS: `run_ex` is called on the turn's worker thread, never
* on the GUI thread. JNI calls are fine from any thread (QJniEnvironment
* attaches it), Qt GUI objects are not -- a tool that needs the widget tree
* has to hop back with a blocking queued call.
*
* WHAT GOES HERE (one at a time, each with its own probe):
* notification_send NotificationChannel + NotificationManager through
* JNI; hidden until POST_NOTIFICATIONS is granted
* (Android 13+), which the UI asks for on first use.
* clipboard_get/set the phone clipboard through Qt's QClipboard, taking
* the name of the desktop tool that cannot work here
* (see app/README.md: a front end may shadow a built-in
* that has no working backend on its host).
* share ACTION_SEND of a file or text through the system share
* sheet -- the phone's answer to "show this to the user".
*
* Nothing is registered yet: the first APK ships the engine, the window and
* the tool contract, and the tools above arrive next (docs/android.md). */
#include <QString>
#include "core/AgentCore.h"
#include "core/ToolProvider.h"
namespace app {
int platformRegisterAndroid(AgentCore *core) {
if (!core)
return 0;
return 0; /* no Android tools yet */
}
} // namespace app
+81
View File
@@ -0,0 +1,81 @@
#include "ui/SessionImages.h"
#include <QFileInfo>
#include <QImageReader>
#include <QQmlApplicationEngine>
#include "cst_attach.h"
namespace app {
namespace {
/* "img-1a2b3c4d.png?w=160": the stored name, then what the view asked for.
* The size comes from the QML side as sourceSize, which Qt passes in as
* requestedSize; the query string is accepted too, so a plain URL can ask. */
QString relOf(const QString &id) {
const int q = id.indexOf(QLatin1Char('?'));
return q < 0 ? id : id.left(q);
}
} // namespace
SessionImages::SessionImages()
: QQuickImageProvider(QQuickImageProvider::Image) {}
void SessionImages::setDir(const QString &dir) {
QMutexLocker lock(&m_mu);
m_dir = dir;
}
QImage SessionImages::requestImage(const QString &id, QSize *size,
const QSize &requestedSize) {
const QString rel = relOf(id);
if (!cst_attach_rel_ok(rel.toUtf8().constData()))
return QImage(); /* QML sees Image.Error and falls back to the name */
QString dir;
{
QMutexLocker lock(&m_mu);
dir = m_dir;
}
if (dir.isEmpty())
return QImage();
const QString path = dir + QLatin1Char('/') + rel;
if (!QFileInfo::exists(path))
return QImage();
QImageReader reader(path);
reader.setAutoTransform(true);
const QSize full = reader.size();
if (!full.isValid()) {
/* No size in the header (or not a picture at all): hand back what Qt
* makes of it and let the view scale it. */
const QImage img = reader.read();
if (size)
*size = img.size();
return img;
}
/* Shrink to what the view shows, never up: a screenshot is decoded at the
* size of its thumbnail instead of at 4K. */
const bool smaller = requestedSize.isValid() && requestedSize.width() > 0 &&
requestedSize.height() > 0 &&
(full.width() > requestedSize.width() ||
full.height() > requestedSize.height());
if (smaller) {
const QSize scaled = full.scaled(requestedSize, Qt::KeepAspectRatio);
reader.setScaledSize(scaled);
if (size)
*size = scaled;
} else if (size) {
*size = full;
}
return reader.read();
}
SessionImages *installSessionImages(QQmlApplicationEngine *engine) {
SessionImages *images = new SessionImages;
if (engine)
engine->addImageProvider(QStringLiteral("cstattach"), images);
return images;
}
} // namespace app
+45
View File
@@ -0,0 +1,45 @@
/* The session's stored images as a QML image provider:
*
* image://cstattach/<stored name> (see cst_attach.h: "img-1a2b3c4d.png")
*
* The files live in the current session's attachment directory, which is what
* AgentCore::attachmentDir() names. Only that directory is served, and only
* names that pass the engine's own cst_attach_rel_ok() -- a transcript can
* carry hand-edited JSON, and nothing in it may reach outside the store.
*
* requestImage() runs on Quick's render thread while setDir() comes from the
* window's thread, hence the mutex. */
#ifndef CST_APP_SESSION_IMAGES_H
#define CST_APP_SESSION_IMAGES_H
#include <QImage>
#include <QMutex>
#include <QQuickImageProvider>
#include <QString>
class QQmlApplicationEngine;
namespace app {
class SessionImages : public QQuickImageProvider {
public:
SessionImages();
/* The directory holding the current session's images; "" disables serving. */
void setDir(const QString &dir);
QImage requestImage(const QString &id, QSize *size,
const QSize &requestedSize) override;
private:
mutable QMutex m_mu;
QString m_dir;
};
/* Registers image://cstattach/ on the engine and hands the provider back, so
* the caller can keep its directory in step with the agent's session. */
SessionImages *installSessionImages(QQmlApplicationEngine *engine);
} // namespace app
#endif // CST_APP_SESSION_IMAGES_H
+17 -4
View File
@@ -48,6 +48,21 @@ void adoptIconTheme() {
}
}
/* The bundled set: app/icons/*.svg, compiled in as :/cst-icons/<name>.svg by
* app/CMakeLists.txt. A machine without an icon theme -- Android, where the
* Qt theme lookup finds nothing and QML was left showing its glyph fallback,
* which the phone fonts cannot draw -- gets the same icons from here. */
QIcon bundledIcon(const QString &name) {
return QIcon(QStringLiteral(":/cst-icons/") + name + QStringLiteral(".svg"));
}
QIcon iconForName(const QString &name) {
const QIcon themed = QIcon::fromTheme(name);
if (!themed.isNull())
return themed;
return bundledIcon(name);
}
/* "?color=%23eff0f1": the tint, and nothing else so far. */
QColor colorOption(const QString &id, QString *name) {
QColor color;
@@ -85,7 +100,7 @@ QImage ThemeIconProvider::requestImage(const QString &id, QSize *size,
if (name.isEmpty())
return QImage();
const QIcon icon = QIcon::fromTheme(name);
const QIcon icon = iconForName(name);
if (icon.isNull())
return QImage(); /* QML shows its fallback glyph */
@@ -116,8 +131,6 @@ void installThemeIcons(QQmlApplicationEngine *engine) {
QString iconThemeName() { return QIcon::themeName(); }
bool themeIconAvailable(const QString &name) {
return QIcon::hasThemeIcon(name);
}
bool themeIconAvailable(const QString &name) { return !iconForName(name).isNull(); }
} // namespace app
+125
View File
@@ -10,10 +10,12 @@
#include <QGuiApplication>
#include <QClipboard>
#include <QBuffer>
#include <QFile>
#include <QImage>
#include <QJsonDocument>
#include <QJsonObject>
#include <QPainter>
#include <QPointingDevice>
#include <QQuickItem>
#include <QQmlApplicationEngine>
@@ -27,6 +29,8 @@
#include <cstring>
#include "cst_agent.h"
#include "cst_attach.h"
#include "ui/SessionImages.h"
#include "ui/ThemeIcons.h"
namespace app {
@@ -118,7 +122,26 @@ QString writeFixtureSession(const QString &ws, const QString &id) {
memset(&m, 0, sizeof(m));
m.role = CST_ROLE_USER;
m.content = strdup("what is here?");
/* The prompt carries a picture, stored exactly the way the engine stores
* one (the file next to the session, the message naming it). */
{
QImage pic(320, 200, QImage::Format_RGB32);
pic.fill(Qt::darkCyan);
QPainter painter(&pic);
painter.fillRect(QRect(0, 0, 160, 100), Qt::darkYellow);
painter.end();
QByteArray png;
QBuffer buf(&png);
buf.open(QIODevice::WriteOnly);
pic.save(&buf, "PNG");
char aerr[256] = "";
m.attach = cst_attach_import_bytes(path, "shot.png", png.constData(),
static_cast<size_t>(png.size()), aerr,
sizeof(aerr));
m.nattach = m.attach ? 1 : 0;
}
cst_session_append(s, &m);
cst_attachments_free(m.attach, m.nattach);
memset(&m, 0, sizeof(m));
m.role = CST_ROLE_ASSISTANT;
m.content = strdup("let me look. ");
@@ -175,6 +198,12 @@ int runUiTest(AgentCore &core, const QString &size) {
QQmlApplicationEngine engine;
installThemeIcons(&engine); /* the window's icons, same as main() */
/* image://cstattach/: the session's stored images, pointed at the session
* the engine has open (and re-pointed on every switch). */
SessionImages *images = installSessionImages(&engine);
images->setDir(core.attachmentDir());
QObject::connect(&core, &AgentCore::sessionChanged, images,
[images, &core] { images->setDir(core.attachmentDir()); });
engine.rootContext()->setContextProperty(QStringLiteral("agent"), &core);
engine.rootContext()->setContextProperty(QStringLiteral("initialWidth"), width);
engine.rootContext()->setContextProperty(QStringLiteral("initialHeight"), height);
@@ -303,6 +332,36 @@ int runUiTest(AgentCore &core, const QString &size) {
clickMouse(win, QStringLiteral("sessionRow:") + fixture);
check(core.sessionId() == fixture && list && list->property("count").toInt() == 5,
QStringLiteral("switching back replays it again"));
{
bool promptImage = false;
const QVariantList rows = core.history();
for (const QVariant &v : rows) {
const QVariantMap r = v.toMap();
if (r.value(QStringLiteral("role")).toString() == QLatin1String("user") &&
r.value(QStringLiteral("images")).toString().contains(
QLatin1String("|")))
promptImage = true;
}
check(promptImage,
QStringLiteral("a stored prompt image comes back with its row"));
/* The rows are in the model by now; the cards -- and the thumbnails in
* them -- are built on the view's next layout pass. A frame forces it,
* and that frame is also the picture a person looks at. */
maybeShot(win, QStringLiteral("-images"));
QTest::qWait(150);
if (qEnvironmentVariableIsSet("CST_APP_DUMP")) {
QVariant v;
QMetaObject::invokeMethod(win, "rowJson", Q_RETURN_ARG(QVariant, v),
Q_ARG(QVariant, 0));
std::printf("note: replayed row 0: %s\n", qPrintable(v.toString()));
std::printf("note: %d strips and %d thumbs in the tree\n",
countNamed(win->contentItem(), QStringLiteral("imageStrip")),
countNamed(win->contentItem(), QStringLiteral("imageThumb")));
}
check(countNamed(win->contentItem(), QStringLiteral("imageThumb")) >= 1,
QStringLiteral("the replayed transcript shows it as a thumbnail (%1)")
.arg(countNamed(win->contentItem(), QStringLiteral("imageThumb"))));
}
/* --- provider, model, key: the settings dialog ------------------------ */
check(core.providers().size() > 0 &&
@@ -558,6 +617,72 @@ int runUiTest(AgentCore &core, const QString &size) {
check(false, QStringLiteral("assistant deltas are reachable"));
}
/* --- the image provider: only the session's own files ----------------- */
{
const QString dir = core.attachmentDir();
check(!dir.isEmpty() && dir.contains(QLatin1String("/attachments/")),
QStringLiteral("the agent names its image directory (%1)").arg(dir));
QImage pic(32, 16, QImage::Format_RGB32);
pic.fill(Qt::darkGreen);
QByteArray png;
QBuffer buf(&png);
buf.open(QIODevice::WriteOnly);
pic.save(&buf, "PNG");
char aerr[256] = "";
cst_attachment_t *stored = cst_attach_import_bytes(
cst_agent_session_path(core.agent()), "probe.png", png.constData(),
static_cast<size_t>(png.size()), aerr, sizeof(aerr));
check(stored != nullptr,
QStringLiteral("a picture can be stored next to the session"));
if (stored) {
QSize got;
const QImage shrunk =
images->requestImage(QString::fromUtf8(stored->rel),
&got, QSize(8, 8));
check(!shrunk.isNull() && shrunk.width() == 8 && shrunk.height() == 4,
QStringLiteral("the provider serves it, shrunk to what was asked "
"(%1x%2)")
.arg(shrunk.width())
.arg(shrunk.height()));
check(images->requestImage(QStringLiteral("../escape.png"), &got,
QSize(64, 64))
.isNull(),
QStringLiteral("a name outside the attachment directory is refused"));
check(images->requestImage(QStringLiteral("no-such-file.png"), &got,
QSize(64, 64))
.isNull(),
QStringLiteral("a file that is not there is refused"));
cst_attach_free(stored);
}
}
/* --- a tool that made a picture: its card shows it -------------------- */
{
const int before = list->property("count").toInt();
const int thumbsBefore =
countNamed(win->contentItem(), QStringLiteral("imageThumb"));
event4("engineToolEvent", QStringLiteral("open"), QStringLiteral("call-img"),
QStringLiteral("view_image"), QStringLiteral("{\"path\":\"shot.png\"}"));
const QString imgs =
QStringLiteral("img-fixture.png|shot.png|view_image|24x8");
check(QMetaObject::invokeMethod(win, "engineToolImages",
Q_ARG(QVariant, QStringLiteral("call-img")),
Q_ARG(QVariant, imgs)),
QStringLiteral("images a tool made reach the view"));
QTest::qWait(150);
const QVariantMap card = rowAt(before);
check(card.value(QStringLiteral("images"))
.toString()
.contains(QLatin1String("img-fixture.png")) &&
card.value(QStringLiteral("name")).toString() ==
QLatin1String("view_image"),
QStringLiteral("the card of that call carries the image"));
check(countNamed(win->contentItem(), QStringLiteral("imageThumb")) >
thumbsBefore,
QStringLiteral("its thumbnail is in the card"));
maybeShot(win, QStringLiteral("-toolimage"));
}
/* --- a narrow window: dock -> drawer, and the button opens it --- */
resizeTo(win, 460, 900);
check(docked && !docked->isVisible(),
+9 -2
View File
@@ -9,6 +9,7 @@
#include "cst_agent.h"
#include "cst_tools.h"
#include "os_compat.h"
#include <stdio.h>
#include <stdlib.h>
@@ -54,8 +55,14 @@ static int probe_missing(char *why, size_t why_sz) {
}
int main(void) {
char ws[1024];
snprintf(ws, sizeof(ws), "/tmp/cst-app-selftest-%d", (int)getpid());
/* The scratch workspace goes where this host can actually write: /tmp on a
* desktop, the app's own directory on Android (where /tmp is off limits,
* so TMPDIR is pointed at it -- see app/src/main.cpp). */
char tmp[1024] = "";
if (os_temp_dir(tmp, sizeof(tmp)) != 0 || !tmp[0])
snprintf(tmp, sizeof(tmp), "%s", "/tmp");
char ws[1200];
snprintf(ws, sizeof(ws), "%s/cst-app-selftest-%d", tmp, (int)getpid());
#ifdef _WIN32
_mkdir(ws); /* the engine wants its workspace to exist */
#else