v0.9.0: the Qt front end as an Android APK, images in the transcript

Two strands land together: the window draws pictures now, and the whole front
end builds, installs and runs on the phone with the engine linked in-process.

Images (the "make a picture actually show up" part)
- Engine, additively: cst_agent_hooks_t.on_tool_images(call, list, n, ud) fires
  when a call left attachments behind, and cst_agent_pending_rel() /
  cst_agent_pending_is_image() read that list without the app touching the
  store. tests/test_attach.c::test_agent_pending pins the three cases (an
  image, a stored name that passes cst_attach_rel_ok, out-of-range NULL).
- App: image://cstattach/<stored name> (app/src/ui/SessionImages.cpp) serves
  only the current session's attachment directory, checked with
  cst_attach_rel_ok, decodes down to the requested sourceSize and never
  upscales, and holds a mutex because requestImage() runs on the render
  thread while the directory can change underneath it. AgentCore gained
  attachmentDir(), pendingImages(), the toolImages()/promptImages() signals
  and hookToolImages(); history() gives replayed rows their images too.
- QML: ImageStrip.qml (a thumb strip, <=150 px tall, shaped by the picture,
  tooltip with name/size/origin, the file's name when it went missing),
  wired into MessageCard and ToolCard. Qt 6's ListModel cannot hold a
  list-valued role (append() reads back undefined, setProperty drops it), so
  a row carries one image per line as <stored>|<name>|<origin>|<w>x<h> and
  ImageStrip decodes that.

Android (the engine is already in the APK; this is the front end)
- app/src/main.cpp: on Android the workspace, config, tmp and HOME go under
  the app's own data directory before the engine opens anything, because
  QDir::currentPath() is "/" there and /tmp is not writable.
- app/android/: the manifest (QtApplication/QtActivity, INTERNET and
  POST_NOTIFICATIONS, adjustResize, no storage permission -- the picker will
  be the system one), strings, a vector icon; minSdk 24 and targetSdk 35 come
  from app/CMakeLists.txt through QT_ANDROID_MIN/TARGET_SDK_VERSION.
- app/src/platform/android/AndroidTools.cpp: the Android file behind
  PlatformInit(), owner "app:android" (notification, clipboard, share sheet,
  file picker are stubs for now, so the tool list does not lie about them).
- app/tools/selftest.c decides its scratch workspace with os_temp_dir(), so
  the C smoke test runs on the device too.
- scripts/build_apk_android.sh: sources to installed APK in one command
  (environment checks, qt-cmake configure, --target apk, adb install), and
  the host-side plumbing the NixOS + qt.io combination needs -- the NDK
  sysroot include dirs for Qt's FindGLESv2/FindEGL (they only look under
  <sysroot>/include), -L$PREFIX/lib on the shared/module link lines as well
  as the executables', the Android debug keystore created on first use with
  QT_ANDROID_SIGN_APK + QT_ANDROID_KEYSTORE_* (androiddeployqt copies the
  unsigned gradle output and signs it itself), and the #!/bin/bash shebang of
  the build-tools wrappers, which NixOS does not have.
- scripts/gradle-android-init.gradle (installed into ~/.gradle/init.d/): the
  release lint task is switched off -- lintVitalAnalyzeRelease resolves its
  own tool jars straight from dl.google.com, bypassing every repository the
  build configures, and fails the build after all the real work is done.
- Icons: app/icons/*.svg (Breeze's 24@3x files, the 19 names the window asks
  for) are compiled in as :/cst-icons and ThemeIcons.cpp falls back to them
  when the platform has no icon theme. On the phone the Qt theme lookup found
  nothing and the QML glyph fallback was an empty box in the phone's fonts.
- docs/android.md: "The APK (Qt front end)" -- the SDK assembled without
  sdkmanager, the host Qt's missing libraries on NixOS, the Gradle mirrors,
  lint, signing -- and a Status that says what runs where. app/README.md
  gains the images and APK lines, and stops claiming Android platform tools
  that are still stubs.

Verified: engine `make check` green; desktop `cst-app-selftest: OK` and
--uitest all checks pass; phone (NX789J, Android 16, arm64-v8a) -- the APK
installs, starts on the workspace under /data/user/0/xyz.whcself.cstagent/
files/, a turn reaches a configured provider over HTTPS from inside the ART
process and renders the reply, rotation keeps the session and docks the
sidebar, and the toolbar/card icons draw.

Version 0.9.0.

mode = coding
This commit is contained in:
WHC
2026-10-11 15:22:10 +08:00
parent 82dee65942
commit c2320f946b
46 changed files with 1370 additions and 26 deletions
+8
View File
@@ -39,5 +39,13 @@ vendor/sashidown/libsashidown-mingw.a
.gen/
.build-tmp/
# Android APK builds (scripts/build_apk_android.sh) and what gradle leaves
build-apk-android/
*.apk
*.aab
app/android/.gradle/
app/android/build/
app/android/local.properties
# the engine as a static library (app/ links it; built by `make`)
libcstcore.a
+78 -2
View File
@@ -76,13 +76,23 @@ if(CST_APP_GUI)
# main.cpp loads) and let CMake generate the qmldir of qml/ for us.
qt_policy(SET QTP0001 NEW)
qt_policy(SET QTP0004 NEW)
# The front end's platform tools: one file per platform, the same split
# core/ToolProvider.cpp makes. Android brings its own (JNI, owner
# "app:android"); the other platforms still use the Linux file, whose
# registration entry point is simply not called there.
if(ANDROID)
set(CST_APP_PLATFORM_SRC src/platform/android/AndroidTools.cpp)
else()
set(CST_APP_PLATFORM_SRC src/platform/linux/NotifyTool.cpp)
endif()
qt_add_executable(cst-agent-app
src/main.cpp
src/core/AgentCore.cpp
src/core/ToolProvider.cpp
src/ui/UiTest.cpp
src/ui/ThemeIcons.cpp
src/platform/linux/NotifyTool.cpp)
src/ui/SessionImages.cpp
${CST_APP_PLATFORM_SRC})
target_include_directories(cst-agent-app PRIVATE
src "${CMAKE_CURRENT_SOURCE_DIR}/../include")
target_compile_definitions(cst-agent-app PRIVATE CST_APP_VERSION="${PROJECT_VERSION}")
@@ -96,6 +106,52 @@ if(CST_APP_GUI)
set_target_properties(cst-agent-app PROPERTIES WIN32_EXECUTABLE TRUE)
endif()
# --- Android (APK) -----------------------------------------------------
# The engine is compiled in (top-level CMakeLists: -DCST_BUILD_TUI=OFF,
# -DCST_BUILD_WEBUI=OFF, so src/webui_stub.c stands in for the server and
# nothing listens on a port). The manifest, icons and strings live in
# app/android/, which androiddeployqt copies next to its own gradle
# project. minSdk 24 is the engine's own API_LEVEL (scripts/
# build_third_party_android.sh builds curl/mbedTLS for android-24).
if(ANDROID)
file(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/../include/cst_version.h"
_cst_ver_lines REGEX "#define CST_VERSION_(MAJOR|MINOR|PATCH) ")
foreach(_line ${_cst_ver_lines})
if(_line MATCHES "CST_VERSION_MAJOR ([0-9]+)")
set(_cst_vmaj "${CMAKE_MATCH_1}")
elseif(_line MATCHES "CST_VERSION_MINOR ([0-9]+)")
set(_cst_vmin "${CMAKE_MATCH_1}")
elseif(_line MATCHES "CST_VERSION_PATCH ([0-9]+)")
set(_cst_vpat "${CMAKE_MATCH_1}")
endif()
endforeach()
set_target_properties(cst-agent-app PROPERTIES
QT_ANDROID_PACKAGE_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/android"
QT_ANDROID_MIN_SDK_VERSION 24
QT_ANDROID_TARGET_SDK_VERSION 35
QT_ANDROID_VERSION_NAME "${_cst_vmaj}.${_cst_vmin}.${_cst_vpat}"
QT_ANDROID_VERSION_CODE 1
QT_ANDROID_ABIS "${ANDROID_ABI}")
message(STATUS "app: Android package "
"${_cst_vmaj}.${_cst_vmin}.${_cst_vpat}, ABI ${ANDROID_ABI}, "
"minSdk 24, targetSdk 35")
# Signing: androiddeployqt signs the APK itself and refuses a signed
# gradle output ("Could not copy unsigned APK"), so the keystore goes
# here, not into gradle. The path, alias and passwords arrive through
# the QT_ANDROID_KEYSTORE_* environment variables;
# scripts/build_apk_android.sh points them at the standard Android debug
# key it creates on first use, and CST_ANDROID_KEYSTORE overrides it.
if(DEFINED ENV{QT_ANDROID_KEYSTORE_PATH} AND NOT "$ENV{QT_ANDROID_KEYSTORE_PATH}" STREQUAL "")
set(QT_ANDROID_SIGN_APK TRUE)
message(STATUS "app: signing the APK with "
"$ENV{QT_ANDROID_KEYSTORE_PATH} ($ENV{QT_ANDROID_KEYSTORE_ALIAS})")
else()
message(STATUS "app: QT_ANDROID_KEYSTORE_PATH unset -- the APK stays "
"unsigned and 'adb install' will refuse it (INSTALL_PARSE_FAILED_NO_CERTIFICATES)")
endif()
endif()
# Theme is read by every component, so it is a QML singleton (one instance,
# no object needed in the tree).
set_source_files_properties(qml/Theme.qml PROPERTIES QT_QML_SINGLETON_TYPE TRUE)
@@ -105,7 +161,7 @@ if(CST_APP_GUI)
# requires absolute URLs" on every start).
foreach(_qml Main ChatView Sidebar SessionRow MessageCard ToolCard ThinkBlock
NoteLine TodoPanel Composer ConsentDialog AskDialog SettingsDialog
ThemeIcon Theme)
ThemeIcon Theme ImageStrip)
set_source_files_properties(qml/${_qml}.qml PROPERTIES QT_RESOURCE_ALIAS ${_qml}.qml)
endforeach()
qt_add_qml_module(cst-agent-app
@@ -126,6 +182,26 @@ if(CST_APP_GUI)
qml/AskDialog.qml
qml/SettingsDialog.qml
qml/ThemeIcon.qml
qml/ImageStrip.qml
qml/Theme.qml)
# The icons the window asks for, compiled in (app/icons, Breeze's scalable
# names copied at 24@3x). The desktop finds them in the platform's icon
# theme and only falls back to these (ThemeIcons.cpp); a machine without
# any theme -- Android -- has nothing else, and before this the QML glyph
# fallback was all that showed, which the phone's fonts cannot render.
set(CST_APP_ICONS
go-down mail-attachment application-menu settings-configure edit-copy
edit-delete arrow-down arrow-right list-add task-complete box
document-edit text-x-generic folder-open edit-find code-context
internet-services format-list-unordered help-hint)
set(_cst_icon_files "")
foreach(_icon ${CST_APP_ICONS})
list(APPEND _cst_icon_files "${CMAKE_CURRENT_SOURCE_DIR}/icons/${_icon}.svg")
endforeach()
qt_add_resources(cst-agent-app "cst_icons"
PREFIX "/cst-icons"
BASE "${CMAKE_CURRENT_SOURCE_DIR}/icons"
FILES ${_cst_icon_files})
endif()
endif()
+28 -3
View File
@@ -212,6 +212,27 @@ is refused with a reason, the composer shows one chip, "clear all" empties the
list, and — with an 8×8 image put on the clipboard — `clipboardHasImage()` and
`attachClipboard()` behave.
## Images
What a session keeps as a picture is shown, not only stored: a file
`view_image` returned, an image an MCP server sent, a screenshot the user
attached or pasted. Each one appears as a thumbnail in the transcript -- a
prompt's pictures under the message, a tool's under its card -- with the name,
size and origin in the tooltip. The height is capped so a 4K screenshot cannot
push the conversation off the screen, and a file the store no longer has says
which image it was instead of leaving a hole.
The view never touches the filesystem on its own authority: the thumbnails
come from `image://cstattach/<stored name>` (`app/src/ui/SessionImages.cpp`),
where the name is the one the engine wrote next to the session and is checked
with the engine's own `cst_attach_rel_ok()`, so only the session's attachment
directory can be read. The picture is decoded at the size of its thumbnail
rather than at its full resolution, and the provider's directory follows the
session: `AgentCore::attachmentDir()` is re-read on every switch.
A document (PDF, UTF-8 text) still arrives as its name in the message text --
those are sent to the model as text, not as a picture.
## Transcript, tool calls, streaming text
- A tool call is a card of its own, and the call **id** is what pairs it with
@@ -307,7 +328,7 @@ GUI only through queued signals.
|---|---|---|
| Linux | `cmake -S app -B build-app` (libcurl via pkg-config) | clipboard (shell helpers), notification (`notify-send` / D-Bus) |
| Windows | Qt's MinGW kit, `cmake -G Ninja -DCMAKE_PREFIX_PATH=<Qt>/lib/cmake`; the engine needs libcurl for Windows (see `docs/README-windows.md`) | notification via the WinRT/tray API, clipboard through the core's Win32 path |
| Android | engine compiled with the NDK inside the APK (`-DCST_TUI=0`, `CST_NO_WEBUI=1`); the app talks to Android APIs through Qt (JNI) | notification (Android), share sheet, clipboard, file picker |
| Android | engine compiled with the NDK inside the APK (`scripts/build_apk_android.sh`, `-DCST_TUI=0`, `CST_NO_WEBUI=1`); no port, QML talks to `AgentCore` in-process | none yet: `app/src/platform/android/AndroidTools.cpp` is a stub, so notification, share sheet, clipboard and the file picker are still missing |
| macOS | Qt from the installer, libcurl from the system | notification (`osascript`), clipboard (`pbpaste`/`pbcopy`) |
Android's long-term shape: the core lives in the APK (no Termux needed), the
@@ -328,8 +349,12 @@ The build flag that keeps Unix helpers out of the picture is
- [x] attachments: picker, drop on the window, Ctrl+V image paste
- [x] icons from the desktop's icon theme (Breeze), with a glyph fallback
- [x] contrast pass: faint text and selected rows are readable
- [x] images in the transcript: tool results and attachments are drawn as
thumbnails (`image://cstattach/<stored name>`, see the Images section)
- [x] Android APK: the engine is linked in (no port), builds, installs under
`xyz.whcself.cstagent` and runs on the device (see `docs/android.md`)
- [ ] a real turn driven through the window (needs a provider key)
- [ ] tool results that carry images (they are stored, not drawn yet)
- [ ] Android platform tools (JNI: notification, share sheet, clipboard, file
picker), the back gesture, the icon glyphs on the phone
- [ ] Kirigami (the rest of the Breeze look: colors, controls), tray,
global shortcuts
- [ ] Android APK (engine in-process, JNI platform tools)
+73
View File
@@ -0,0 +1,73 @@
<?xml version="1.0"?>
<!-- The APK's manifest (QT_ANDROID_PACKAGE_SOURCE_DIR in app/CMakeLists.txt):
androiddeployqt copies this file and res/ into its gradle project and
fills in the INSERT_* placeholders below (version code and name, library
name, app arguments). An XML comment may not contain a double hyphen, so
the placeholder syntax is only spelled out in the attributes themselves.
The SDK levels are not here on purpose: they come from
QT_ANDROID_MIN_SDK_VERSION and QT_ANDROID_TARGET_SDK_VERSION in
app/CMakeLists.txt, and androiddeployqt writes them into the generated
gradle.properties. The app name and icon come from res/ next to this
file (values/strings.xml, drawable/ic_launcher.xml).
WHAT THIS APP IS: the engine runs in this process (no server, no port) and
writes everything (workspace, session store, settings, credentials) under
the app's own data directory, which main.cpp points the engine at (see
androidRoot() there).
Permissions: INTERNET for the providers the user configures,
POST_NOTIFICATIONS for the app's notification_send tool (Android 13+
asks at runtime; the tool stays hidden until it is granted). No storage
permission: the file picker is the system one (SAF) and its input is
copied into the app's directory. No cleartext traffic: every provider URL
is https. -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="xyz.whcself.cstagent"
android:installLocation="auto"
android:versionCode="-- %%INSERT_VERSION_CODE%% --"
android:versionName="-- %%INSERT_VERSION_NAME%% --">
<!-- %%INSERT_PERMISSIONS -->
<!-- %%INSERT_FEATURES -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<supports-screens
android:anyDensity="true"
android:largeScreens="true"
android:normalScreens="true"
android:smallScreens="true" />
<application
android:name="org.qtproject.qt.android.bindings.QtApplication"
android:label="@string/app_name"
android:icon="@drawable/ic_launcher"
android:allowBackup="true"
android:hardwareAccelerated="true"
android:enableOnBackInvokedCallback="false">
<activity
android:name="org.qtproject.qt.android.bindings.QtActivity"
android:label="@string/app_name"
android:configChanges="orientation|uiMode|screenLayout|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|keyboard|keyboardHidden|navigation|density|mcc|mnc"
android:launchMode="singleTop"
android:screenOrientation="unspecified"
android:exported="true"
android:windowSoftInputMode="adjustResize">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<meta-data
android:name="android.app.lib_name"
android:value="-- %%INSERT_APP_LIB_NAME%% --" />
<meta-data
android:name="android.app.arguments"
android:value="-- %%INSERT_APP_ARGUMENTS%% --" />
</activity>
</application>
</manifest>
+30
View File
@@ -0,0 +1,30 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Launcher icon: the terminal prompt the app is about ("&gt;_"), the window's
own colors (bg #12161a, accent #5cc0f0, faint #8d959c). A vector drawable
works from API 24 up; a proper adaptive icon (mipmap-anydpi-v26 + a PNG
fallback) is polish for later. -->
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path
android:fillColor="#12161a"
android:pathData="M0,0h108v108h-108z" />
<path
android:fillColor="#00000000"
android:strokeColor="#5cc0f0"
android:strokeWidth="7"
android:strokeLineCap="round"
android:strokeLineJoin="round"
android:pathData="M32,38 L52,54 L32,70" />
<path
android:fillColor="#00000000"
android:strokeColor="#8d959c"
android:strokeWidth="7"
android:strokeLineCap="round"
android:pathData="M58,74 L78,74" />
</vector>
+5
View File
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- The name under the launcher icon. -->
<resources>
<string name="app_name">CST-agent</string>
</resources>
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3 5v2h16v-2h-16m0 5v2h16v-2h-16m0 5v2h16v-2h-16" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 446 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3.707031 7l-.707031.707031 6.125 6.125 1.875 1.875 1.875-1.875 6.125-6.125-.707031-.707031-6.125 6.125-1.167969 1.167969-1.167969-1.167969-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 549 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m7.707031 3l-.707031.707031 6.125 6.125 1.167969 1.167969-1.167969 1.167969-6.125 6.125.707031.707031 6.125-6.125 1.875-1.875-1.875-1.875-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 547 B

+10
View File
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 24 24" width="24" height="24">
<defs>
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" transform="translate(-.99999-.99999)" d="m4 4v16h16v-16h-16m1 1h14v14h-14v-14" fill="currentColor"/>
<path d="m3.5056345 10.977883v-7.5052069h14.878582v15.010415h-14.878582z" class="ColorScheme-Text" fill="currentColor" fill-opacity=".50196078"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 589 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m10.04492 3c-1.77683 0-2.521484.706446-2.521484 2.509766v2.441406c0 1.11382-.034195 2.404297-1.015625 2.404297h-.507813v1.289062h.507813c.98148 0 1.015625 1.290477 1.015625 2.404297v2.441406c0 1.80332.744654 2.509766 2.521484 2.509766v-1.183594c-1.18455 0-1.201172-.478327-1.201172-2.210937v-2.01563c0-.83093-.204705-2.067378-1.203125-2.580078.99842-.51271 1.203125-1.751092 1.203125-2.582032v-2.033203c0-1.7326.016592-2.210937 1.201172-2.210937v-1.183594m1.894531 0v1.183594c1.18456 0 1.216797.478337 1.216797 2.210937v2.033203c0 .83094.204705 2.069322 1.203125 2.582032-.99842.5127-1.203125 1.749148-1.203125 2.580078v2.01563c0 1.73261-.032187 2.210937-1.216797 2.210937v1.183594c1.77684 0 2.537109-.706446 2.537109-2.509766v-2.441406c0-1.11382.068644-2.404297 1.033204-2.404297h.490234v-1.289062h-.490234c-.96457 0-1.033204-1.290477-1.033204-2.404297v-2.441406c0-1.80332-.760269-2.509766-2.537109-2.509766" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m14.996094 3l-11.992188 11.992188h-.003906v4.00781h1 2 1.00781v-.003906l11.992188-11.992188-.001953-.001953.001953-.001953-4-4-.001953.001953-.001953-.001953m-1.998047 3.412109l2.589844 2.589844-7.587891 7.587891v-1.589844h-1-1v-1-.589844l6.998047-6.998047m-7.998047 7.998047v1.589844h1 1v1 .589844l-.410156.410156h-1.589844l-1-1v-1.589844l1-1" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 741 B

+10
View File
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" version="1.1" width="24" height="24">
<defs>
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" style="fill:currentColor; fill-opacity:1; stroke:none" d="M 3 3 L 3 17 L 7 17 L 7 19 L 17 19 L 17 10 L 13 6 L 12 6 L 9 3 L 3 3 Z M 4 4 L 8 4 L 8 6 L 7 6 L 7 16 L 4 16 L 4 4 Z M 8 7 L 12 7 L 12 11 L 16 11 L 16 18 L 8 18 L 8 7 Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 583 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-NegativeText { color: #da4453; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" class="ColorScheme-NegativeText" d="M8 3v2h1V4h4v1h1V3H8M4 6v1h14V6H4m2 2v11h10V8h-1v10H7V8H6"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 471 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 9 3 C 5.6759952 3 3 5.6759952 3 9 C 3 12.324005 5.6759952 15 9 15 C 10.481205 15 11.830584 14.465318 12.875 13.582031 L 18.292969 19 L 19 18.292969 L 13.582031 12.875 C 14.465318 11.830584 15 10.481205 15 9 C 15 5.6759952 12.324005 3 9 3 z M 9 4 C 11.770005 4 14 6.2299952 14 9 C 14 11.770005 11.770005 14 9 14 C 6.2299952 14 4 11.770005 4 9 C 4 6.2299952 6.2299952 4 9 4 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 774 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor" d="M 3 3 L 3 4 L 3 19 L 4 19 L 19 19 L 19 18 L 19 5 L 12. 5 L 10. 3 L 10 3 L 10 3 L 4 3 L 3 3 z M 9 8 L 18 8 L 18 18 L 4 18 L 4 10 L 7 10 L 7 10 L 7 10 L 9 8 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 528 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 4 3 C 3.4477153 3 3 3.4477153 3 4 C 3 4.5522847 3.4477153 5 4 5 C 4.5522847 5 5 4.5522847 5 4 C 5 3.4477153 4.5522847 3 4 3 z M 10 3 L 10 4 L 19 4 L 19 3 L 10 3 z M 10 5 L 10 6 L 16 6 L 16 5 L 10 5 z M 4 9 C 3.4477153 9 3 9.4477153 3 10 C 3 10.552285 3.4477153 11 4 11 C 4.5522847 11 5 10.552285 5 10 C 5 9.4477153 4.5522847 9 4 9 z M 10 9 L 10 10 L 19 10 L 19 9 L 10 9 z M 10 11 L 10 12 L 12 12 L 12 11 L 10 11 z M 4 15 C 3.4477153 15 3 15.447715 3 16 C 3 16.552285 3.4477153 17 4 17 C 4.5522847 17 5 16.552285 5 16 C 5 15.447715 4.5522847 15 4 15 z M 10 15 L 10 16 L 19 16 L 19 15 L 10 15 z M 10 17 L 10 18 L 16 18 L 16 17 L 10 17 z M 17 18 L 17 19 L 19 19 L 19 18 L 17 18 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.1 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="m3.707031 7l-.707031.707031 6.125 6.125 1.875 1.875 1.875-1.875 6.125-6.125-.707031-.707031-6.125 6.125-1.167969 1.167969-1.167969-1.167969-6.125-6.125" class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 549 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 11 3 C 6.568 3 3 6.568 3 11 C 3 15.432 6.568 19 11 19 C 15.432 19 19 15.432 19 11 C 19 6.568 15.432 3 11 3 z M 11 4 C 14.878 4 18 7.122 18 11 C 18 14.878 14.878 18 11 18 C 7.122 18 4 14.878 4 11 C 4 7.122 7.122 4 11 4 z M 10 6 L 10 8 L 12 8 L 12 6 L 10 6 z M 10 9 L 10 16 L 12 16 L 12 9 L 10 9 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 696 B

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 24 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 10 4 L 10 11 L 3 11 L 3 12 L 10 12 L 10 19 L 11 19 L 11 12 L 18 12 L 18 11 L 11 11 L 11 4 L 10 4 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 499 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 12.902344 3 L 3.734375 12.167969 C 2.167415 13.734929 2.167415 16.257279 3.734375 17.824219 C 5.301315 19.391169 7.823675 19.391169 9.390625 17.824219 L 17.851562 9.3632812 L 18.558594 8.65625 C 19.733804 7.48104 19.733794 5.5892625 18.558594 4.4140625 C 17.383384 3.2388525 15.491616 3.2388525 14.316406 4.4140625 L 13.609375 5.1210938 L 5.1484375 13.582031 C 4.3649575 14.365511 4.3649675 15.626686 5.1484375 16.410156 C 5.9319175 17.193636 7.1930825 17.193646 7.9765625 16.410156 L 16.4375 7.9492188 L 15.730469 7.2421875 L 7.9765625 14.996094 L 7.2695312 15.703125 C 6.8777912 16.094865 6.2471888 16.094865 5.8554688 15.703125 C 5.4637287 15.311405 5.4637187 14.680813 5.8554688 14.289062 L 14.316406 5.828125 L 15.023438 5.1210938 C 15.806917 4.3376037 17.068092 4.3376037 17.851562 5.1210938 C 18.635042 5.9045538 18.635042 7.1657387 17.851562 7.9492188 L 8.6835938 17.117188 C 7.5083838 18.292397 5.6166162 18.292397 4.4414062 17.117188 C 3.2661862 15.941977 3.2661862 14.05021 4.4414062 12.875 L 13.609375 3.7070312 L 12.902344 3 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style type="text/css" id="current-color-scheme">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path style="fill:currentColor;fill-opacity:1;stroke:none" d="M 11.5 3 C 10.286139 3 9.2809778 3.8559279 9.0507812 5 L 3 5 L 3 6 L 9.0507812 6 C 9.2809778 7.1440721 10.286139 8 11.5 8 C 12.713861 8 13.719022 7.1440721 13.949219 6 L 19 6 L 19 5 L 13.949219 5 C 13.719022 3.8559279 12.713861 3 11.5 3 z M 5.5 14 C 4.1149999 14 3 15.115 3 16.5 C 3 17.885 4.1149999 19 5.5 19 C 6.7138604 19 7.7190223 18.144072 7.9492188 17 L 19 17 L 19 16 L 7.9492188 16 C 7.7190223 14.855928 6.7138604 14 5.5 14 z M 5.5 15 C 6.3310001 15 7 15.669 7 16.5 C 7 17.331 6.3310001 18 5.5 18 C 4.6689999 18 4 17.331 4 16.5 C 4 15.669 4.6689999 15 5.5 15 z " class="ColorScheme-Text"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 966 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24">
<defs id="defs3051">
<style id="current-color-scheme" type="text/css">.ColorScheme-Text { color: #fcfcfc; } </style>
</defs>
<g transform="translate(1,1)">
<path class="ColorScheme-Text" d="M 19,9 12,16 8,12 9.4896719,10.543135 12,13 17.482382,7.5492102 Z M 4,3 V 5 H 3 V 6 H 4 V 7 H 3 v 1 h 1 v 6 H 3 v 1 h 1 v 1 H 3 v 1 h 1 v 2 h 3 12 v -1 -7 h -1 v 1 1 2 1 2 H 10 9 7 V 16 15 13 12 10 9 7 6 4 h 2 1 8 v 2 h 1 V 3 H 7 Z m 4,3 v 2 h 2 V 6 Z m 3,0 v 1 h 6 V 6 Z" style="fill:currentColor;fill-opacity:1;stroke:none"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 669 B

+14
View File
@@ -0,0 +1,14 @@
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg xmlns="http://www.w3.org/2000/svg" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" width="24" version="1.1" height="24" viewBox="0 0 24 24">
<defs id="defs3871"/>
<metadata id="metadata3874"/>
<g transform="translate(1,1)">
<g inkscape:label="Capa 1" inkscape:groupmode="layer" id="layer1" transform="matrix(1 0 0 1 -326 -534.3622)">
<rect width="8" x="329" y="537.36218" height="1.00002" style="fill:#6c7a89" id="rect4279"/>
<rect width="16" x="329" y="541.36218" height="1" style="fill:#6c7a89" id="rect4281"/>
<rect width="16" x="329" y="548.36218" height="1" style="fill:#6c7a89" id="rect4283"/>
<rect width="8" x="329" y="545.36218" height="1" style="fill:#6c7a89" id="rect4285"/>
<rect width="12" x="329" y="552.36218" height="0.999983" style="fill:#6c7a89" id="rect4287"/>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 919 B

+2
View File
@@ -40,6 +40,7 @@ Item {
role: model.role
text: model.text
complete: model.complete
images: model.images !== undefined ? model.images : ""
/* html comes from the streaming renderer (Main.qml) or,
* for a replayed session, from here once. */
html: model.html.length > 0 ? model.html
@@ -65,6 +66,7 @@ Item {
output: model.output
exitCode: model.exitCode
hasResult: model.complete
images: model.images !== undefined ? model.images : ""
}
}
Component {
+110
View File
@@ -0,0 +1,110 @@
import QtQuick
import QtQuick.Controls
import QtQuick.Layouts
// The images a message or a tool call carries: thumbnails from the session's
// own store (image://cstattach/, app/src/ui/SessionImages.cpp). The names are
// the ones the engine wrote next to the session -- view_image, an MCP image, a
// picture the user attached -- never a path the view invented.
//
// One row of thumbnails: the height is capped so a screenshot cannot push the
// conversation off the screen, the width follows the picture's shape.
Flow {
id: strip
objectName: "imageStrip"
/* One line per picture, written by the engine side (AgentCore.cpp:
* "<stored name>|<name>|<origin>|<w>x<h>"). A row carries this as text
* because a ListModel role cannot hold a list. */
property string text: ""
property var images: decode(text)
spacing: 6
function decode(t) {
const out = []
const lines = String(t).split("\n")
for (let i = 0; i < lines.length; i++) {
if (!lines[i].length)
continue
const f = lines[i].split("|")
if (!f[0].length)
continue
let w = 0, h = 0
const dim = String(f[3] || "").split("x")
if (dim.length === 2) {
w = parseInt(dim[0], 10) || 0
h = parseInt(dim[1], 10) || 0
}
out.push({ url: "image://cstattach/" + f[0], rel: f[0],
name: f[1] || f[0], origin: f[2] || "", w: w, h: h })
}
return out
}
Repeater {
model: strip.images
delegate: Rectangle {
id: thumb
required property var modelData
required property int index
readonly property real ratio: {
const w = Number(modelData.w), h = Number(modelData.h)
return (w > 0 && h > 0) ? w / h : 1.5
}
readonly property int cap: 150 // the tallest a thumbnail gets
objectName: "imageThumb" + index
width: Math.max(56, Math.min(Math.round(cap * ratio), 320))
height: Math.max(40, Math.min(cap, Math.round(width / ratio)))
radius: Theme.radiusSm
color: Theme.bg
border.width: 1
border.color: mouse.containsMouse ? Theme.accentDim : Theme.border
clip: true
Image {
id: pic
anchors.fill: parent
anchors.margins: 2
source: String(thumb.modelData.url)
sourceSize.width: 512
sourceSize.height: 512
fillMode: Image.PreserveAspectFit
/* Not asynchronous: the provider already hands back a
* thumbnail-sized picture, so there is nothing to wait for. */
asynchronous: false
visible: status === Image.Ready
}
// A file the store no longer has is not worth shouting about: the
// frame says which image it was.
Text {
anchors.centerIn: parent
width: parent.width - 8
visible: pic.status === Image.Error
text: String(thumb.modelData.name)
elide: Text.ElideMiddle
horizontalAlignment: Text.AlignHCenter
color: Theme.textFaint
font.pixelSize: Theme.smallSize
}
MouseArea {
id: mouse
anchors.fill: parent
hoverEnabled: true
acceptedButtons: Qt.NoButton
}
ToolTip.visible: mouse.containsMouse
ToolTip.delay: 400
ToolTip.text: {
const m = thumb.modelData
let s = String(m.name)
if (m.w > 0 && m.h > 0)
s += " · " + m.w + "×" + m.h
if (m.origin && m.origin !== "user")
s += " · " + m.origin
return s
}
}
}
}
+33 -2
View File
@@ -61,7 +61,7 @@ ApplicationWindow {
function addRow(kind, role, state) {
transcript.append({ kind: kind, role: role, text: "", html: "", complete: false,
name: "", callId: "", state: state, detail: "", output: "",
exitCode: 0 })
exitCode: 0, images: "" })
}
// The engine streams one kind at a time: close whatever was open before.
@@ -112,7 +112,7 @@ ApplicationWindow {
function submit(text) {
transcript.append({ kind: "message", role: "user", text: text, html: "", complete: true,
name: "", callId: "", state: "", detail: "", output: "",
exitCode: 0 })
exitCode: 0, images: "" })
agent.send(text)
}
@@ -545,6 +545,29 @@ ApplicationWindow {
win.statusText = ""
}
/* The images the prompt just carried (AgentCore::promptImages): the
* composer has appended the user row, so they attach to it. */
function enginePromptImages(images) {
if (!images.length)
return
for (let i = transcript.count - 1; i >= 0; i--) {
const r = transcript.get(i)
if (r.kind === "message" && r.role === "user") {
transcript.setProperty(i, "images", images)
return
}
}
}
/* Images a tool produced, on the card of the call that made them. */
function engineToolImages(callId, images) {
if (!images.length)
return
const i = win.findTool(callId, "")
if (i >= 0)
transcript.setProperty(i, "images", images)
}
/* For --uitest: one transcript row, or null. */
function rowAt(i) {
return (i >= 0 && i < transcript.count) ? transcript.get(i) : null
@@ -584,6 +607,14 @@ ApplicationWindow {
win.engineToolResult(id, name, exitCode, output)
}
function onToolImages(callId, images) {
win.engineToolImages(callId, images)
}
function onPromptImages(images) {
win.enginePromptImages(images)
}
function onPermissionRequested(tool, detail, preview) {
consent.tool = tool
consent.detail = detail
+8
View File
@@ -11,6 +11,7 @@ Rectangle {
property string text: ""
property string html: ""
property bool complete: true
property string images: ""
radius: Theme.radius
color: role === "user" ? Theme.surfaceAlt : Theme.surface
@@ -105,5 +106,12 @@ Rectangle {
wrapMode: Text.Wrap
onLinkActivated: function (link) { Qt.openUrlExternally(link) }
}
// What the prompt carried: pictures from the store, not their names.
ImageStrip {
Layout.fillWidth: true
visible: card.images.length > 0
text: card.images
}
}
}
+9
View File
@@ -14,6 +14,7 @@ Rectangle {
property int exitCode: 0
property bool hasResult: false
property bool expanded: false
property string images: ""
readonly property bool running: state === "pending" || state === "ready" || state === "running"
readonly property color stateColor: state === "denied" ? Theme.err
@@ -125,6 +126,14 @@ Rectangle {
}
}
// What the tool made (view_image, an MCP image): shown as it is, with
// the output left folded -- the picture is the result.
ImageStrip {
Layout.fillWidth: true
visible: card.images.length > 0
text: card.images
}
// Folded view: the first line of the result, so a done card still says
// something without being opened.
Text {
+94 -2
View File
@@ -17,6 +17,7 @@
#include <cstdlib>
#include "cst_agent.h"
#include "cst_attach.h"
#include "cst_models.h"
#include "cst_provider_switch.h"
#include "cst_web_md.h"
@@ -53,6 +54,7 @@ AgentCore::AgentCore(QObject *parent) : QObject(parent) {
m_hooks.on_think_delta = hookThink;
m_hooks.on_tool_event = hookToolEvent;
m_hooks.on_tool_result = hookToolResult;
m_hooks.on_tool_images = hookToolImages;
m_hooks.on_todo = hookTodo;
m_hooks.on_permission = hookPermission;
m_hooks.on_ask_user = hookAskUser;
@@ -172,7 +174,12 @@ void AgentCore::close() {
void AgentCore::send(const QString &prompt) {
if (!m_agent || m_running || prompt.trimmed().isEmpty())
return;
/* The prompt takes the pending images with it: the view shows them on the
* message the composer just appended, before the turn can say anything. */
const QString sending = pendingImages();
setRunning(true);
if (!sending.isEmpty())
emit promptImages(sending);
m_turn = new Turn(this, prompt);
m_turn->start();
}
@@ -314,9 +321,41 @@ QVariantMap sessionRow(const QString &kind, const QString &role) {
r["detail"] = QString();
r["output"] = QString();
r["exitCode"] = 0;
r["images"] = QString();
return r;
}
/* One field of one picture line: names come from the user and from tool
* output, so a separator may not split a line in two. */
QString field(const char *s) {
QString out = utf8(s);
out.replace(QLatin1Char('|'), QLatin1Char('/'));
out.replace(QLatin1Char('\n'), QLatin1Char(' '));
return out;
}
/* The images of one message or tool result, one line per picture:
*
* <stored name>|<name>|<origin>|<w>x<h>
*
* A list role is not an option: a ListModel keeps what fits in a QVariant of
* a simple type -- a list reads back as undefined there, and setProperty drops
* the role -- so the rows carry this text and ImageStrip.qml decodes it. A
* document (PDF, text) is not a picture and stays out of it. */
QString imagesText(const cst_attachment_t *list, int n) {
QStringList lines;
for (int i = 0; list && i < n; i++) {
const cst_attachment_t *a = &list[i];
if (!cst_attach_is_image(a))
continue;
lines << QStringLiteral("%1|%2|%3|%4x%5")
.arg(field(a->rel), field(a->name), field(a->origin))
.arg(a->width)
.arg(a->height);
}
return lines.join(QLatin1Char('\n'));
}
/* The open tool row this result belongs to, -1 when none. Rows are paired by
* call id first: the same tool may run twice in a turn, and pairing by name
* would hang the first card on "running" forever. Only a call without an id
@@ -354,6 +393,20 @@ QString AgentCore::sessionId() const {
return utf8(id);
}
/* Where the session keeps its images: what image://cstattach/ serves, and
* what the view asks for when the session changes. */
QString AgentCore::attachmentDir() const {
if (!m_agent)
return QString();
const char *sess = cst_agent_session_path(m_agent);
if (!sess || !sess[0])
return QString();
char dir[4096] = "";
if (cst_attach_dir(sess, dir, sizeof(dir)) != 0)
return QString();
return utf8(dir);
}
QVariantList AgentCore::sessions() const {
QVariantList out;
if (m_workspace.isEmpty())
@@ -404,12 +457,14 @@ QVariantList AgentCore::history() const {
r["name"] = name;
r["callId"] = callId;
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows.append(r);
} else {
QVariantMap r = rows.at(at).toMap();
r["output"] = utf8(m->content);
r["images"] = imagesText(m->attach, m->nattach);
r["state"] = "done";
r["complete"] = true;
rows[at] = r;
@@ -426,13 +481,16 @@ QVariantList AgentCore::history() const {
if (m->role == CST_ROLE_USER) {
QVariantMap r = sessionRow(QStringLiteral("message"), QStringLiteral("user"));
QString text = utf8(m->content);
if (m->nattach > 0) {
/* A document keeps its name in the text; a picture is shown instead
* (the images below), so only the rest is worth naming. */
QStringList names;
for (int a = 0; a < m->nattach; a++)
if (!cst_attach_is_image(&m->attach[a]))
names << utf8(m->attach[a].name);
if (!names.isEmpty())
text += QStringLiteral("\n\n[attached: %1]").arg(names.join(", "));
}
r["text"] = text;
r["images"] = imagesText(m->attach, m->nattach);
r["complete"] = true;
rows.append(r);
continue;
@@ -802,6 +860,30 @@ QVariantMap AgentCore::usage() const {
/* --- attachments ---------------------------------------------------------- */
/* What the prompt about to be sent carries, ready for the view. The stored
* name is the engine's (the file already lives next to the session), the
* display name is the one the window kept when it copied the file in. */
QString AgentCore::pendingImages() const {
QStringList lines;
if (!m_agent)
return QString();
const int n = cst_agent_pending_count(m_agent);
for (int i = 0; i < n; i++) {
if (!cst_agent_pending_is_image(m_agent, i))
continue;
const char *rel = cst_agent_pending_rel(m_agent, i);
if (!rel || !rel[0])
continue;
const QString name =
i < m_attachNames.size() ? m_attachNames.at(i) : utf8(rel);
/* The size is not known before the engine stores the file; the strip
* falls back to a picture's usual shape until it is. */
lines << QStringLiteral("%1|%2|user|0x0")
.arg(field(rel), field(name.toUtf8().constData()));
}
return lines.join(QLatin1Char('\n'));
}
QStringList AgentCore::pendingAttachments() const {
QStringList out;
if (!m_agent)
@@ -937,6 +1019,16 @@ void AgentCore::hookToolResult(const ctl_tool_call_t *call, const char *out,
[self, id, n, exit_code, o] { emit self->toolResult(id, n, exit_code, o); });
}
void AgentCore::hookToolImages(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString images = imagesText(list, n);
if (images.isEmpty())
return; /* a call that made no picture needs no signal */
const QString id = QString::fromUtf8(call && call->id ? call->id : "");
self->post([self, id, images] { emit self->toolImages(id, images); });
}
void AgentCore::hookTodo(const char *text, void *ud) {
AgentCore *self = static_cast<AgentCore *>(ud);
const QString t = QString::fromUtf8(text ? text : "");
+20 -1
View File
@@ -63,7 +63,8 @@ class AgentCore : public QObject {
Q_INVOKABLE QVariantList sessions() const;
/* The open session as transcript rows, in exactly the shape the live
* signals build (kind, role, text, name, state, detail, output, exitCode,
* complete), so a reopened session renders like the turn that wrote it. */
* complete, images), so a reopened session renders like the turn that wrote
* it. */
Q_INVOKABLE QVariantList history() const;
/* None of these run during a turn; all return false when one does. */
Q_INVOKABLE bool newSession();
@@ -128,6 +129,16 @@ class AgentCore : public QObject {
* clipboard) or stay with the editor (text). */
Q_INVOKABLE bool clipboardHasImage() const;
Q_INVOKABLE void clearPending();
/* The images among the pending attachments, in the engine's order, in the
* same text history() and toolImages() carry (see imagesText()): one line
* per picture, "<stored name>|<name>|<origin>|<w>x<h>". Empty for a PDF or
* a text file, and for a picture whose stored name the engine did not
* report. */
Q_INVOKABLE QString pendingImages() const;
/* Where the current session keeps its stored images: the directory
* image://cstattach/ serves (SessionImages.cpp). "" until the engine is
* open. It follows the session. */
Q_INVOKABLE QString attachmentDir() const;
/* 1 the model is shown images, 0 it is not (they go as notes), -1 unknown. */
Q_INVOKABLE int seesImages() const;
@@ -176,6 +187,12 @@ class AgentCore : public QObject {
const QString &detail);
void toolResult(const QString &id, const QString &name, int exitCode,
const QString &output);
/* Images a tool produced (view_image, an MCP image), right after its
* result, as the text ImageStrip.qml decodes. */
void toolImages(const QString &callId, const QString &images);
/* The images the prompt that was just sent carries, in the same shape,
* fired before the turn's first delta. */
void promptImages(const QString &images);
void todoChanged(const QString &text);
void permissionRequested(const QString &tool, const QString &detail,
const QString &preview);
@@ -203,6 +220,8 @@ class AgentCore : public QObject {
const char *detail, void *ud);
static void hookToolResult(const ctl_tool_call_t *call, const char *out,
size_t len, int exit_code, void *ud);
static void hookToolImages(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud);
static void hookTodo(const char *text, void *ud);
static int hookPermission(const char *tool, const char *detail,
const char *preview, void *ud);
+39 -1
View File
@@ -27,9 +27,24 @@
#include "core/AgentCore.h"
#include "core/ToolProvider.h"
#include "ui/UiTest.h"
#include "ui/SessionImages.h"
#include "ui/ThemeIcons.h"
#include "cst_version.h"
#if defined(Q_OS_ANDROID)
/* Where an Android app is allowed to write: its own data directory. There is
* no usable $HOME, no writable "/" (QDir::currentPath()) and no /tmp, while
* the engine wants a config dir ($XDG_CONFIG_HOME/cst-agent, os_config_dir())
* and a scratch dir (the first writable entry of os_temp_dir()'s list). All
* of them are pointed here -- once, before the engine exists. */
QString androidRoot() {
const QString root =
QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);
return root.isEmpty() ? QDir::tempPath() + QStringLiteral("/cst-agent")
: root;
}
#endif
namespace {
int runSelftest() {
@@ -141,8 +156,13 @@ int main(int argc, char **argv) {
parser.process(app);
QString ws = parser.value(wsOpt);
if (ws.isEmpty())
if (ws.isEmpty()) {
#if defined(Q_OS_ANDROID)
ws = androidRoot() + QStringLiteral("/workspace");
#else
ws = QDir::currentPath();
#endif
}
ws = QFileInfo(ws).absoluteFilePath();
/* --uitest drives a scratch workspace when none was given: the session
@@ -160,6 +180,17 @@ int main(int argc, char **argv) {
}
}
#if defined(Q_OS_ANDROID)
/* Before the engine is created: its settings, credentials and session store
* all resolve their paths from the environment. */
const QString root = androidRoot();
for (const QString &sub : {QStringLiteral("config"), QStringLiteral("tmp")})
QDir().mkpath(root + QLatin1Char('/') + sub);
qputenv("XDG_CONFIG_HOME", (root + QStringLiteral("/config")).toUtf8());
qputenv("TMPDIR", (root + QStringLiteral("/tmp")).toUtf8());
qputenv("HOME", root.toUtf8());
#endif
AgentCore core;
QString err;
if (!core.open(ws, parser.value(providerOpt), parser.value(modelOpt),
@@ -195,6 +226,13 @@ int main(int argc, char **argv) {
QQmlApplicationEngine engine;
/* image://csttheme/ plus the desktop's icon theme behind it. */
app::installThemeIcons(&engine);
/* image://cstattach/: the session's own store, so the pictures a tool made
* (view_image, an MCP image) and the ones the user attached are shown, not
* only named. The directory follows the session. */
app::SessionImages *images = app::installSessionImages(&engine);
images->setDir(core.attachmentDir());
QObject::connect(&core, &AgentCore::sessionChanged, images,
[images, &core] { images->setDir(core.attachmentDir()); });
engine.rootContext()->setContextProperty(QStringLiteral("agent"), &core);
engine.rootContext()->setContextProperty(QStringLiteral("initialWidth"), winW);
engine.rootContext()->setContextProperty(QStringLiteral("initialHeight"), winH);
+43
View File
@@ -0,0 +1,43 @@
/* Platform tools on Android, implemented by the app: the same contract as
* app/src/platform/linux/NotifyTool.cpp, with Android APIs behind it.
*
* The engine only ever sees the capability: `platforms = kAndroidOnly` (the
* engine reports "android" when it was compiled with the NDK, see
* src/tools/registry.c: cst_platform_name()) plus an `available` probe decide
* whether a tool reaches the prompt and tools[] at all -- a Termux build on
* the same device reports "termux" and gets the android tools too, an APK
* never gets the desktop ones. `owner = "app:android"` is what
* unregisterPlatformTools() drops again.
*
* WHERE THE CODE RUNS: `run_ex` is called on the turn's worker thread, never
* on the GUI thread. JNI calls are fine from any thread (QJniEnvironment
* attaches it), Qt GUI objects are not -- a tool that needs the widget tree
* has to hop back with a blocking queued call.
*
* WHAT GOES HERE (one at a time, each with its own probe):
* notification_send NotificationChannel + NotificationManager through
* JNI; hidden until POST_NOTIFICATIONS is granted
* (Android 13+), which the UI asks for on first use.
* clipboard_get/set the phone clipboard through Qt's QClipboard, taking
* the name of the desktop tool that cannot work here
* (see app/README.md: a front end may shadow a built-in
* that has no working backend on its host).
* share ACTION_SEND of a file or text through the system share
* sheet -- the phone's answer to "show this to the user".
*
* Nothing is registered yet: the first APK ships the engine, the window and
* the tool contract, and the tools above arrive next (docs/android.md). */
#include <QString>
#include "core/AgentCore.h"
#include "core/ToolProvider.h"
namespace app {
int platformRegisterAndroid(AgentCore *core) {
if (!core)
return 0;
return 0; /* no Android tools yet */
}
} // namespace app
+81
View File
@@ -0,0 +1,81 @@
#include "ui/SessionImages.h"
#include <QFileInfo>
#include <QImageReader>
#include <QQmlApplicationEngine>
#include "cst_attach.h"
namespace app {
namespace {
/* "img-1a2b3c4d.png?w=160": the stored name, then what the view asked for.
* The size comes from the QML side as sourceSize, which Qt passes in as
* requestedSize; the query string is accepted too, so a plain URL can ask. */
QString relOf(const QString &id) {
const int q = id.indexOf(QLatin1Char('?'));
return q < 0 ? id : id.left(q);
}
} // namespace
SessionImages::SessionImages()
: QQuickImageProvider(QQuickImageProvider::Image) {}
void SessionImages::setDir(const QString &dir) {
QMutexLocker lock(&m_mu);
m_dir = dir;
}
QImage SessionImages::requestImage(const QString &id, QSize *size,
const QSize &requestedSize) {
const QString rel = relOf(id);
if (!cst_attach_rel_ok(rel.toUtf8().constData()))
return QImage(); /* QML sees Image.Error and falls back to the name */
QString dir;
{
QMutexLocker lock(&m_mu);
dir = m_dir;
}
if (dir.isEmpty())
return QImage();
const QString path = dir + QLatin1Char('/') + rel;
if (!QFileInfo::exists(path))
return QImage();
QImageReader reader(path);
reader.setAutoTransform(true);
const QSize full = reader.size();
if (!full.isValid()) {
/* No size in the header (or not a picture at all): hand back what Qt
* makes of it and let the view scale it. */
const QImage img = reader.read();
if (size)
*size = img.size();
return img;
}
/* Shrink to what the view shows, never up: a screenshot is decoded at the
* size of its thumbnail instead of at 4K. */
const bool smaller = requestedSize.isValid() && requestedSize.width() > 0 &&
requestedSize.height() > 0 &&
(full.width() > requestedSize.width() ||
full.height() > requestedSize.height());
if (smaller) {
const QSize scaled = full.scaled(requestedSize, Qt::KeepAspectRatio);
reader.setScaledSize(scaled);
if (size)
*size = scaled;
} else if (size) {
*size = full;
}
return reader.read();
}
SessionImages *installSessionImages(QQmlApplicationEngine *engine) {
SessionImages *images = new SessionImages;
if (engine)
engine->addImageProvider(QStringLiteral("cstattach"), images);
return images;
}
} // namespace app
+45
View File
@@ -0,0 +1,45 @@
/* The session's stored images as a QML image provider:
*
* image://cstattach/<stored name> (see cst_attach.h: "img-1a2b3c4d.png")
*
* The files live in the current session's attachment directory, which is what
* AgentCore::attachmentDir() names. Only that directory is served, and only
* names that pass the engine's own cst_attach_rel_ok() -- a transcript can
* carry hand-edited JSON, and nothing in it may reach outside the store.
*
* requestImage() runs on Quick's render thread while setDir() comes from the
* window's thread, hence the mutex. */
#ifndef CST_APP_SESSION_IMAGES_H
#define CST_APP_SESSION_IMAGES_H
#include <QImage>
#include <QMutex>
#include <QQuickImageProvider>
#include <QString>
class QQmlApplicationEngine;
namespace app {
class SessionImages : public QQuickImageProvider {
public:
SessionImages();
/* The directory holding the current session's images; "" disables serving. */
void setDir(const QString &dir);
QImage requestImage(const QString &id, QSize *size,
const QSize &requestedSize) override;
private:
mutable QMutex m_mu;
QString m_dir;
};
/* Registers image://cstattach/ on the engine and hands the provider back, so
* the caller can keep its directory in step with the agent's session. */
SessionImages *installSessionImages(QQmlApplicationEngine *engine);
} // namespace app
#endif // CST_APP_SESSION_IMAGES_H
+17 -4
View File
@@ -48,6 +48,21 @@ void adoptIconTheme() {
}
}
/* The bundled set: app/icons/*.svg, compiled in as :/cst-icons/<name>.svg by
* app/CMakeLists.txt. A machine without an icon theme -- Android, where the
* Qt theme lookup finds nothing and QML was left showing its glyph fallback,
* which the phone fonts cannot draw -- gets the same icons from here. */
QIcon bundledIcon(const QString &name) {
return QIcon(QStringLiteral(":/cst-icons/") + name + QStringLiteral(".svg"));
}
QIcon iconForName(const QString &name) {
const QIcon themed = QIcon::fromTheme(name);
if (!themed.isNull())
return themed;
return bundledIcon(name);
}
/* "?color=%23eff0f1": the tint, and nothing else so far. */
QColor colorOption(const QString &id, QString *name) {
QColor color;
@@ -85,7 +100,7 @@ QImage ThemeIconProvider::requestImage(const QString &id, QSize *size,
if (name.isEmpty())
return QImage();
const QIcon icon = QIcon::fromTheme(name);
const QIcon icon = iconForName(name);
if (icon.isNull())
return QImage(); /* QML shows its fallback glyph */
@@ -116,8 +131,6 @@ void installThemeIcons(QQmlApplicationEngine *engine) {
QString iconThemeName() { return QIcon::themeName(); }
bool themeIconAvailable(const QString &name) {
return QIcon::hasThemeIcon(name);
}
bool themeIconAvailable(const QString &name) { return !iconForName(name).isNull(); }
} // namespace app
+125
View File
@@ -10,10 +10,12 @@
#include <QGuiApplication>
#include <QClipboard>
#include <QBuffer>
#include <QFile>
#include <QImage>
#include <QJsonDocument>
#include <QJsonObject>
#include <QPainter>
#include <QPointingDevice>
#include <QQuickItem>
#include <QQmlApplicationEngine>
@@ -27,6 +29,8 @@
#include <cstring>
#include "cst_agent.h"
#include "cst_attach.h"
#include "ui/SessionImages.h"
#include "ui/ThemeIcons.h"
namespace app {
@@ -118,7 +122,26 @@ QString writeFixtureSession(const QString &ws, const QString &id) {
memset(&m, 0, sizeof(m));
m.role = CST_ROLE_USER;
m.content = strdup("what is here?");
/* The prompt carries a picture, stored exactly the way the engine stores
* one (the file next to the session, the message naming it). */
{
QImage pic(320, 200, QImage::Format_RGB32);
pic.fill(Qt::darkCyan);
QPainter painter(&pic);
painter.fillRect(QRect(0, 0, 160, 100), Qt::darkYellow);
painter.end();
QByteArray png;
QBuffer buf(&png);
buf.open(QIODevice::WriteOnly);
pic.save(&buf, "PNG");
char aerr[256] = "";
m.attach = cst_attach_import_bytes(path, "shot.png", png.constData(),
static_cast<size_t>(png.size()), aerr,
sizeof(aerr));
m.nattach = m.attach ? 1 : 0;
}
cst_session_append(s, &m);
cst_attachments_free(m.attach, m.nattach);
memset(&m, 0, sizeof(m));
m.role = CST_ROLE_ASSISTANT;
m.content = strdup("let me look. ");
@@ -175,6 +198,12 @@ int runUiTest(AgentCore &core, const QString &size) {
QQmlApplicationEngine engine;
installThemeIcons(&engine); /* the window's icons, same as main() */
/* image://cstattach/: the session's stored images, pointed at the session
* the engine has open (and re-pointed on every switch). */
SessionImages *images = installSessionImages(&engine);
images->setDir(core.attachmentDir());
QObject::connect(&core, &AgentCore::sessionChanged, images,
[images, &core] { images->setDir(core.attachmentDir()); });
engine.rootContext()->setContextProperty(QStringLiteral("agent"), &core);
engine.rootContext()->setContextProperty(QStringLiteral("initialWidth"), width);
engine.rootContext()->setContextProperty(QStringLiteral("initialHeight"), height);
@@ -303,6 +332,36 @@ int runUiTest(AgentCore &core, const QString &size) {
clickMouse(win, QStringLiteral("sessionRow:") + fixture);
check(core.sessionId() == fixture && list && list->property("count").toInt() == 5,
QStringLiteral("switching back replays it again"));
{
bool promptImage = false;
const QVariantList rows = core.history();
for (const QVariant &v : rows) {
const QVariantMap r = v.toMap();
if (r.value(QStringLiteral("role")).toString() == QLatin1String("user") &&
r.value(QStringLiteral("images")).toString().contains(
QLatin1String("|")))
promptImage = true;
}
check(promptImage,
QStringLiteral("a stored prompt image comes back with its row"));
/* The rows are in the model by now; the cards -- and the thumbnails in
* them -- are built on the view's next layout pass. A frame forces it,
* and that frame is also the picture a person looks at. */
maybeShot(win, QStringLiteral("-images"));
QTest::qWait(150);
if (qEnvironmentVariableIsSet("CST_APP_DUMP")) {
QVariant v;
QMetaObject::invokeMethod(win, "rowJson", Q_RETURN_ARG(QVariant, v),
Q_ARG(QVariant, 0));
std::printf("note: replayed row 0: %s\n", qPrintable(v.toString()));
std::printf("note: %d strips and %d thumbs in the tree\n",
countNamed(win->contentItem(), QStringLiteral("imageStrip")),
countNamed(win->contentItem(), QStringLiteral("imageThumb")));
}
check(countNamed(win->contentItem(), QStringLiteral("imageThumb")) >= 1,
QStringLiteral("the replayed transcript shows it as a thumbnail (%1)")
.arg(countNamed(win->contentItem(), QStringLiteral("imageThumb"))));
}
/* --- provider, model, key: the settings dialog ------------------------ */
check(core.providers().size() > 0 &&
@@ -558,6 +617,72 @@ int runUiTest(AgentCore &core, const QString &size) {
check(false, QStringLiteral("assistant deltas are reachable"));
}
/* --- the image provider: only the session's own files ----------------- */
{
const QString dir = core.attachmentDir();
check(!dir.isEmpty() && dir.contains(QLatin1String("/attachments/")),
QStringLiteral("the agent names its image directory (%1)").arg(dir));
QImage pic(32, 16, QImage::Format_RGB32);
pic.fill(Qt::darkGreen);
QByteArray png;
QBuffer buf(&png);
buf.open(QIODevice::WriteOnly);
pic.save(&buf, "PNG");
char aerr[256] = "";
cst_attachment_t *stored = cst_attach_import_bytes(
cst_agent_session_path(core.agent()), "probe.png", png.constData(),
static_cast<size_t>(png.size()), aerr, sizeof(aerr));
check(stored != nullptr,
QStringLiteral("a picture can be stored next to the session"));
if (stored) {
QSize got;
const QImage shrunk =
images->requestImage(QString::fromUtf8(stored->rel),
&got, QSize(8, 8));
check(!shrunk.isNull() && shrunk.width() == 8 && shrunk.height() == 4,
QStringLiteral("the provider serves it, shrunk to what was asked "
"(%1x%2)")
.arg(shrunk.width())
.arg(shrunk.height()));
check(images->requestImage(QStringLiteral("../escape.png"), &got,
QSize(64, 64))
.isNull(),
QStringLiteral("a name outside the attachment directory is refused"));
check(images->requestImage(QStringLiteral("no-such-file.png"), &got,
QSize(64, 64))
.isNull(),
QStringLiteral("a file that is not there is refused"));
cst_attach_free(stored);
}
}
/* --- a tool that made a picture: its card shows it -------------------- */
{
const int before = list->property("count").toInt();
const int thumbsBefore =
countNamed(win->contentItem(), QStringLiteral("imageThumb"));
event4("engineToolEvent", QStringLiteral("open"), QStringLiteral("call-img"),
QStringLiteral("view_image"), QStringLiteral("{\"path\":\"shot.png\"}"));
const QString imgs =
QStringLiteral("img-fixture.png|shot.png|view_image|24x8");
check(QMetaObject::invokeMethod(win, "engineToolImages",
Q_ARG(QVariant, QStringLiteral("call-img")),
Q_ARG(QVariant, imgs)),
QStringLiteral("images a tool made reach the view"));
QTest::qWait(150);
const QVariantMap card = rowAt(before);
check(card.value(QStringLiteral("images"))
.toString()
.contains(QLatin1String("img-fixture.png")) &&
card.value(QStringLiteral("name")).toString() ==
QLatin1String("view_image"),
QStringLiteral("the card of that call carries the image"));
check(countNamed(win->contentItem(), QStringLiteral("imageThumb")) >
thumbsBefore,
QStringLiteral("its thumbnail is in the card"));
maybeShot(win, QStringLiteral("-toolimage"));
}
/* --- a narrow window: dock -> drawer, and the button opens it --- */
resizeTo(win, 460, 900);
check(docked && !docked->isVisible(),
+9 -2
View File
@@ -9,6 +9,7 @@
#include "cst_agent.h"
#include "cst_tools.h"
#include "os_compat.h"
#include <stdio.h>
#include <stdlib.h>
@@ -54,8 +55,14 @@ static int probe_missing(char *why, size_t why_sz) {
}
int main(void) {
char ws[1024];
snprintf(ws, sizeof(ws), "/tmp/cst-app-selftest-%d", (int)getpid());
/* The scratch workspace goes where this host can actually write: /tmp on a
* desktop, the app's own directory on Android (where /tmp is off limits,
* so TMPDIR is pointed at it -- see app/src/main.cpp). */
char tmp[1024] = "";
if (os_temp_dir(tmp, sizeof(tmp)) != 0 || !tmp[0])
snprintf(tmp, sizeof(tmp), "%s", "/tmp");
char ws[1200];
snprintf(ws, sizeof(ws), "%s/cst-app-selftest-%d", tmp, (int)getpid());
#ifdef _WIN32
_mkdir(ws); /* the engine wants its workspace to exist */
#else
+99 -5
View File
@@ -38,6 +38,95 @@ nix-shell -p cmake ninja curl android-tools --run '
`--run-tests` builds the whole `tests/` suite, pushes it and runs every binary
on the phone, printing a pass/fail line per test.
## The APK (Qt front end)
`scripts/build_apk_android.sh [--install] [--logs] [--clean]` configures with
the Qt for Android kit, builds the `apk` target (Gradle, no port opened) and
optionally installs it. Bits it needs, all overridable through the
environment:
| variable | what it points at |
| --- | --- |
| `QT_ANDROID` | Qt for Android 6.11.2 (`aqt install-qt all_os android 6.11.2 android_arm64_v8a -O ~/Qt --archives qtbase qtdeclarative qtquickcontrols2`) |
| `QT_HOST` | the host Qt of the same version (`aqt install-qt linux desktop 6.11.2 linux_gcc_64 -O ~/Qt --archives qtbase qtdeclarative qtsvg`) |
| `ANDROID_SDK_ROOT` | SDK with `platforms/android-<n>` and `build-tools/<v>` |
| `ANDROID_NDK_HOME` | the NDK the engine was built with |
| `JAVA_HOME` | JDK 17+ for Gradle |
### The SDK without sdkmanager
`nixpkgs`'s full `androidsdk` pulls the emulator and system images (gigabytes);
the two components actually needed are cheap. The platform comes from the nix
store (unfree, so the license flag is required) and build-tools from Google's
own zip, laid out as the SDK `sdkmanager` would:
```sh
PLATFORM=$(NIXPKGS_ALLOW_UNFREE=1 NIXPKGS_ACCEPT_ANDROID_SDK_LICENSE=1 nix build \
--impure --no-link --print-out-paths --expr \
'builtins.elemAt (import <nixpkgs> {}).androidenv.androidPkgs.platforms 2') # android-35
mkdir -p ~/android-sdk/{build-tools,platforms}
ln -s $PLATFORM/libexec/android-sdk/platforms/android-35 ~/android-sdk/platforms/
```
`androiddeployqt` then picks `build-tools` and `platform-tools` from there. For
build-tools, `dl.google.com` is the only source: from here it hands out
~100 KB/s per connection, so the script downloads the zip in 16 ranged
requests (≈25x faster) and fixes the exec bits Python's `zipfile` drops.
### Host Qt on NixOS
The host Qt is built for a regular distribution: its tools (`qmlimportscanner`
first of all) look for `libgssapi_krb5.so.2`, `libglib-2.0.so.0`, `libz.so.1`,
`libfontconfig`, `libX11`, `libEGL`/`libGL`, `libzstd`, `libbrotlidec`,
`libdbus-1`, `libfreetype`, `libstdc++`, none of which exist in NixOS's search
paths. The tools carry `RUNPATH=$ORIGIN/../lib`, so symlinking those libraries
from the nix store into `<QT_HOST>/lib/` is enough:
```sh
nix build --no-link --print-out-paths nixpkgs#krb5.lib nixpkgs#dbus.lib \
nixpkgs#libglvnd nixpkgs#fontconfig.lib nixpkgs#freetype nixpkgs#glib.lib \
nixpkgs#libx11 nixpkgs#libxkbcommon nixpkgs#zlib nixpkgs#zstd \
nixpkgs#brotli.lib nixpkgs#stdenv.cc.cc.lib # then ln -s <lib> ~/Qt/6.11.2/gcc_64/lib/
```
Two more things the script does on the way:
* **GLESv2/EGL**: Qt's `FindGLESv2.cmake` only searches `CMAKE_FIND_ROOT_PATH`
prefixes (`<sysroot>/include`), but the NDK keeps the headers in
`<sysroot>/usr/include` -- without `-DGLESv2_INCLUDE_DIR`/`-DEGL_INCLUDE_DIR`
`find_package(Qt6 Gui)` fails with "dependency GLESv2 could not be found".
* **`-L$PREFIX/lib` on every link type**: the APK's payload is a *shared
module*, so `CMAKE_SHARED_LINKER_FLAGS`/`CMAKE_MODULE_LINKER_FLAGS` need the
prefix too, not just the EXE flags.
### Gradle
The generated project resolves plugins and dependencies from `google()` and
`mavenCentral()`, plus a Gradle distribution from `services.gradle.org`. From
here all three are slow (dl.google.com ~100 KB/s), so two local tweaks:
* `~/.gradle/init.d/*.gradle` (machine-local, not in the repo) prepends the
Aliyun/Tencent mirrors for `pluginManagement`, `dependencyResolutionManagement`
and every `buildscript`.
* the wrapper in `<QT_ANDROID>/src/3rdparty/gradle/gradle/wrapper/` points at
the Tencent Gradle mirror (51 MB/s vs 100 KB/s).
`scripts/gradle-android-init.gradle` (installed into `~/.gradle/init.d/` by the
build script) switches off release lint: `lintVitalAnalyzeRelease` resolves its
own tool jars straight from `dl.google.com`, bypassing every repository the
build configures, and fails the build after all the real work is done.
### Signing
`androiddeployqt` copies the **unsigned** Gradle output and signs it itself
(`QT_ANDROID_SIGN_APK` + the `QT_ANDROID_KEYSTORE_*` variables; a signed Gradle
output makes it fail with "Could not copy unsigned APK"). The script exports
those variables and creates the standard Android debug keystore
(`~/.android/debug.keystore`, alias `androiddebugkey`) on first use;
`CST_ANDROID_KEYSTORE` and `CST_ANDROID_KEY_ALIAS`/`_PASS` override it.
`apksigner` is a bash script with a `#!/bin/bash` shebang, which NixOS does not
have, so the script rewrites the shebang of the build-tools wrappers.
## What is Android-specific
* **API level 24** (`API_LEVEL=24`): bionic provides threads, `exec` and
@@ -66,8 +155,13 @@ on the phone, printing a pass/fail line per test.
* The whole suite runs on the device (`scripts/build_android.sh arm64-v8a
--run-tests`): 36/36 after the diff test learned to tell toybox's `diff` from
GNU diffutils.
* Still to come: the Qt front end packaged as an APK. nixpkgs has no Qt for
Android build, so that needs a Qt host kit plus the `android_arm64_v8a` kit
(qt.io or aqtinstall), a JDK and the Android SDK platform/build-tools;
`app/src/platform/android/` (JNI tools, owner `app:android`) and the Gradle
project under `app/android/` do not exist yet.
* The APK builds, installs and runs: the Qt front end links the engine
in-process (no port), `main.cpp` puts the workspace, config and tmp under
`/data/user/0/xyz.whcself.cstagent/files/`, and a turn does a real HTTPS
request from inside the ART process (with no API key the provider answers
401 -- the engine, curl and TLS parts of the round trip are what that
shows). Rotation keeps the session; the sidebar appears in landscape.
* Still to come: the Android platform tools behind `app:android`
(`notification_send`, clipboard, share sheet, SAF file picker), the back
gesture, and the icon glyphs -- `ThemeIcon` finds no icon files on the phone
and draws empty boxes where the desktop shows Breeze icons.
+12
View File
@@ -334,6 +334,12 @@ typedef struct {
/* Full tool output, fired right after execution (before it is persisted). */
void (*on_tool_result)(const ctl_tool_call_t *call, const char *out,
size_t len, int exit_code, void *ud);
/* Images a tool produced (view_image, MCP image content), fired right after
* the call ran and before the result is persisted. Only a call that made at
* least one image is reported, so a front end can show it at once instead of
* reading the store. */
void (*on_tool_images)(const ctl_tool_call_t *call,
const cst_attachment_t *list, int n, void *ud);
void *userdata;
} cst_agent_hooks_t;
@@ -378,6 +384,12 @@ int cst_agent_pending_count(const cst_agent_t *a);
/* One line "<name> (<mime>, 12 KB)"; -1 when i is out of range. */
int cst_agent_pending_describe(const cst_agent_t *a, int i, char *out,
size_t out_sz);
/* The stored name of pending attachment i ("img-1a2b3c4d.png"), or NULL when
* i is out of range. The file is already in the session store, so a front end
* can show the image before the message is sent. */
const char *cst_agent_pending_rel(const cst_agent_t *a, int i);
/* 1 when pending attachment i is an image, 0 otherwise (or out of range). */
int cst_agent_pending_is_image(const cst_agent_t *a, int i);
void cst_agent_pending_clear(cst_agent_t *a);
/* Latest todo_write list of this session, or NULL. */
const char *cst_agent_todo(const cst_agent_t *a);
+2 -2
View File
@@ -2,8 +2,8 @@
#define CST_VERSION_H
#define CST_VERSION_MAJOR 0
#define CST_VERSION_MINOR 8
#define CST_VERSION_PATCH 15
#define CST_VERSION_MINOR 9
#define CST_VERSION_PATCH 0
#define CST_VERSION_STR_(x) #x
#define CST_VERSION_STR(x) CST_VERSION_STR_(x)
+168
View File
@@ -0,0 +1,168 @@
#!/usr/bin/env bash
# Build the Qt Quick Android APK of app/ (the engine is linked in-process, so
# the APK opens no port). This is the front end counterpart of
# scripts/build_android.sh, which cross-builds the engine alone.
#
# scripts/build_apk_android.sh [--install] [--logs] [--clean]
#
# --install adb install -r the debug APK on the connected device
# --logs stream logcat after installing (Qt + engine tags)
# --clean wipe the build directory first
#
# Needs, all overridable through the environment:
# QT_ANDROID Qt for Android (default ~/Qt/<ver>/android_arm64_v8a)
# QT_HOST matching host Qt (default ~/Qt/<ver>/gcc_64)
# ANDROID_SDK_ROOT SDK with platforms/ + build-tools/ (see docs/android.md)
# ANDROID_NDK_HOME NDK, same one as scripts/build_third_party_android.sh
# JAVA_HOME JDK 17+ for Gradle (falls back to the javac on PATH)
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
ABI="${ABI:-arm64-v8a}"
API_LEVEL="${API_LEVEL:-24}"
BUILD="$ROOT/build-apk-android"
PREFIX="$ROOT/third_party/prefix-android-$ABI"
JOBS="${JOBS:-$(nproc 2>/dev/null || echo 4)}"
DO_INSTALL=0
DO_LOGS=0
for a in "$@"; do
case "$a" in
--install) DO_INSTALL=1 ;;
--logs) DO_INSTALL=1; DO_LOGS=1 ;;
--clean) rm -rf "$BUILD" ;;
*) echo "usage: $0 [--install] [--logs] [--clean]" >&2; exit 1 ;;
esac
done
# --- Qt: Android target first, then the host tools of the same version ----
if [[ -z "${QT_ANDROID:-}" ]]; then
# aqt names the directory after the ABI with underscores: android_arm64_v8a
for q in "$HOME"/Qt/*/android_"${ABI//-/_}"/lib/cmake/Qt6/qt.toolchain.cmake; do
[[ -f "$q" ]] && QT_ANDROID="${q%/lib/cmake/Qt6/qt.toolchain.cmake}"
done
fi
if [[ -z "${QT_ANDROID:-}" || ! -x "$QT_ANDROID/bin/qt-cmake" ]]; then
echo "set QT_ANDROID to a Qt for Android install ('aqt install-qt all_os android ...')" >&2
exit 1
fi
QT_VER="$(basename "$(dirname "$QT_ANDROID")")"
QT_HOST="${QT_HOST:-$(dirname "$QT_ANDROID")/gcc_64}"
if [[ ! -x "$QT_HOST/bin/qmake" && ! -x "$QT_HOST/bin/androiddeployqt" ]]; then
echo "set QT_HOST to the host Qt $QT_VER (same version, needed for androiddeployqt)" >&2
exit 1
fi
NDK="${ANDROID_NDK_HOME:-${ANDROID_NDK_ROOT:-}}"
if [[ -z "$NDK" || ! -f "$NDK/build/cmake/android.toolchain.cmake" ]]; then
echo "set ANDROID_NDK_HOME to an NDK (see scripts/build_third_party_android.sh)" >&2
exit 1
fi
if [[ -z "${ANDROID_SDK_ROOT:-}" || ! -d "$ANDROID_SDK_ROOT/platforms" ]]; then
echo "set ANDROID_SDK_ROOT to an SDK with platforms/android-<n> and build-tools/" >&2
exit 1
fi
if [[ -z "${JAVA_HOME:-}" ]]; then
JAVAC="$(command -v javac || true)"
[[ -n "$JAVAC" ]] && JAVA_HOME="$(dirname "$(dirname "$(readlink -f "$JAVAC")")")"
fi
if [[ -z "${JAVA_HOME:-}" || ! -x "$JAVA_HOME/bin/java" ]]; then
echo "set JAVA_HOME to a JDK 17+ (Gradle needs it)" >&2
exit 1
fi
if [[ ! -f "$PREFIX/lib/libcurl.a" ]]; then
echo "missing $PREFIX/lib/libcurl.a -- run:" >&2
echo " scripts/build_third_party_android.sh $ABI all" >&2
exit 1
fi
export ANDROID_SDK_ROOT ANDROID_NDK_HOME="$NDK" ANDROID_NDK_ROOT="$NDK" JAVA_HOME
export PATH="$JAVA_HOME/bin:$PATH"
# androiddeployqt shells out to the generated project's gradlew, which picks up
# init scripts from $GRADLE_USER_HOME/init.d. Install ours (release lint off)
# so every build behaves the same. See the file for why lint has to go.
GRADLE_INIT_DIR="${GRADLE_USER_HOME:-$HOME/.gradle}/init.d"
mkdir -p "$GRADLE_INIT_DIR"
if ! cmp -s "$ROOT/scripts/gradle-android-init.gradle" "$GRADLE_INIT_DIR/cst-android.gradle"; then
install -m644 "$ROOT/scripts/gradle-android-init.gradle" "$GRADLE_INIT_DIR/cst-android.gradle"
echo "installed $GRADLE_INIT_DIR/cst-android.gradle"
fi
# The release APK has to be signed to install. With CST_ANDROID_KEYSTORE the
# init script uses that keystore; otherwise it falls back to the standard
# Android debug key, created here on first use (same one Android Studio makes).
KEYSTORE="${CST_ANDROID_KEYSTORE:-$HOME/.android/debug.keystore}"
if [[ ! -f "$KEYSTORE" && -z "${CST_ANDROID_KEYSTORE:-}" ]]; then
mkdir -p "$(dirname "$KEYSTORE")"
keytool -genkeypair -keystore "$KEYSTORE" -storepass android -keypass android \
-alias androiddebugkey -keyalg RSA -keysize 2048 -validity 10000 \
-dname "CN=Android Debug,O=Android,C=US" >/dev/null
chmod 600 "$KEYSTORE"
echo "created debug keystore: $KEYSTORE"
fi
# androiddeployqt reads these (the path/alias/passwords never reach the CMake
# command line, only the QT_ANDROID_SIGN_APK flag does).
export QT_ANDROID_KEYSTORE_PATH="$KEYSTORE"
export QT_ANDROID_KEYSTORE_ALIAS="${CST_ANDROID_KEY_ALIAS:-androiddebugkey}"
export QT_ANDROID_KEYSTORE_STORE_PASS="${CST_ANDROID_KEYSTORE_PASS:-android}"
export QT_ANDROID_KEYSTORE_KEY_PASS="${CST_ANDROID_KEY_PASS:-android}"
# NixOS has no /bin/bash, and build-tools' shell wrappers (apksigner is the one
# androiddeployqt calls, d8 and friends are the rest) start with #!/bin/bash.
# Point them at the bash that exists here; the SDK is a local copy, so this
# sticks.
BT_DIR="$(ls -d "$ANDROID_SDK_ROOT"/build-tools/* 2>/dev/null | sort -V | tail -1)"
BASH_BIN="$(command -v bash || echo /bin/sh)"
if [[ -n "$BT_DIR" ]]; then
while IFS= read -r f; do
sed -i "1s|^#!/bin/bash|#!$BASH_BIN|" "$f" && echo "fixed shebang: ${f#"$ANDROID_SDK_ROOT"/}"
done < <(grep -rls '^#!/bin/bash' "$BT_DIR" 2>/dev/null)
fi
# Qt6Gui wants GLESv2/EGL: the NDK sysroot has both, but Qt's Find modules
# only search CMAKE_FIND_ROOT_PATH prefixes (<sysroot>/include), while the
# headers live in <sysroot>/usr/include -- point them there directly.
NDK_SYSROOT_INC="$NDK/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include"
[[ -d "$NDK_SYSROOT_INC/GLES2" ]] || echo "warning: no GLES2 headers under $NDK_SYSROOT_INC" >&2
echo "Qt for Android : $QT_ANDROID ($QT_VER)"
echo "host Qt : $QT_HOST"
echo "SDK : $ANDROID_SDK_ROOT ($(ls "$ANDROID_SDK_ROOT/build-tools" | tr '\n' ' '))"
echo "NDK : $NDK"
echo "JDK : $JAVA_HOME"
echo "ABI : $ABI, minSdk $API_LEVEL"
# The engine comes from libcstcore.a, exactly as on the desktop: no TUI, no
# HTTP server (src/webui_stub.c stands in), no bundled busybox inside an APK.
"$QT_ANDROID/bin/qt-cmake" -S "$ROOT" -B "$BUILD" -G Ninja \
-DQT_HOST_PATH="$QT_HOST" \
-DANDROID_ABI="$ABI" -DANDROID_PLATFORM="android-$API_LEVEL" \
-DANDROID_SDK_ROOT="$ANDROID_SDK_ROOT" -DANDROID_NDK="$NDK" \
-DGLESv2_INCLUDE_DIR="$NDK_SYSROOT_INC" -DEGL_INCLUDE_DIR="$NDK_SYSROOT_INC" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_PREFIX_PATH="$PREFIX" \
-DCURL_INCLUDE_DIR="$PREFIX/include" -DCURL_LIBRARY="$PREFIX/lib/libcurl.a" \
-DCMAKE_EXE_LINKER_FLAGS="-L$PREFIX/lib" \
-DCMAKE_SHARED_LINKER_FLAGS="-L$PREFIX/lib" \
-DCMAKE_MODULE_LINKER_FLAGS="-L$PREFIX/lib" \
-DCST_BUILD_TUI=OFF -DCST_BUILD_WEBUI=OFF -DCST_BUILD_APP=ON \
-DCST_BUNDLE_BUSYBOX=OFF
cmake --build "$BUILD" --target apk -j"$JOBS"
# AGP names the file after the variant and signing state
# (android-build-release.apk vs ...-unsigned.apk), so take the newest one.
APK="$(find "$BUILD/app/android-build/build/outputs/apk" "$BUILD" -name '*.apk' \
-printf '%T@ %p\n' 2>/dev/null | sort -rn | head -1 | cut -d' ' -f2-)"
[[ -n "$APK" && -f "$APK" ]] || { echo "no APK produced" >&2; exit 1; }
echo
echo "APK: $APK ($(du -h "$APK" | cut -f1))"
[[ "$DO_INSTALL" == 1 ]] || exit 0
ADB="${ADB:-adb}"
command -v "$ADB" >/dev/null || { echo "adb not found (nix-shell -p android-tools)" >&2; exit 1; }
"$ADB" install -r "$APK"
echo "installed: $("$ADB" shell pm list packages | grep cstagent)"
[[ "$DO_LOGS" == 1 ]] || exit 0
echo "--- am start (Ctrl-C to stop; logcat follows)"
"$ADB" shell monkey -p xyz.whcself.cstagent -c android.intent.category.LAUNCHER 1 >/dev/null
sleep 2
exec "$ADB" logcat -s cst-agent:V Qt:V AndroidRuntime:E libc:V DEBUG:V
+25
View File
@@ -0,0 +1,25 @@
// Gradle init script for the Android APK build, installed as
// ~/.gradle/init.d/cst-android.gradle by scripts/build_apk_android.sh.
//
// Qt's generated gradle project only sets "lintOptions { abortOnError false }",
// so the release build still runs lintVitalAnalyzeRelease -- and that task
// resolves its own tool jars (manifest-merger, lint-typedef-remover, ...)
// straight from dl.google.com, bypassing every repository the project (or an
// init script) configures. On a slow or filtered route that fails the build
// after all the real work is done, and nothing here wants lint output anyway.
gradle.afterProject { project ->
def android = project.extensions.findByName('android')
if (android != null) {
// AGP keeps both spellings around: lintOptions (deprecated) and lint.
try { android.lintOptions.checkReleaseBuilds = false } catch (Exception ignored) { }
try { android.lint.checkReleaseBuilds = false } catch (Exception ignored) { }
// NOTE: no signing config here on purpose. androiddeployqt copies the
// *unsigned* gradle output and signs it itself (--sign); a signed
// gradle APK makes it fail with "Could not copy unsigned APK". The
// keystore is wired up from app/CMakeLists.txt through
// QT_ANDROID_SIGN_APK + the QT_ANDROID_KEYSTORE_* environment.
}
// Belt and braces: switch the tasks themselves off.
project.tasks.matching { it.name.startsWith('lintVital') }.configureEach { it.enabled = false }
}
+16
View File
@@ -1181,6 +1181,8 @@ static void exec_end(turn_t *t, pcall_t *pc, exec_t *x) {
if (t->hooks && t->hooks->on_tool_result)
t->hooks->on_tool_result(c, pc->output, pc->output_len, pc->exit_code,
t->hooks->userdata);
if (t->hooks && t->hooks->on_tool_images && pc->nattach > 0)
t->hooks->on_tool_images(c, pc->attach, pc->nattach, t->hooks->userdata);
if (pc->status != PC_DENIED) {
char d[96];
snprintf(d, sizeof(d), "%sexit=%d bytes=%zu",
@@ -2173,6 +2175,20 @@ int cst_agent_attach_bytes(cst_agent_t *a, const char *name, const char *data,
err, err_sz);
}
/* The stored name of a pending attachment: the window serves the file from the
* session's attachment directory while the message is still being written. */
const char *cst_agent_pending_rel(const cst_agent_t *a, int i) {
if (!a || i < 0 || i >= a->npending)
return NULL;
return a->pending[i].rel;
}
int cst_agent_pending_is_image(const cst_agent_t *a, int i) {
if (!a || i < 0 || i >= a->npending)
return 0;
return cst_attach_is_image(&a->pending[i]);
}
const char *cst_agent_todo(const cst_agent_t *a) { return a ? a->todo : NULL; }
const char *cst_agent_notes(const cst_agent_t *a) {
+9
View File
@@ -540,6 +540,15 @@ static void test_agent_pending(const char *root) {
EXPECT(cst_agent_pending_describe(a, 0, line, sizeof(line)) == 0 &&
strstr(line, "4x2") != NULL,
"pending describe shows the size");
EXPECT(cst_agent_pending_is_image(a, 0) == 1,
"a pending image says that it is one");
EXPECT(cst_agent_pending_rel(a, 0) != NULL &&
cst_attach_rel_ok(cst_agent_pending_rel(a, 0)),
"its stored name is one a view may serve");
EXPECT(cst_agent_pending_rel(a, -1) == NULL &&
cst_agent_pending_rel(a, 99) == NULL &&
cst_agent_pending_is_image(a, 99) == 0,
"out of range has no name and is no image");
EXPECT(cst_agent_attach_file(a, "/nonexistent/x.png", err, sizeof(err)) == -1,
"missing file refused");