v0.9.0: the Qt front end as an Android APK, images in the transcript
Two strands land together: the window draws pictures now, and the whole front end builds, installs and runs on the phone with the engine linked in-process. Images (the "make a picture actually show up" part) - Engine, additively: cst_agent_hooks_t.on_tool_images(call, list, n, ud) fires when a call left attachments behind, and cst_agent_pending_rel() / cst_agent_pending_is_image() read that list without the app touching the store. tests/test_attach.c::test_agent_pending pins the three cases (an image, a stored name that passes cst_attach_rel_ok, out-of-range NULL). - App: image://cstattach/<stored name> (app/src/ui/SessionImages.cpp) serves only the current session's attachment directory, checked with cst_attach_rel_ok, decodes down to the requested sourceSize and never upscales, and holds a mutex because requestImage() runs on the render thread while the directory can change underneath it. AgentCore gained attachmentDir(), pendingImages(), the toolImages()/promptImages() signals and hookToolImages(); history() gives replayed rows their images too. - QML: ImageStrip.qml (a thumb strip, <=150 px tall, shaped by the picture, tooltip with name/size/origin, the file's name when it went missing), wired into MessageCard and ToolCard. Qt 6's ListModel cannot hold a list-valued role (append() reads back undefined, setProperty drops it), so a row carries one image per line as <stored>|<name>|<origin>|<w>x<h> and ImageStrip decodes that. Android (the engine is already in the APK; this is the front end) - app/src/main.cpp: on Android the workspace, config, tmp and HOME go under the app's own data directory before the engine opens anything, because QDir::currentPath() is "/" there and /tmp is not writable. - app/android/: the manifest (QtApplication/QtActivity, INTERNET and POST_NOTIFICATIONS, adjustResize, no storage permission -- the picker will be the system one), strings, a vector icon; minSdk 24 and targetSdk 35 come from app/CMakeLists.txt through QT_ANDROID_MIN/TARGET_SDK_VERSION. - app/src/platform/android/AndroidTools.cpp: the Android file behind PlatformInit(), owner "app:android" (notification, clipboard, share sheet, file picker are stubs for now, so the tool list does not lie about them). - app/tools/selftest.c decides its scratch workspace with os_temp_dir(), so the C smoke test runs on the device too. - scripts/build_apk_android.sh: sources to installed APK in one command (environment checks, qt-cmake configure, --target apk, adb install), and the host-side plumbing the NixOS + qt.io combination needs -- the NDK sysroot include dirs for Qt's FindGLESv2/FindEGL (they only look under <sysroot>/include), -L$PREFIX/lib on the shared/module link lines as well as the executables', the Android debug keystore created on first use with QT_ANDROID_SIGN_APK + QT_ANDROID_KEYSTORE_* (androiddeployqt copies the unsigned gradle output and signs it itself), and the #!/bin/bash shebang of the build-tools wrappers, which NixOS does not have. - scripts/gradle-android-init.gradle (installed into ~/.gradle/init.d/): the release lint task is switched off -- lintVitalAnalyzeRelease resolves its own tool jars straight from dl.google.com, bypassing every repository the build configures, and fails the build after all the real work is done. - Icons: app/icons/*.svg (Breeze's 24@3x files, the 19 names the window asks for) are compiled in as :/cst-icons and ThemeIcons.cpp falls back to them when the platform has no icon theme. On the phone the Qt theme lookup found nothing and the QML glyph fallback was an empty box in the phone's fonts. - docs/android.md: "The APK (Qt front end)" -- the SDK assembled without sdkmanager, the host Qt's missing libraries on NixOS, the Gradle mirrors, lint, signing -- and a Status that says what runs where. app/README.md gains the images and APK lines, and stops claiming Android platform tools that are still stubs. Verified: engine `make check` green; desktop `cst-app-selftest: OK` and --uitest all checks pass; phone (NX789J, Android 16, arm64-v8a) -- the APK installs, starts on the workspace under /data/user/0/xyz.whcself.cstagent/ files/, a turn reaches a configured provider over HTTPS from inside the ART process and renders the reply, rotation keeps the session and docks the sidebar, and the toolbar/card icons draw. Version 0.9.0. mode = coding
This commit is contained in:
WHC
@@ -40,6 +40,7 @@ Item {
|
||||
role: model.role
|
||||
text: model.text
|
||||
complete: model.complete
|
||||
images: model.images !== undefined ? model.images : ""
|
||||
/* html comes from the streaming renderer (Main.qml) or,
|
||||
* for a replayed session, from here once. */
|
||||
html: model.html.length > 0 ? model.html
|
||||
@@ -65,6 +66,7 @@ Item {
|
||||
output: model.output
|
||||
exitCode: model.exitCode
|
||||
hasResult: model.complete
|
||||
images: model.images !== undefined ? model.images : ""
|
||||
}
|
||||
}
|
||||
Component {
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
import QtQuick
|
||||
import QtQuick.Controls
|
||||
import QtQuick.Layouts
|
||||
|
||||
// The images a message or a tool call carries: thumbnails from the session's
|
||||
// own store (image://cstattach/, app/src/ui/SessionImages.cpp). The names are
|
||||
// the ones the engine wrote next to the session -- view_image, an MCP image, a
|
||||
// picture the user attached -- never a path the view invented.
|
||||
//
|
||||
// One row of thumbnails: the height is capped so a screenshot cannot push the
|
||||
// conversation off the screen, the width follows the picture's shape.
|
||||
Flow {
|
||||
id: strip
|
||||
objectName: "imageStrip"
|
||||
/* One line per picture, written by the engine side (AgentCore.cpp:
|
||||
* "<stored name>|<name>|<origin>|<w>x<h>"). A row carries this as text
|
||||
* because a ListModel role cannot hold a list. */
|
||||
property string text: ""
|
||||
property var images: decode(text)
|
||||
spacing: 6
|
||||
|
||||
function decode(t) {
|
||||
const out = []
|
||||
const lines = String(t).split("\n")
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
if (!lines[i].length)
|
||||
continue
|
||||
const f = lines[i].split("|")
|
||||
if (!f[0].length)
|
||||
continue
|
||||
let w = 0, h = 0
|
||||
const dim = String(f[3] || "").split("x")
|
||||
if (dim.length === 2) {
|
||||
w = parseInt(dim[0], 10) || 0
|
||||
h = parseInt(dim[1], 10) || 0
|
||||
}
|
||||
out.push({ url: "image://cstattach/" + f[0], rel: f[0],
|
||||
name: f[1] || f[0], origin: f[2] || "", w: w, h: h })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
Repeater {
|
||||
model: strip.images
|
||||
|
||||
delegate: Rectangle {
|
||||
id: thumb
|
||||
required property var modelData
|
||||
required property int index
|
||||
readonly property real ratio: {
|
||||
const w = Number(modelData.w), h = Number(modelData.h)
|
||||
return (w > 0 && h > 0) ? w / h : 1.5
|
||||
}
|
||||
readonly property int cap: 150 // the tallest a thumbnail gets
|
||||
objectName: "imageThumb" + index
|
||||
width: Math.max(56, Math.min(Math.round(cap * ratio), 320))
|
||||
height: Math.max(40, Math.min(cap, Math.round(width / ratio)))
|
||||
radius: Theme.radiusSm
|
||||
color: Theme.bg
|
||||
border.width: 1
|
||||
border.color: mouse.containsMouse ? Theme.accentDim : Theme.border
|
||||
clip: true
|
||||
|
||||
Image {
|
||||
id: pic
|
||||
anchors.fill: parent
|
||||
anchors.margins: 2
|
||||
source: String(thumb.modelData.url)
|
||||
sourceSize.width: 512
|
||||
sourceSize.height: 512
|
||||
fillMode: Image.PreserveAspectFit
|
||||
/* Not asynchronous: the provider already hands back a
|
||||
* thumbnail-sized picture, so there is nothing to wait for. */
|
||||
asynchronous: false
|
||||
visible: status === Image.Ready
|
||||
}
|
||||
|
||||
// A file the store no longer has is not worth shouting about: the
|
||||
// frame says which image it was.
|
||||
Text {
|
||||
anchors.centerIn: parent
|
||||
width: parent.width - 8
|
||||
visible: pic.status === Image.Error
|
||||
text: String(thumb.modelData.name)
|
||||
elide: Text.ElideMiddle
|
||||
horizontalAlignment: Text.AlignHCenter
|
||||
color: Theme.textFaint
|
||||
font.pixelSize: Theme.smallSize
|
||||
}
|
||||
|
||||
MouseArea {
|
||||
id: mouse
|
||||
anchors.fill: parent
|
||||
hoverEnabled: true
|
||||
acceptedButtons: Qt.NoButton
|
||||
}
|
||||
ToolTip.visible: mouse.containsMouse
|
||||
ToolTip.delay: 400
|
||||
ToolTip.text: {
|
||||
const m = thumb.modelData
|
||||
let s = String(m.name)
|
||||
if (m.w > 0 && m.h > 0)
|
||||
s += " · " + m.w + "×" + m.h
|
||||
if (m.origin && m.origin !== "user")
|
||||
s += " · " + m.origin
|
||||
return s
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+33
-2
@@ -61,7 +61,7 @@ ApplicationWindow {
|
||||
function addRow(kind, role, state) {
|
||||
transcript.append({ kind: kind, role: role, text: "", html: "", complete: false,
|
||||
name: "", callId: "", state: state, detail: "", output: "",
|
||||
exitCode: 0 })
|
||||
exitCode: 0, images: "" })
|
||||
}
|
||||
|
||||
// The engine streams one kind at a time: close whatever was open before.
|
||||
@@ -112,7 +112,7 @@ ApplicationWindow {
|
||||
function submit(text) {
|
||||
transcript.append({ kind: "message", role: "user", text: text, html: "", complete: true,
|
||||
name: "", callId: "", state: "", detail: "", output: "",
|
||||
exitCode: 0 })
|
||||
exitCode: 0, images: "" })
|
||||
agent.send(text)
|
||||
}
|
||||
|
||||
@@ -545,6 +545,29 @@ ApplicationWindow {
|
||||
win.statusText = ""
|
||||
}
|
||||
|
||||
/* The images the prompt just carried (AgentCore::promptImages): the
|
||||
* composer has appended the user row, so they attach to it. */
|
||||
function enginePromptImages(images) {
|
||||
if (!images.length)
|
||||
return
|
||||
for (let i = transcript.count - 1; i >= 0; i--) {
|
||||
const r = transcript.get(i)
|
||||
if (r.kind === "message" && r.role === "user") {
|
||||
transcript.setProperty(i, "images", images)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Images a tool produced, on the card of the call that made them. */
|
||||
function engineToolImages(callId, images) {
|
||||
if (!images.length)
|
||||
return
|
||||
const i = win.findTool(callId, "")
|
||||
if (i >= 0)
|
||||
transcript.setProperty(i, "images", images)
|
||||
}
|
||||
|
||||
/* For --uitest: one transcript row, or null. */
|
||||
function rowAt(i) {
|
||||
return (i >= 0 && i < transcript.count) ? transcript.get(i) : null
|
||||
@@ -584,6 +607,14 @@ ApplicationWindow {
|
||||
win.engineToolResult(id, name, exitCode, output)
|
||||
}
|
||||
|
||||
function onToolImages(callId, images) {
|
||||
win.engineToolImages(callId, images)
|
||||
}
|
||||
|
||||
function onPromptImages(images) {
|
||||
win.enginePromptImages(images)
|
||||
}
|
||||
|
||||
function onPermissionRequested(tool, detail, preview) {
|
||||
consent.tool = tool
|
||||
consent.detail = detail
|
||||
|
||||
@@ -11,6 +11,7 @@ Rectangle {
|
||||
property string text: ""
|
||||
property string html: ""
|
||||
property bool complete: true
|
||||
property string images: ""
|
||||
|
||||
radius: Theme.radius
|
||||
color: role === "user" ? Theme.surfaceAlt : Theme.surface
|
||||
@@ -105,5 +106,12 @@ Rectangle {
|
||||
wrapMode: Text.Wrap
|
||||
onLinkActivated: function (link) { Qt.openUrlExternally(link) }
|
||||
}
|
||||
|
||||
// What the prompt carried: pictures from the store, not their names.
|
||||
ImageStrip {
|
||||
Layout.fillWidth: true
|
||||
visible: card.images.length > 0
|
||||
text: card.images
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ Rectangle {
|
||||
property int exitCode: 0
|
||||
property bool hasResult: false
|
||||
property bool expanded: false
|
||||
property string images: ""
|
||||
|
||||
readonly property bool running: state === "pending" || state === "ready" || state === "running"
|
||||
readonly property color stateColor: state === "denied" ? Theme.err
|
||||
@@ -125,6 +126,14 @@ Rectangle {
|
||||
}
|
||||
}
|
||||
|
||||
// What the tool made (view_image, an MCP image): shown as it is, with
|
||||
// the output left folded -- the picture is the result.
|
||||
ImageStrip {
|
||||
Layout.fillWidth: true
|
||||
visible: card.images.length > 0
|
||||
text: card.images
|
||||
}
|
||||
|
||||
// Folded view: the first line of the result, so a done card still says
|
||||
// something without being opened.
|
||||
Text {
|
||||
|
||||
Reference in New Issue
Block a user