feat(transcription): show live dictation preview while recording#238
Open
mml555 wants to merge 10 commits into
Open
feat(transcription): show live dictation preview while recording#238mml555 wants to merge 10 commits into
mml555 wants to merge 10 commits into
Conversation
Add Parakeet snapshot preview during hotkey hold with cursor or overlay display modes. Tag synthetic keystrokes so modifier-only hotkeys are not false-released during live insertion. Revert preview text on cancel or discard, reset Parakeet decoder before final pass, and cap in-memory preview buffer at ten minutes. Fixes kitlangton#237 Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds live dictation preview during recording: new dependency clients (live transcription, live text insertion), capture/controller plumbing for live PCM and snapshotting, insertion/runtime paths (AX and keystroke/pasteboard), settings/UI for cursor vs overlay, hotkey/permission gating, tests, localization, and a changeset. ChangesLive dictation preview while speaking
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f9c1461b6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+364
to
+366
| let pasteboard = NSPasteboard.general | ||
| pasteboard.clearContents() | ||
| pasteboard.setString(text, forType: .string) |
There was a problem hiding this comment.
When live preview falls back to keystroke insertion, this writes each preview/final fragment directly to NSPasteboard.general and never restores the prior contents or honors copyToClipboard == false. With the new default livePreviewDisplayMode == .cursor and copyToClipboard == false, any target that uses this fallback leaves the user's clipboard replaced by the transcript, unlike the existing pasteWithClipboard path that snapshots and restores it.
Useful? React with 👍 / 👎.
Live keystroke insertion no longer leaves transcript text on the general pasteboard when copy-to-clipboard is disabled; finalize restores the prior contents or writes the final transcript when enabled. Co-authored-by: Cursor <cursoragent@cursor.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 15
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
Hex/Features/Settings/SettingsFeature.swift (1)
420-422:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMirror the permission gate for paste-last-transcript capture.
This path enters capture mode even when accessibility/input monitoring are denied, but the monitor cannot deliver the key events needed to finish capture. That leaves the settings flow stuck and also suppresses the existing paste-last-transcript hotkey while the capture flag stays set.
Suggested fix
case .startSettingPasteLastTranscriptHotkey: + if state.hotkeyPermissionState.accessibility != .granted + || state.hotkeyPermissionState.inputMonitoring != .granted + { + return .merge( + .send(.requestAccessibility), + .send(.requestInputMonitoring) + ) + } beginCapture(.pasteLastTranscript, state: &state) return .none🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Hex/Features/Settings/SettingsFeature.swift` around lines 420 - 422, The startSettingPasteLastTranscriptHotkey branch calls beginCapture(.pasteLastTranscript, state: &state) unconditionally; mirror the existing permission gate used for other paste capture flows by first checking the same accessibility/input-monitoring flags (the same predicate used elsewhere for paste captures) and only call beginCapture when those permissions are granted; if permissions are denied, do not set the capture flag (avoid calling beginCapture) and instead return .none (or trigger the existing permission-denied UI action) so the settings flow and the existing hotkey are not left stuck.Hex/Features/Transcription/TranscriptionFeature.swift (1)
350-377:⚠️ Potential issue | 🟠 Major | ⚡ Quick winThe start sound is delayed until the preview loop is cancelled.
When
useLiveTranscriptionis true, thiswithTaskGroupdoes not finish untilrunSnapshotLivePreviewexits, sosoundEffect.play(.startRecording)runs on stop/cancel instead of at recording start. Split the preview loop into its own cancellable effect (for exampleCancelID.liveTranscription) or play the sound immediately afterrecording.startRecording(...)succeeds.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Hex/Features/Transcription/TranscriptionFeature.swift` around lines 350 - 377, The start sound is delayed because runSnapshotLivePreview in the withTaskGroup blocks completion; fix by ensuring soundEffect.play(.startRecording) runs as soon as recording.startRecording succeeds instead of after the group completes — either move soundEffect.play into the task that calls recording.startRecording (so it executes immediately after await recording.startRecording(...)) or extract runSnapshotLivePreview into its own .run cancellable effect (e.g. use a new CancelID.liveTranscription) launched concurrently from the parent effect, leaving the recording.startRecording task to complete so soundEffect.play can be called right away; update CancelID usage to include the new live transcription cancel id if you choose the separate-effect approach.
🧹 Nitpick comments (1)
HexCore/Tests/HexCoreTests/LivePreviewLogicTests.swift (1)
5-35: ⚡ Quick winAdd a regression test for AX/Cocoa range handling.
The new suite never exercises
snapshot(fullValue:selectionLocation:selectionLength:), which is the boundary where accessibilityCFRangevalues are converted into insertion state. A case with emoji/combining characters would have caught the indexing bug here, and a selected-text fallback case would help lock down repeated preview updates/cancel behavior once fixed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@HexCore/Tests/HexCoreTests/LivePreviewLogicTests.swift` around lines 5 - 35, Tests never call snapshot(fullValue:selectionLocation:selectionLength:), so add a regression test in LivePreviewLogicTests that invokes LiveTextInsertionLogic.snapshot(fullValue:selectionLocation:selectionLength:) (or the type that exposes that method) with a string containing emoji/combining characters and with a non-zero selectionLength to exercise the CFRange → insertion-state conversion; assert the produced insertion state (or resulting preview) matches expected indices and text (including correct handling of grapheme clusters), and add a second test that supplies a selected-text fallback (selectionLength > 0) and then simulates repeated preview updates/cancels to ensure the gate/preview behavior remains stable.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Hex/Clients/LiveTextInsertionClient.swift`:
- Around line 336-340: The fallback construction of CapturedTextField wipes out
the original selection by initializing LiveTextInsertionState with empty
prefix/suffix; instead capture and pass the original selection range and the
replaced text into the state (or into a new fields on LiveTextInsertionState) so
the insertion offset and any selected content are preserved for future updates
and cancel/revert. Locate the return that creates CapturedTextField (the
element/bundleID branch) and change the LiveTextInsertionState initialization to
include the original selection start/length and selected snapshot (or call a
constructor like LiveTextInsertionState(originalSelection:...,
originalText:...)), ensuring subsequent AX updates compute replacement ranges
relative to the preserved selection rather than index 0.
- Around line 161-197: revert() and resetSession() race with
keystrokeUpdateChain: ensure teardown synchronizes with the outstanding
updateTask (keystrokeUpdateChain) so a resumed pasteboard.replaceLiveText(...)
cannot mutate insertedText after the session is cleared. Fix by updating
revert() and resetSession() to await the current keystrokeUpdateChain.value (or
atomically capture and cancel+await updateTask) before clearing session state,
and also have the replacement loop in the Task check a
cancellation/closed-session flag (or the current
keystrokeUpdateGeneration/isKeystrokeUpdateInFlight) and return early so
pasteboard.replaceLiveText is not called after cancellation; reference
keystrokeUpdateChain, updateTask, keystrokeUpdateGeneration,
isKeystrokeUpdateInFlight, pendingKeystrokeText, revert(), resetSession(), and
pasteboard.replaceLiveText when applying the change.
In `@Hex/Clients/LiveTranscriptionClient.swift`:
- Around line 62-63: The AsyncStream created by LiveTranscriptionClientLive
(updateStream/updateContinuation) is long-lived and can retain buffered updates
across sessions; modify the session lifecycle so cancelSessionOnly() finishes
the current updateContinuation and subsequent session start recreates a fresh
AsyncStream/updateContinuation pair (or explicitly clear buffered state) before
returning a new observeUpdates() iterator. Concretely, update
LiveTranscriptionClientLive to: 1) expose logic to finish the existing
updateContinuation when cancelling bridgeTask/manager in cancelSessionOnly(), 2)
create a new AsyncStream/updateContinuation pair when starting a new session or
on first observeUpdates() after cancel, and 3) ensure observeUpdates() uses the
current stream instance so no stale buffered LiveTranscriptionUpdate values are
delivered. Ensure references: updateStream, updateContinuation,
LiveTranscriptionClientLive, cancelSessionOnly(), observeUpdates(), bridgeTask,
and manager are updated accordingly.
In `@Hex/Clients/PasteboardClient.swift`:
- Around line 280-315: replaceLiveText is using pasteTextAtCursor which
currently writes to NSPasteboard.general and never restores it, clobbering the
user clipboard during live preview; update the live-preview paths in
replaceLiveText (and the similar branch referenced later) to use a
clipboard-safe paste routine: either modify pasteTextAtCursor to snapshot
NSPasteboard.general contents, perform the paste, and then restore the snapshot,
or add a new method (e.g., pasteTextAtCursorWithoutAffectingClipboard) that
performs the same insertion without mutating the system pasteboard; ensure
replaceLiveText calls this safe routine for append/replaceTail/insert flows and
respects the existing copyToClipboard behavior used by the regular paste(text:)
flow.
In `@Hex/Clients/SuperFastCaptureController.swift`:
- Around line 190-231: makePreviewSnapshotURL currently performs file creation
and writes the full WAV while synchronously holding processingQueue (used by
process(_:)), which can starve capture/metering; change it so
processingQueue.sync only checks activeRecording and copies the minimal state
needed (e.g., snapshotSamples = previewSamples, compute duration and
sampleCount) then return or hand off to a background queue to create the
temporary URL, create AVAudioFile and call write(samples:to:) outside
processingQueue; keep the early guards (activeRecording and minimumDuration)
inside makePreviewSnapshotURL and use the same helper symbols (previewSamples,
processingQueue, write(samples:to:), runSnapshotLivePreview, activeRecording) to
locate where to extract state and move file I/O off the processingQueue.
In `@Hex/Clients/TranscriptionClient.swift`:
- Around line 298-304: The `waitForParakeetIdle()` function currently waits only
on the inner ASR slot by calling `parakeet.waitUntilTranscribeIdle()`, but it
should wait on the outer Parakeet pipeline instead. This is necessary because
`handleStopRecording` relies on this function before calling
`resetParakeetTranscriberState()`, and cancelled preview work can still be in
flight within `transcribePreview`'s outer pipeline during `ensureLoaded` or clip
prep. Replace the call to `waitUntilTranscribeIdle()` with a method that waits
for the entire outer Parakeet pipeline to become idle to ensure all upstream
work is fully completed before the function returns.
In `@Hex/Features/App/AppFeature.swift`:
- Around line 139-143: The handler for the settings action
.settings(.revealAppInFinder) is revealing the app's ancestor directory because
it calls deletingLastPathComponent() twice; change it to use
Bundle.main.bundleURL (the Hex.app bundle URL) directly and pass that URL to
NSWorkspace.shared.activateFileViewerSelecting so Finder selects the app bundle
itself (update the guard/optional handling around Bundle.main.bundleURL and the
call in the case for .settings(.revealAppInFinder)).
- Around line 231-237: The key-event handler is spawning a new Task per event
which breaks ordering; instead forward events serially to preserve modifier
press/release order by making the handleKeyEvent callback await the send call
instead of fire-and-forgetting a Task. Modify the keyEventMonitor.handleKeyEvent
closure so it can call await send(.settings(.keyEvent(keyEvent))) directly (or
use Task.runInline / an async callback that awaits send) and then return true,
ensuring SettingsFeature.handleCapture receives events in sequence.
In `@Hex/Features/Settings/GeneralSectionView.swift`:
- Line 85: The helper Text literal in GeneralSectionView currently adds
"Requires a Parakeet model and Super Fast Mode." which no longer matches the
Localizable.xcstrings entry; update either the source string in Text("Show live
transcription at the text cursor, or in Hex's floating overlay while recording.
Requires a Parakeet model and Super Fast Mode.") to exactly match the existing
catalog key, or update the Localizable.xcstrings entry (key at the existing
catalog line) to include the added sentence so the Text(...) string and the
catalog key are identical; locate the Text(...) usage in GeneralSectionView and
the corresponding key in Localizable.xcstrings to make the strings match
exactly.
- Around line 76-83: The Picker in GeneralSectionView that binds to
store.hexSettings.livePreviewDisplayMode and sends .setLivePreviewDisplayMode
currently has an empty label; give it an explicit accessibility label so
VoiceOver announces what the control changes—either replace the empty label
parameter with a descriptive label string (e.g., "Live preview display mode") or
add .accessibilityLabel("Live preview display mode") to the Picker; keep the
tags LivePreviewDisplayMode.cursor and LivePreviewDisplayMode.overlay and the
existing binding logic intact.
In `@Hex/Features/Settings/SettingsFeature.swift`:
- Around line 201-207: The handler currently treats a Return press as a
“modifier-only” confirmation when any modifiers are held, converting combos like
⌘↩ into key: nil; change the condition so Return only triggers the modifier-only
confirm when there are no modifiers (i.e. require updatedModifiers.isEmpty) so
that combos like .return + modifiers are handled as real hotkeys; update the
branch that checks keyEvent.key == .return (and calls applyCapturedHotKey(...)
and endCapture(...)) to only run when updatedModifiers.isEmpty so
applyCapturedHotKey will receive the actual .return key for modifier+Return
combos.
In `@Hex/Features/Transcription/TranscriptionFeature.swift`:
- Around line 528-565: runSnapshotLivePreview currently only handles file-backed
snapshots via recording.snapshotRecordingForPreview(), so when
SuperFastCaptureController.makePreviewSnapshotURL() returns nil (AVAudioRecorder
fallback exposing live PCM buffers) the loop stops producing live preview;
update runSnapshotLivePreview to handle both cases: if
recording.snapshotRecordingForPreview() returns a URL use the existing flow,
otherwise fetch the live PCM buffer from the recording fallback API (e.g., a new
or existing method like recording.previewPCMBuffer() or similar), convert or
wrap it into the form expected by transcription.transcribePreview (either by
writing a temp file before calling transcribePreview or by calling a transcribe
API that accepts raw PCM/Data), and ensure proper cleanup (only remove the temp
file if created) and the same inFlightTranscribe/defer behavior is preserved;
reference runSnapshotLivePreview, recording.snapshotRecordingForPreview(),
SuperFastCaptureController.makePreviewSnapshotURL(), and
transcription.transcribePreview to locate and implement the change.
- Around line 461-473: The current logic transcribes previewSnapshotURL first
(when useLiveTranscription is true) which can omit the end of the recording;
change the order so that capturedURL is transcribed first using
transcription.transcribe(capturedURL, model, decodeOptions) and only if that
result is empty or throws should you then attempt
transcription.transcribe(previewSnapshotURL, model, decodeOptions). Update the
control flow around useLiveTranscription, previewSnapshotURL, capturedURL and
the logging via transcriptionFeatureLogger.notice to reflect that the finalized
capture (capturedURL) is the authoritative final pass and previewSnapshotURL is
a fallback.
In `@HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift`:
- Around line 46-57: snapshot currently uses Swift String.character counts and
prefix/dropFirst which mismatch AX CFRange UTF-16 indices; update
LiveTextInsertionState.snapshot to compute selection end and slices using UTF-16
code‑unit indices (e.g. use NSString/utf16-based length and bridging to create
prefix/suffix at UTF-16 offsets), and change any other places that compute AX
CFRange offsets (LiveTextInsertionState.cursorLocation, revertedCursorLocation,
and the offsets used in LiveTextInsertionClient.apply such as insertionStart and
selection.location + state.insertedText.count) to use utf16.count or
NSString.length so all CFRange locations/lengths are computed in UTF-16 code
units rather than Swift grapheme counts.
In `@Localizable.xcstrings`:
- Around line 386-388: The new user-facing keys in Localizable.xcstrings lack
German entries; add German ("de") localization blocks for each new key (e.g. the
"Insert at cursor" key and the renamed Quit menu item) so they match the
adjacent localized entries; locate the keys in the file (the reported ranges:
around the "Insert at cursor" key and the other ranges 392-394, 485-487,
548-549, 729-730) and insert corresponding "de" translations consistent with
surrounding style and formatting.
---
Outside diff comments:
In `@Hex/Features/Settings/SettingsFeature.swift`:
- Around line 420-422: The startSettingPasteLastTranscriptHotkey branch calls
beginCapture(.pasteLastTranscript, state: &state) unconditionally; mirror the
existing permission gate used for other paste capture flows by first checking
the same accessibility/input-monitoring flags (the same predicate used elsewhere
for paste captures) and only call beginCapture when those permissions are
granted; if permissions are denied, do not set the capture flag (avoid calling
beginCapture) and instead return .none (or trigger the existing
permission-denied UI action) so the settings flow and the existing hotkey are
not left stuck.
In `@Hex/Features/Transcription/TranscriptionFeature.swift`:
- Around line 350-377: The start sound is delayed because runSnapshotLivePreview
in the withTaskGroup blocks completion; fix by ensuring
soundEffect.play(.startRecording) runs as soon as recording.startRecording
succeeds instead of after the group completes — either move soundEffect.play
into the task that calls recording.startRecording (so it executes immediately
after await recording.startRecording(...)) or extract runSnapshotLivePreview
into its own .run cancellable effect (e.g. use a new CancelID.liveTranscription)
launched concurrently from the parent effect, leaving the
recording.startRecording task to complete so soundEffect.play can be called
right away; update CancelID usage to include the new live transcription cancel
id if you choose the separate-effect approach.
---
Nitpick comments:
In `@HexCore/Tests/HexCoreTests/LivePreviewLogicTests.swift`:
- Around line 5-35: Tests never call
snapshot(fullValue:selectionLocation:selectionLength:), so add a regression test
in LivePreviewLogicTests that invokes
LiveTextInsertionLogic.snapshot(fullValue:selectionLocation:selectionLength:)
(or the type that exposes that method) with a string containing emoji/combining
characters and with a non-zero selectionLength to exercise the CFRange →
insertion-state conversion; assert the produced insertion state (or resulting
preview) matches expected indices and text (including correct handling of
grapheme clusters), and add a second test that supplies a selected-text fallback
(selectionLength > 0) and then simulates repeated preview updates/cancels to
ensure the gate/preview behavior remains stable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b99fa507-fc2d-45be-9091-07e9013f7a54
📒 Files selected for processing (21)
.changeset/403811d5.mdHex.xcodeproj/project.pbxprojHex/Clients/KeyEventMonitorClient.swiftHex/Clients/LiveTextInsertionClient.swiftHex/Clients/LiveTranscriptionClient.swiftHex/Clients/ParakeetClient.swiftHex/Clients/ParakeetClipPreparer.swiftHex/Clients/PasteboardClient.swiftHex/Clients/RecordingClient.swiftHex/Clients/SuperFastCaptureController.swiftHex/Clients/TranscriptionClient.swiftHex/Features/App/AppFeature.swiftHex/Features/Settings/GeneralSectionView.swiftHex/Features/Settings/SettingsFeature.swiftHex/Features/Transcription/TranscriptionFeature.swiftHex/Features/Transcription/TranscriptionIndicatorView.swiftHex/Utilities/SyntheticKeyboardEvent.swiftHexCore/Sources/HexCore/Logic/LiveTextInsertion.swiftHexCore/Sources/HexCore/Settings/HexSettings.swiftHexCore/Tests/HexCoreTests/LivePreviewLogicTests.swiftLocalizable.xcstrings
Comment on lines
+46
to
+57
| public static func snapshot( | ||
| fullValue: String, | ||
| selectionLocation: Int, | ||
| selectionLength: Int | ||
| ) -> LiveTextInsertionState? { | ||
| guard selectionLocation >= 0, selectionLength >= 0 else { return nil } | ||
| let endIndex = selectionLocation + selectionLength | ||
| guard selectionLocation <= fullValue.count, endIndex <= fullValue.count else { return nil } | ||
|
|
||
| let prefix = String(fullValue.prefix(selectionLocation)) | ||
| let suffix = String(fullValue.dropFirst(endIndex)) | ||
| return LiveTextInsertionState(prefix: prefix, suffix: suffix) |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the target file and inspect snapshot implementation
fd -a "LiveTextInsertion.swift" -t f . | head
fd -a "LiveTextInsertionClient.swift" -t f . | head
# Print the snapshot function region
python3 - <<'PY'
import subprocess, re, pathlib, sys, os, textwrap
def pick_file(name):
out = subprocess.check_output(["bash","-lc", f"fd -a '{name}' -t f . | head -n 1"]).decode().strip()
if not out:
raise SystemExit(f"File {name} not found")
return out
path = pick_file("LiveTextInsertion.swift")
print("=== File:", path, "===")
# show around snapshot
text = pathlib.Path(path).read_text(encoding="utf-8")
m = re.search(r"static func snapshot\\(.*?\\)\\s*->\\s*LiveTextInsertionState\\?\\s*\\{", text, re.S)
if not m:
# fallback: find 'snapshot('
idx = text.find("snapshot(")
print("snapshot found at idx", idx)
start = max(0, idx-300)
end = min(len(text), idx+800)
print(text[start:end])
else:
start = max(0, m.start()-200)
# find closing brace by counting braces from the '{'
brace_start = text.find("{", m.end()-1)
depth=0
i=brace_start
for i in range(brace_start, len(text)):
if text[i]=='{': depth+=1
elif text[i]=='}':
depth-=1
if depth==0:
end=i+1
break
else:
end= m.end()+400
print(text[start:end])
PY
# Find all call sites of snapshot and show the relevant argument derivation
rg -n "LiveTextInsertionLogic\\.snapshot\\(" -S . || true
# Inspect LiveTextInsertionClient.swift where AXSelectedTextRange is read
python3 - <<'PY'
import subprocess, pathlib, re
def pick_file(name):
out = subprocess.check_output(["bash","-lc", f"fd -a '{name}' -t f . | head -n 1"]).decode().strip()
if not out:
raise SystemExit(f"File {name} not found")
return out
path = pick_file("LiveTextInsertionClient.swift")
print("=== File:", path, "===")
text = pathlib.Path(path).read_text(encoding="utf-8")
# show occurrences of AXSelectedTextRange
for m in re.finditer(r"AXSelectedTextRange", text):
start=max(0,m.start()-250)
end=min(len(text), m.start()+400)
print("\n--- Context around AXSelectedTextRange ---\n")
print(text[start:end])
# show any place where selectionLocation/selectionLength are set or converted
for pat in ["selectionLocation", "selectionLength", "CFRange", "location", "length"]:
if pat in text:
print("\n=== First 50 lines around first occurrence of", pat, "===\n")
i=text.find(pat)
start=max(0,i-200); end=min(len(text), i+500)
print(text[start:end])
break
PYRepository: kitlangton/Hex
Length of output: 3637
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find definition of LiveTextInsertionState and any index usage expectations
rg -n "struct LiveTextInsertionState|typealias LiveTextInsertionState|LiveTextInsertionState" HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift Hex/Clients/LiveTextInsertionClient.swift
# Show LiveTextInsertionState definition area
python3 - <<'PY'
import pathlib, re, subprocess
paths = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertion.swift' -t f . | head -n 1"]).decode().strip().splitlines()
path = paths[0]
text = pathlib.Path(path).read_text(encoding='utf-8')
m = re.search(r"(struct|enum)\\s+LiveTextInsertionState\\b[\\s\\S]{0,600}", text)
print("Match:", bool(m))
if m:
start=max(0,m.start()-200)
end=min(len(text), m.start()+900)
print(text[start:end])
PY
# Inspect more of LiveTextInsertionClient around the snapshot call and cursor logic
python3 - <<'PY'
import pathlib, re, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
text = pathlib.Path(path).read_text(encoding='utf-8')
# print larger section around snapshot call at ~line 394 (as seen)
lines = text.splitlines()
start=360; end=460
for i in range(start,end):
if i < len(lines):
print(f"{i+1}: {lines[i]}")
PY
# Search for any conversions from CFRange indices to String indices
rg -n "utf16|UTF16|substring|NSString|String.Index|index\\(offsetBy:|offsetBy|count\\b|prefix\\(|dropFirst\\(" Hex/Clients/LiveTextInsertionClient.swift HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift
# Specifically see if cursorLocation and other index params are also CFRange-based
rg -n "cursorLocation" Hex/Clients/LiveTextInsertionClient.swift
# Inspect apply() around cursorLocation computations
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
text = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
# locate 'static func apply' line
for idx,line in enumerate(text):
if "static func apply(" in line:
start=idx-20
end=idx+160
for i in range(start,end):
if 0 <= i < len(text):
print(f"{i+1}: {text[i]}")
break
PYRepository: kitlangton/Hex
Length of output: 15568
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "LiveTextInsertionState" HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, re, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertion.swift' -t f . | head -n 1"]).decode().strip()
text = pathlib.Path(path).read_text(encoding='utf-8')
m = re.search(r"(public\s+)?(struct|enum)\s+LiveTextInsertionState\b", text)
print("Found state declaration at", m.start() if m else None)
if m:
start=max(0,m.start()-200)
end=min(len(text), m.start()+800)
print(text[start:end])
PY
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
start=360; end=470
for i in range(start,end):
if i < len(lines):
print(f"{i+1}: {lines[i]}")
PY
rg -n "utf16|UTF16|NSString|substring\\(|String\\.Index|index\\(offsetBy:|offsetBy:|\\.count\\b|prefix\\(|dropFirst\\(" Hex/Clients/LiveTextInsertionClient.swift HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift
rg -n "cursorLocation" Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
for idx,line in enumerate(lines):
if "static func apply(" in line:
start=idx-20; end=idx+200
for i in range(start,end):
if 0 <= i < len(lines):
print(f"{i+1}: {lines[i]}")
break
PYRepository: kitlangton/Hex
Length of output: 17874
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "LiveTextInsertionState" HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, re, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertion.swift' -t f . | head -n 1"]).decode().strip()
text = pathlib.Path(path).read_text(encoding='utf-8')
m = re.search(r"(public\s+)?(struct|enum)\s+LiveTextInsertionState\b", text)
print("Found state declaration at", m.start() if m else None)
if m:
start=max(0,m.start()-200)
end=min(len(text), m.start()+900)
print(text[start:end])
PY
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
start=360; end=470
for i in range(start,end):
if i < len(lines):
print(f"{i+1}: {lines[i]}")
PY
rg -n "utf16|UTF16|NSString|substring\\(|String\\.Index|index\\(offsetBy:|offsetBy:|\\.count\\b|prefix\\(|dropFirst\\(" Hex/Clients/LiveTextInsertionClient.swift HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift
rg -n "cursorLocation" Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
for idx,line in enumerate(lines):
if "static func apply(" in line:
start=idx-20; end=idx+200
for i in range(start,end):
if 0 <= i < len(lines):
print(f"{i+1}: {lines[i]}")
break
PYRepository: kitlangton/Hex
Length of output: 17974
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "LiveTextInsertionState" HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, re, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertion.swift' -t f . | head -n 1"]).decode().strip()
text = pathlib.Path(path).read_text(encoding='utf-8')
m = re.search(r"(public\s+)?(struct|enum)\s+LiveTextInsertionState\b", text)
print("Found state declaration at", m.start() if m else None)
if m:
start=max(0,m.start()-200)
end=min(len(text), m.start()+900)
print(text[start:end])
PY
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
start=360; end=470
for i in range(start,end):
if i < len(lines):
print(f"{i+1}: {lines[i]}")
PY
rg -n "utf16|UTF16|NSString|substring\\(|String\\.Index|index\\(offsetBy:|offsetBy:|\\.count\\b|prefix\\(|dropFirst\\(" Hex/Clients/LiveTextInsertionClient.swift HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift
rg -n "cursorLocation" Hex/Clients/LiveTextInsertionClient.swift
python3 - <<'PY'
import pathlib, subprocess
path = subprocess.check_output(["bash","-lc","fd -a 'LiveTextInsertionClient.swift' -t f . | head -n 1"]).decode().strip()
lines = pathlib.Path(path).read_text(encoding='utf-8').splitlines()
for idx,line in enumerate(lines):
if "static func apply(" in line:
start=idx-20; end=idx+200
for i in range(start,end):
if 0 <= i < len(lines):
print(f"{i+1}: {lines[i]}")
break
PYRepository: kitlangton/Hex
Length of output: 17974
🌐 Web query:
AXSelectedTextRange CFRange UTF-16 code units location length
💡 Result:
The AXSelectedTextRange accessibility attribute uses a CFRange structure to define the selection within an editable text element [1][2]. In this context, the CFRange values represent positions and lengths in terms of UTF-16 code units, not bytes [1][2]. The CFRange structure is defined in Core Foundation as [3]: typedef struct { CFIndex location; CFIndex length; } CFRange; Key details regarding this implementation include: 1. UTF-16 Code Units: Apple’s Core Foundation string APIs, which define the environment for these ranges, measure length and location based on 16-bit Unicode code units (UTF-16 code pairs) [4]. This ensures consistent indexing regardless of whether a character is represented by one or two code units (e.g., supplementary characters). 2. Usage with Accessibility API: The kAXSelectedTextRangeAttribute is typically retrieved or set using an AXValueRef of type kAXValueCFRangeType [1][5][6]. When setting the selection, you create the AXValueRef from a CFRange and apply it to the target UI element via AXUIElementSetAttributeValue [7][6]. 3. Character vs. Byte: Documentation explicitly clarifies that these ranges denote the number of characters (interpreted as UTF-16 code units) rather than raw byte counts [1][2]. 4. Accessing Values: To work with these ranges programmatically, developers use AXValueGetValue with the kAXValueCFRangeType constant to extract the CFRange structure from the AXValueRef returned by the accessibility system [5][6]. Similarly, CFRangeMake is commonly used to construct new ranges [8][3].
Citations:
- 1: https://gist.github.com/p6p/24fbac5d12891fcfffa2b53761f4343e
- 2: https://github.com/phracker/MacOSX-SDKs/blob/master/MacOSX10.7.sdk/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/Headers/AXAttributeConstants.h
- 3: https://opensource.apple.com/source/CF/CF-855.17/CFBase.h.auto.html
- 4: https://developer.apple.com/documentation/corefoundation/cfstring
- 5: https://macdevelopers.wordpress.com/2014/02/05/how-to-get-selected-text-and-its-coordinates-from-any-system-wide-application-using-accessibility-api/
- 6: https://stackoverflow.com/questions/50418321/macos-accessibility-api-change-value-of-selected-range-location
- 7: https://stackoverflow.com/questions/72562896/set-selected-text-through-the-macos-accessibility-apis-in-catalyst-apps
- 8: https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFStrings/Articles/UsingRanges.html
Fix UTF-16 index handling for AXSelectedTextRange-based insertion
AXSelectedTextRange’s CFRange.location/length are UTF-16 code-unit indices, but LiveTextInsertionLogic.snapshot uses Swift String.count / prefix(_:) / dropFirst(_:), which operate on Swift character/grapheme boundaries. This will break bounds checks and slice the wrong prefix/suffix for emoji/combining sequences. The same mismatch appears in the AX cursor/replacement math: LiveTextInsertionState.cursorLocation/revertedCursorLocation and LiveTextInsertionClient.apply(...) use prefix.count / insertedText.count as CFRange locations/lengths.
Suggested direction (for snapshot slicing)
public static func snapshot(
fullValue: String,
selectionLocation: Int,
selectionLength: Int
) -> LiveTextInsertionState? {
guard selectionLocation >= 0, selectionLength >= 0 else { return nil }
let endIndex = selectionLocation + selectionLength
- guard selectionLocation <= fullValue.count, endIndex <= fullValue.count else { return nil }
-
- let prefix = String(fullValue.prefix(selectionLocation))
- let suffix = String(fullValue.dropFirst(endIndex))
+ let nsValue = fullValue as NSString
+ guard selectionLocation <= nsValue.length, endIndex <= nsValue.length else { return nil }
+
+ let prefix = nsValue.substring(to: selectionLocation)
+ let suffix = nsValue.substring(from: endIndex)
return LiveTextInsertionState(prefix: prefix, suffix: suffix)
}Update all count-based offsets that feed AX CFRange (including cursorLocation, insertionStart, and selection.location + state.insertedText.count) to use UTF-16 code-unit counts (utf16.count / NSString.length) so the coordinate systems stay consistent.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| public static func snapshot( | |
| fullValue: String, | |
| selectionLocation: Int, | |
| selectionLength: Int | |
| ) -> LiveTextInsertionState? { | |
| guard selectionLocation >= 0, selectionLength >= 0 else { return nil } | |
| let endIndex = selectionLocation + selectionLength | |
| guard selectionLocation <= fullValue.count, endIndex <= fullValue.count else { return nil } | |
| let prefix = String(fullValue.prefix(selectionLocation)) | |
| let suffix = String(fullValue.dropFirst(endIndex)) | |
| return LiveTextInsertionState(prefix: prefix, suffix: suffix) | |
| public static func snapshot( | |
| fullValue: String, | |
| selectionLocation: Int, | |
| selectionLength: Int | |
| ) -> LiveTextInsertionState? { | |
| guard selectionLocation >= 0, selectionLength >= 0 else { return nil } | |
| let endIndex = selectionLocation + selectionLength | |
| let nsValue = fullValue as NSString | |
| guard selectionLocation <= nsValue.length, endIndex <= nsValue.length else { return nil } | |
| let prefix = nsValue.substring(to: selectionLocation) | |
| let suffix = nsValue.substring(from: endIndex) | |
| return LiveTextInsertionState(prefix: prefix, suffix: suffix) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift` around lines 46 - 57,
snapshot currently uses Swift String.character counts and prefix/dropFirst which
mismatch AX CFRange UTF-16 indices; update LiveTextInsertionState.snapshot to
compute selection end and slices using UTF-16 code‑unit indices (e.g. use
NSString/utf16-based length and bridging to create prefix/suffix at UTF-16
offsets), and change any other places that compute AX CFRange offsets
(LiveTextInsertionState.cursorLocation, revertedCursorLocation, and the offsets
used in LiveTextInsertionClient.apply such as insertionStart and
selection.location + state.insertedText.count) to use utf16.count or
NSString.length so all CFRange locations/lengths are computed in UTF-16 code
units rather than Swift grapheme counts.
5 tasks
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Hex/Clients/PasteboardClient.swift`:
- Around line 98-108: In endLiveKeystrokePasteboardSession, when
hexSettings.copyToClipboard is true the current code returns early unless
finalText is non-empty, leaving partial preview text on NSPasteboard.general;
instead, if copyToClipboard is true and finalText is nil or empty you must
restore the user's snapshot to the pasteboard before returning. Update
endLiveKeystrokePasteboardSession to: obtain NSPasteboard.general, then if
hexSettings.copyToClipboard check if let finalText, !finalText.isEmpty to
clearContents() and setString(finalText, forType: .string), else call
liveKeystrokePasteboardSession.userSnapshot?.restore(to: pasteboard); finally
ensure the existing defer that nils userSnapshot remains.
- Around line 403-406: The current hot path in pasteTextAtCursor (used by
replaceLiveText) awaits the 500ms sleep and snapshot.restore, causing
keystroke-mode latency; instead, when hexSettings.copyToClipboard && succeeded
and liveKeystrokePasteboardSession.userSnapshot is present, schedule the
delay+restore in a detached Task (or Task { ... } ) just like pasteWithClipboard
does so pasteTextAtCursor returns immediately; update the block referencing
liveKeystrokePasteboardSession.userSnapshot and snapshot.restore(to: pasteboard)
to spawn the background Task that sleeps for .milliseconds(500) then calls
snapshot.restore(to: pasteboard), rather than awaiting the sleep on the caller
path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f119d4f1-538f-480c-89f0-b2904fd33af6
📒 Files selected for processing (2)
Hex/Clients/LiveTextInsertionClient.swiftHex/Clients/PasteboardClient.swift
🚧 Files skipped from review as they are similar to previous changes (1)
- Hex/Clients/LiveTextInsertionClient.swift
Incorporate Codex and CodeRabbit review: clipboard session restore, keystroke cancel/revert races, final transcribe ordering, preview snapshot I/O off the capture queue, hotkey capture ordering, and WhatsApp-style full-replace keystroke updates. Co-authored-by: Cursor <cursoragent@cursor.com>
Keystroke insertion now always replaces the preview block instead of incremental appends, covering WhatsApp-like Electron apps without a bundle-ID allowlist. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (2)
Hex/Clients/LiveTextInsertionClient.swift (2)
95-95:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftSerialize
prepare()with the previous keystroke task.
prepare()now tears down viaresetSessionImmediately(), but that path still only cancelskeystrokeUpdateChainand clears state. If the old task is suspended inpasteboard.replaceLiveText(...), it can resume after a newprepareNow()and still writeinsertedTextor mutate the newly captured session. Reuse the awaited teardown here, or make the queued task bail out before touching shared state once its generation/session is stale.Also applies to: 182-229, 328-350
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Hex/Clients/LiveTextInsertionClient.swift` at line 95, The call to resetSessionImmediately() in prepare() is racing with an already-running keystroke task (keystrokeUpdateChain) that may be suspended inside pasteboard.replaceLiveText(...) and later resume, allowing it to write insertedText or mutate the new session; serialize prepare()/prepareNow() with that previous keystroke task by awaiting the same teardown path (reuse the awaited teardown/future used to cancel keystrokeUpdateChain) or add a stale-check at the start of the queued keystroke task so it bails if its generation/session id differs from the current one; update the logic in prepare(), prepareNow(), and the keystroke task that calls pasteboard.replaceLiveText(...) to either await the teardown future or check a sessionGeneration/sessionId before touching insertedText or session state (also apply this pattern to the other occurrences noted around the 182-229 and 328-350 regions).
426-445:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftPreserve the original selection context in this fallback.
When this path falls back to
LiveTextInsertionState(prefix: "", suffix: ""), the first AX replace may succeed, but later updates/revert compute offsets as if the preview started at index0. That can replace the wrong range in the target app and leaves cancel with no way to restore the original selected text.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Hex/Clients/LiveTextInsertionClient.swift` around lines 426 - 445, The fallback creates LiveTextInsertionState(prefix: "", suffix: "") which loses the original selection context; instead, try to read the element's full value and selected range separately (using readValue(from:) and readSelectedRange(from:)) and, if either or both are available, compute prefix = String(fullValue[..<fullValue.index(fullValue.startIndex, offsetBy: selection.location)]) and suffix = String(fullValue[fullValue.index(fullValue.startIndex, offsetBy: selection.location + selection.length)...]) and construct LiveTextInsertionState(prefix: prefix, suffix: suffix) so offsets remain correct; keep using LiveTextInsertionLogic.snapshot(...) when it succeeds, and only fall back to empty prefix/suffix if value or selection cannot be read at all, updating the CapturedTextField(state:) creation accordingly (referencing CapturedTextField, LiveTextInsertionState, readValue(from:), readSelectedRange(from:), and LiveTextInsertionLogic.snapshot).
🧹 Nitpick comments (1)
Hex/Clients/SuperFastCaptureController.swift (1)
499-500: ⚡ Quick winRelease the preview buffer between sessions.
removeAll(keepingCapacity: true)keeps the full 10-minute backing store alive after live monitoring/finish. At 16 kHz float32, that is about 38 MB retained for the lifetime of this controller even though the next session rebuilds the preview from scratch. UsingkeepingCapacity: false(or reassigning[]) on teardown would avoid that steady-state memory cost.Also applies to: 516-517
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Hex/Clients/SuperFastCaptureController.swift` around lines 499 - 500, The preview buffer is being cleared with removeAll(keepingCapacity: true) which retains the large backing store; change those calls (the one that clears previewSamples and the other at the second occurrence) to removeAll(keepingCapacity: false) or reassign previewSamples = [] so the backing storage is released between sessions; update both places where previewSamples is cleared (and leave didLogPreviewSampleCap handling unchanged).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@Hex/Clients/LiveTextInsertionClient.swift`:
- Line 95: The call to resetSessionImmediately() in prepare() is racing with an
already-running keystroke task (keystrokeUpdateChain) that may be suspended
inside pasteboard.replaceLiveText(...) and later resume, allowing it to write
insertedText or mutate the new session; serialize prepare()/prepareNow() with
that previous keystroke task by awaiting the same teardown path (reuse the
awaited teardown/future used to cancel keystrokeUpdateChain) or add a
stale-check at the start of the queued keystroke task so it bails if its
generation/session id differs from the current one; update the logic in
prepare(), prepareNow(), and the keystroke task that calls
pasteboard.replaceLiveText(...) to either await the teardown future or check a
sessionGeneration/sessionId before touching insertedText or session state (also
apply this pattern to the other occurrences noted around the 182-229 and 328-350
regions).
- Around line 426-445: The fallback creates LiveTextInsertionState(prefix: "",
suffix: "") which loses the original selection context; instead, try to read the
element's full value and selected range separately (using readValue(from:) and
readSelectedRange(from:)) and, if either or both are available, compute prefix =
String(fullValue[..<fullValue.index(fullValue.startIndex, offsetBy:
selection.location)]) and suffix =
String(fullValue[fullValue.index(fullValue.startIndex, offsetBy:
selection.location + selection.length)...]) and construct
LiveTextInsertionState(prefix: prefix, suffix: suffix) so offsets remain
correct; keep using LiveTextInsertionLogic.snapshot(...) when it succeeds, and
only fall back to empty prefix/suffix if value or selection cannot be read at
all, updating the CapturedTextField(state:) creation accordingly (referencing
CapturedTextField, LiveTextInsertionState, readValue(from:),
readSelectedRange(from:), and LiveTextInsertionLogic.snapshot).
---
Nitpick comments:
In `@Hex/Clients/SuperFastCaptureController.swift`:
- Around line 499-500: The preview buffer is being cleared with
removeAll(keepingCapacity: true) which retains the large backing store; change
those calls (the one that clears previewSamples and the other at the second
occurrence) to removeAll(keepingCapacity: false) or reassign previewSamples = []
so the backing storage is released between sessions; update both places where
previewSamples is cleared (and leave didLogPreviewSampleCap handling unchanged).
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: df44badf-3508-422d-84d6-a736c3b079c8
📒 Files selected for processing (13)
Hex/Clients/LiveTextInsertionClient.swiftHex/Clients/LiveTranscriptionClient.swiftHex/Clients/PasteboardClient.swiftHex/Clients/RecordingClient.swiftHex/Clients/SuperFastCaptureController.swiftHex/Clients/TranscriptionClient.swiftHex/Features/App/AppFeature.swiftHex/Features/Settings/GeneralSectionView.swiftHex/Features/Settings/SettingsFeature.swiftHex/Features/Transcription/TranscriptionFeature.swiftHexCore/Sources/HexCore/Logic/LiveTextInsertion.swiftHexCore/Tests/HexCoreTests/LivePreviewLogicTests.swiftLocalizable.xcstrings
✅ Files skipped from review due to trivial changes (1)
- Localizable.xcstrings
🚧 Files skipped from review as they are similar to previous changes (9)
- Hex/Features/Settings/GeneralSectionView.swift
- HexCore/Tests/HexCoreTests/LivePreviewLogicTests.swift
- Hex/Features/App/AppFeature.swift
- Hex/Clients/PasteboardClient.swift
- Hex/Clients/TranscriptionClient.swift
- HexCore/Sources/HexCore/Logic/LiveTextInsertion.swift
- Hex/Clients/LiveTranscriptionClient.swift
- Hex/Clients/RecordingClient.swift
- Hex/Features/Transcription/TranscriptionFeature.swift
Wait longer before preview transcribes, pad clips to 1s to reduce Parakeet hallucinations, reject unstable preview revisions, and finalize keystroke sessions without duplicate pastes in Electron apps. Co-authored-by: Cursor <cursoragent@cursor.com>
Advance the preview scheduler after empty results, skip preview reinitialize churn, rebuild AsrManager before final transcription with retry, and align snapshot minimum duration with 1s padding. Co-authored-by: Cursor <cursoragent@cursor.com>
Track peak speech levels during capture, skip preview and final transcribe when the mic is silent or disconnected, and filter phrases like "thank you" and "okay" that Parakeet emits on noise. Co-authored-by: Cursor <cursoragent@cursor.com>
Require both RMS and peak for speech detection, always filter short hallucinations unless audio is loud, analyze preview buffer levels instead of session peaks, and back off preview transcribe cadence after several seconds. Co-authored-by: Cursor <cursoragent@cursor.com>
Wire previewSpeechMetrics in the live dependency client and remove unnecessary await on synchronous actor checks. Co-authored-by: Cursor <cursoragent@cursor.com>
Improve whisper detection and hold-window speech activity so preview keeps updating through pauses. Speed transcribe scheduling, tune the update gate, and evaluate preview speech atomically. Prefer incremental keystroke edits in Electron/Cursor instead of full replace each pass. Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
Ready to merge from my side. Latest commit
Please merge when you have a moment — I don't have merge permissions on upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #237
Summary
SyntheticKeyboardEvent) so modifier-only hotkeys are not false-released during live insertionTest plan
cd HexCore && swift test --filter LivePreviewLogicTestspassesNotes
Show live dictation preview while recording (#237))Made with Cursor
Summary by CodeRabbit
New Features
Settings & Configuration
Improvements
Localization