Skip to content

Modularized channels and added a TUI channel as example#172

Open
dagelf wants to merge 3 commits intoTinyAGI:mainfrom
dagelf:tui-only
Open

Modularized channels and added a TUI channel as example#172
dagelf wants to merge 3 commits intoTinyAGI:mainfrom
dagelf:tui-only

Conversation

@dagelf
Copy link

@dagelf dagelf commented Mar 9, 2026

No description provided.

@greptile-apps
Copy link

greptile-apps bot commented Mar 9, 2026

Greptile Summary

This PR replaces the hardcoded ALL_CHANNELS bash array and scattered case statements in lib/common.sh with a JSON-manifest-driven channel registry (channels/*.json), making it trivial to add new channels without touching shell code. It demonstrates the pattern with a new TUI (stdin/stdout) channel — useful for local testing and SSH sessions — and updates the setup wizard, daemon, and type definitions to use the dynamic registry.

Key changes:

  • lib/common.sh: new load_channel_registry() and _channel_manifest_value() helpers replace all case-based channel lookups with jq reads from channels/*.json.
  • channels/: four new JSON manifests (discord, telegram, whatsapp, tui) act as the single source of truth for each channel's ID, display name, script path, alias, and token metadata.
  • src/channels/tui-client.ts: new TUI client using readline, file-based queue polling (400 ms), optional pairing, and /help /as /whoami commands.
  • lib/setup-wizard.sh: sources common.sh and calls load_channel_registry instead of duplicating channel definitions; however, the CHANNEL_CONFIG_JSON generation loop iterates ALL_CHANNELS instead of ENABLED_CHANNELS, writing empty-credential stubs for every manifest on disk into settings.json, not just channels the user chose to enable.
  • src/lib/types.ts: Settings.channels now uses a generic ChannelConfig index signature instead of per-channel typed fields.
  • Orphaned outgoing response files from previous TUI sessions are never deleted in checkOutgoing, causing them to be re-read on every 400 ms poll indefinitely.

Confidence Score: 3/5

  • Safe to merge after fixing the ALL_CHANNELS→ENABLED_CHANNELS loop in setup-wizard.sh, which silently writes empty-credential stubs for non-enabled channels into settings.json.
  • The core channel modularization is clean and well-structured. The one logic bug (CHANNEL_CONFIG_JSON iterating ALL_CHANNELS) produces incorrect settings.json output during setup and should be fixed before merging. The TUI client issues (orphaned file accumulation, double tui_ prefix) are style-level and won't break functionality.
  • lib/setup-wizard.sh (line 335: ALL_CHANNELS → ENABLED_CHANNELS) and src/channels/tui-client.ts (orphaned outgoing file cleanup)

Important Files Changed

Filename Overview
lib/common.sh Replaced hardcoded channel arrays with JSON manifest-driven load_channel_registry() and per-channel _channel_manifest_value() helpers. Adds jq calls for every lookup; adds channel_token_prompt/help accessors. load_settings correctly resets token parallel-arrays before repopulating. Solid refactor overall.
lib/setup-wizard.sh Sources common.sh and calls load_channel_registry instead of maintaining a duplicate channel list. However, CHANNEL_CONFIG_JSON (line 335) iterates ALL_CHANNELS rather than ENABLED_CHANNELS, writing empty-credential entries for every discovered manifest into settings.json, not just the ones the user enabled.
src/channels/tui-client.ts New stdin/stdout channel client with readline prompt, 400 ms outgoing-queue polling, pairing support, and /help /as /whoami commands. Two style issues: double tui_ prefix on queue filenames, and orphaned outgoing response files from previous sessions are never deleted, causing perpetual I/O on each poll cycle.
lib/daemon.sh Adapted to use channel_script()/channel_display() from manifests; adds a guard against channels with missing script entries; properly declares loop variables as local. Comment-only removals elsewhere. Clean changes.
channels/discord.json New channel manifest for Discord; correctly declares id, display_name, alias, script, and token fields.
channels/tui.json New minimal manifest for the TUI channel; no token field required, consistent with other token-free channels like whatsapp.
src/lib/types.ts Replaces the three hardcoded channel sub-types with a generic ChannelConfig interface and an index signature, allowing any channel ID in the settings type. Clean generalization.
tinyclaw.sh Adds an explicit load_channel_registry() call after sourcing all libs and updates the new-channel instructions to reference the manifest file instead of common.sh. Straightforward.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    subgraph Boot["Boot sequence (tinyclaw.sh / setup-wizard.sh)"]
        A[tinyclaw.sh starts] --> B[source lib/common.sh]
        B --> C[load_channel_registry\nreads channels/*.json via jq]
        C --> D{channels found?}
        D -- no --> E[exit 1]
        D -- yes --> F[ALL_CHANNELS array populated]
        F --> G[load_settings\nreads settings.json]
        G --> H[ACTIVE_CHANNELS populated\ntokens loaded per manifest key]
    end

    subgraph Manifest["Channel manifest resolution"]
        M1[_channel_manifest_path\nchannel_id] --> M2{id.json exists?}
        M2 -- yes --> M3[return direct path]
        M2 -- no --> M4[scan all *.json\ncheck .id field]
        M4 --> M3
        M3 --> M5[_channel_manifest_value\njq query on file]
    end

    subgraph TUI["TUI client lifecycle"]
        T1[node tui-client.js] --> T2[readline prompt on stdin]
        T2 --> T3{user input}
        T3 -- /command --> T4[handle locally]
        T3 -- message --> T5{REQUIRE_PAIRING?}
        T5 -- yes → not approved --> T6[print pairing code]
        T5 -- no / approved --> T7[enqueueMessage\nQUEUE_INCOMING/tui_*.json]
        T7 --> T8[setInterval 400ms\ncheckOutgoing]
        T8 --> T9{response file\nQUEUE_OUTGOING/tui_*.json?}
        T9 -- matched messageId --> T10[printResponse\ndelete file]
        T9 -- timeout 60s --> T11[printResponse timeout]
    end
Loading

Last reviewed commit: c1a2e5f

Comment on lines +137 to +138
if (!data || data.channel !== CHANNEL_ID) continue;
if (!pending.has(data.messageId)) continue;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orphaned outgoing response files are never cleaned up

Files that don't satisfy data.channel !== CHANNEL_ID or !pending.has(data.messageId) are silently skipped without being deleted. This means:

  1. Stale response files from a previous TUI session — responses that arrived after the previous session exited will be re-read and re-parsed on every 400 ms poll indefinitely.
  2. Responses produced for a different TUI instance (a different senderId) — those files will never be consumed or removed either.

Over time, these accumulate in QUEUE_OUTGOING and add perpetual unnecessary I/O. Consider deleting files that pass the channel check but have no matching pending entry, since they clearly belong to a past session:

if (!data || data.channel !== CHANNEL_ID) continue;
if (!pending.has(data.messageId)) {
    // Orphaned response from a previous session — clean it up
    try { fs.unlinkSync(fullPath); } catch { /* ignore */ }
    continue;
}

Comment on lines +64 to +67
function makeMessageId(): string {
messageCounter += 1;
return `tui_${senderId}_${Date.now()}_${messageCounter}`;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double tui_ prefix in message IDs and queue filenames

makeMessageId returns `tui_${senderId}_${Date.now()}_${messageCounter}`, and enqueueMessage then names the file `tui_${messageId}.json`. The result is a filename like tui_tui_local_1234_1741478400000_1.json — the tui_ prefix is duplicated.

Either remove the tui_ prefix from makeMessageId:

Suggested change
function makeMessageId(): string {
messageCounter += 1;
return `tui_${senderId}_${Date.now()}_${messageCounter}`;
}
function makeMessageId(): string {
messageCounter += 1;
return `${senderId}_${Date.now()}_${messageCounter}`;
}

Or remove it from the filename construction in enqueueMessage (tui_${messageId}.json${messageId}.json). Keeping them consistent avoids confusion when reading the queue directory.

Comment on lines +335 to +343
for ch in "${ALL_CHANNELS[@]}"; do
token_key="$(channel_token_key "$ch")"
if [ -n "$token_key" ]; then
token_value="$(_get_token "$ch")"
CHANNEL_CONFIG_JSON="$CHANNEL_CONFIG_JSON\"${ch}\": { \"${token_key}\": \"${token_value}\" },"
else
CHANNEL_CONFIG_JSON="$CHANNEL_CONFIG_JSON\"${ch}\": {},"
fi
done
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config written for all channels, not just enabled ones

The loop iterates over ALL_CHANNELS (every discovered manifest in channels/) rather than ENABLED_CHANNELS (what the user actually selected). For any channel present on disk but not enabled by the user, _get_token returns an empty string — so the config entry written to settings.json will have a blank credential value. This clutters the settings file with stubs for channels the user never configured.

The loop variable on line 335 should be changed from "${ALL_CHANNELS[@]}" to "${ENABLED_CHANNELS[@]}" so that only channels the user opted into receive a config block.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c1a2e5fd1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SCRIPT_DIR="$PROJECT_ROOT"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore executable bit on setup wizard

This script is invoked directly (e.g. tinyclaw setup and the auto-setup path in daemon startup), so it must remain executable; in this commit lib/setup-wizard.sh was changed from mode 100755 to 100644, which causes a Permission denied failure when users try to run setup on a normal shell. That blocks first-run configuration and reconfiguration flows.

Useful? React with 👍 / 👎.

Comment on lines +78 to +79
const queueFile = path.join(QUEUE_INCOMING, `tui_${messageId}.json`);
fs.writeFileSync(queueFile, JSON.stringify(payload, null, 2));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route TUI traffic through the API queue

The new TUI client writes requests to queue/incoming JSON files and polls queue/outgoing files, but the current queue processor path is DB/API-based (POST /api/message in src/server/routes/messages.ts and /api/responses/pending in src/server/routes/queue.ts), with no reader for those filesystem queue files. In practice, enabling the TUI channel lets users type messages that never get processed or answered.

Useful? React with 👍 / 👎.

@jlia0
Copy link
Collaborator

jlia0 commented Mar 9, 2026

thanks for making this modular, would this make adding a channel like voice trivial?

@dagelf
Copy link
Author

dagelf commented Mar 10, 2026

I haven't looked at voice APIs but pretty sure it is part of the solution. There are a few minor regressions here, I will try to catch up with the codebase on the weekend, I just cherry picked the channel config from the previous refactor that also turned inference agents into config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants