Design Document: AI Trend Search Engine

[kien-ship-it]

Design Document: AI Trend Search Engine

Overview

The AI Trend Search Engine is a new module within the Marketing Engine service that accepts a natural language prompt and company context, searches the web for recent news and events, and returns 5 structured results with citations. It runs as an Inngest background job and is designed as a self-contained module that can later be invoked by AI agents.

The pipeline follows four stages:

Input Validation → Query Planning (LLM) → Web Search (Tavily) → Content Synthesis (LLM) → Return

Persistence is handled by the caller (Inngest job), not the core pipeline. This keeps the module stateless and reusable.

The module lives under src/server/trend-search/ as a standalone directory, with an Inngest function in src/server/inngest/functions/ and an API route in src/app/api/trend-search/.

Architecture

sequenceDiagram
    participant Client
    participant API as API Route<br/>/api/trend-search
    participant Inngest as Inngest Job
    participant QP as Query Planner<br/>(LLM)
    participant WS as Web Search<br/>(Tavily)
    participant CS as Content Synthesizer<br/>(LLM)
    participant DB as PostgreSQL

    Client->>API: POST { query, companyContext, categories? }
    API->>API: Validate input (Zod)
    API->>Inngest: Dispatch "trend-search/run.requested"
    API-->>Client: 202 { jobId, status: "queued" }

    Inngest->>QP: Step 1: Plan queries
    QP-->>Inngest: sub-queries[]

    loop For each sub-query
        Inngest->>WS: Step 2: Execute search
        WS-->>Inngest: raw results[]
    end

    Inngest->>CS: Step 3: Synthesize results
    CS-->>Inngest: SearchResult[5]

    Inngest->>DB: Step 4: Persist results
    DB-->>Inngest: saved

    Note over Client,DB: Client polls GET /api/trend-search/[jobId] for results

Loading

The module integrates with the existing architecture:

• Services Layer: Lives within the Marketing Engine boundary
• Tools Layer: Uses Web Search (Tavily) and LLM capabilities (OpenAI via LangChain)
• Physical Layer: Persists to PostgreSQL, runs via Inngest

Components and Interfaces

1. Input Types (src/server/trend-search/types.ts)

import { z } from "zod";

export const SearchCategoryEnum = z.enum([
  "fashion",
  "finance",
  "business",
  "tech",
]);
export type SearchCategory = z.infer<typeof SearchCategoryEnum>;

export const TrendSearchInputSchema = z.object({
  query: z.string().min(1).max(1000),
  companyContext: z.string().min(1).max(2000),
  categories: z.array(SearchCategoryEnum).optional(),
});
export type TrendSearchInput = z.infer<typeof TrendSearchInputSchema>;

export interface SearchResult {
  sourceUrl: string;
  summary: string;
  description: string;
}

export interface TrendSearchOutput {
  results: SearchResult[];
  metadata: {
    query: string;
    companyContext: string;
    categories: SearchCategory[];
    createdAt: string;
  };
}

export type TrendSearchJobStatus =
  | "queued"
  | "planning"
  | "searching"
  | "synthesizing"
  | "completed"
  | "failed";

export interface TrendSearchJobRecord {
  id: string;
  companyId: bigint;
  userId: string;
  status: TrendSearchJobStatus;
  input: TrendSearchInput;
  output: TrendSearchOutput | null;
  errorMessage: string | null;
  createdAt: Date;
  completedAt: Date | null;
}

// Inngest event payload
export const TrendSearchEventDataSchema = z.object({
  jobId: z.string(),
  companyId: z.string(), // serialized as string for Inngest
  userId: z.string(),
  query: z.string(),
  companyContext: z.string(),
  categories: z.array(SearchCategoryEnum).optional(),
});
export type TrendSearchEventData = z.infer<typeof TrendSearchEventDataSchema>;

2. Query Planner (src/server/trend-search/query-planner.ts)

Responsible for taking the user's prompt and company context and generating optimized sub-queries for Tavily.

interface PlannedQuery {
  searchQuery: string;
  category: SearchCategory;
  rationale: string;
}

async function planQueries(
  query: string,
  companyContext: string,
  categories?: SearchCategory[]
): Promise<PlannedQuery[]>;

Implementation approach:

• Uses OpenAI (via LangChain ChatOpenAI) with a structured output prompt
• System prompt instructs the LLM to generate 3-5 sub-queries focused on recent news/events
• If categories are not provided, the LLM infers them from the query and company context
• Each sub-query includes category-specific terms and company-relevant keywords
• Output is parsed via Zod schema for type safety

3. Web Search Executor (src/server/trend-search/web-search.ts)

Executes sub-queries against Tavily and collects raw results.

interface RawSearchResult {
  url: string;
  title: string;
  content: string;
  score: number;
  publishedDate?: string;
}

async function executeSearch(
  subQueries: PlannedQuery[]
): Promise<RawSearchResult[]>;

Implementation approach:

• Uses @langchain/community TavilySearchResults tool or direct Tavily API
• Configures Tavily with search_depth: "advanced" and topic: "news" to focus on news/events
• Executes sub-queries sequentially (Inngest step per query for retry isolation)
• Retries each sub-query up to 2 times on failure
• Deduplicates results by URL across sub-queries
• Returns combined raw results for synthesis

4. Content Synthesizer (src/server/trend-search/synthesizer.ts)

Takes raw search results and produces exactly 5 structured results.

async function synthesizeResults(
  rawResults: RawSearchResult[],
  query: string,
  companyContext: string,
  categories: SearchCategory[]
): Promise<SearchResult[]>;

Implementation approach:

• Uses OpenAI with a structured output prompt
• System prompt instructs the LLM to:
  • Select the 5 most relevant results to the company context
  • Rank by relevance (most relevant first)
  • Generate a concise summary (1-2 sentences) and detailed description for each
  • Preserve the original source URL
• If fewer than 5 distinct results are available, pads with placeholder entries (sourceUrl: "", summary: "Insufficient results")
• Output validated via Zod schema

5. Inngest Function (src/server/inngest/functions/trendSearch.ts)

Orchestrates the pipeline as a multi-step Inngest function.

export const trendSearchJob = inngest.createFunction(
  {
    id: "trend-search-run",
    name: "AI Trend Search Pipeline",
    retries: 3,
  },
  { event: "trend-search/run.requested" },
  async ({ event, step }) => {
    // Step 1: Plan queries
    // Step 2: Execute web searches
    // Step 3: Synthesize results
    // Step 4: Persist to DB
  }
);

6. API Routes

POST /api/trend-search — Initiate a search

• Validates input with Zod
• Creates a job record in DB with status "queued"
• Dispatches Inngest event
• Returns 202 { jobId, status: "queued" }

GET /api/trend-search/[jobId] — Poll for results

• Looks up job by ID, scoped to company_id
• Returns current status and results if completed

GET /api/trend-search — List past searches

• Returns paginated list of past searches for the company

7. Module Entry Point (src/server/trend-search/index.ts)

Exposes the public interface for programmatic invocation (agent-callable in the future).

export async function runTrendSearch(
  input: TrendSearchInput
): Promise<TrendSearchOutput>;

This function runs the pipeline directly (without Inngest, without DB) for synchronous invocation by agents or any caller. It accepts only the search input and returns the result — no side effects. Persistence is the responsibility of the caller.

The Inngest function is the only caller that persists results to the DB. This keeps runTrendSearch() stateless and reusable across contexts (agents, tests, scripts) without requiring DB access.

Data Models

New Table: trend_search_jobs

CREATE TABLE trend_search_jobs (
  id VARCHAR(256) PRIMARY KEY,
  company_id BIGINT NOT NULL REFERENCES company(id) ON DELETE CASCADE,
  user_id VARCHAR(256) NOT NULL,
  status VARCHAR(50) NOT NULL DEFAULT 'queued',
  query TEXT NOT NULL,
  company_context TEXT NOT NULL,
  categories JSONB,
  results JSONB,
  error_message TEXT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
  completed_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ
);

CREATE INDEX trend_search_jobs_company_id_idx ON trend_search_jobs(company_id);
CREATE INDEX trend_search_jobs_status_idx ON trend_search_jobs(status);
CREATE INDEX trend_search_jobs_company_status_idx ON trend_search_jobs(company_id, status);

Drizzle schema definition in src/server/db/schema/trend-search.ts:

import { sql } from "drizzle-orm";
import { index, jsonb, serial, text, timestamp, varchar, bigint } from "drizzle-orm/pg-core";
import { pgTable } from "./helpers";
import { company } from "./base";
import type { SearchCategory, SearchResult } from "~/server/trend-search/types";

export const trendSearchJobs = pgTable(
  "trend_search_jobs",
  {
    id: varchar("id", { length: 256 }).primaryKey(),
    companyId: bigint("company_id", { mode: "bigint" })
      .notNull()
      .references(() => company.id, { onDelete: "cascade" }),
    userId: varchar("user_id", { length: 256 }).notNull(),
    status: varchar("status", {
      length: 50,
      enum: ["queued", "planning", "searching", "synthesizing", "completed", "failed"],
    }).notNull().default("queued"),
    query: text("query").notNull(),
    companyContext: text("company_context").notNull(),
    categories: jsonb("categories").$type<SearchCategory[]>(),
    results: jsonb("results").$type<SearchResult[]>(),
    errorMessage: text("error_message"),
    createdAt: timestamp("created_at", { withTimezone: true })
      .default(sql`CURRENT_TIMESTAMP`)
      .notNull(),
    completedAt: timestamp("completed_at", { withTimezone: true }),
    updatedAt: timestamp("updated_at", { withTimezone: true }).$onUpdate(() => new Date()),
  },
  (table) => ({
    companyIdIdx: index("trend_search_jobs_company_id_idx").on(table.companyId),
    statusIdx: index("trend_search_jobs_status_idx").on(table.status),
    companyStatusIdx: index("trend_search_jobs_company_status_idx").on(table.companyId, table.status),
  })
);

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Valid input creates a job

For any valid Search_Query (non-empty, ≤1000 chars) and valid Company_Context (non-empty, ≤2000 chars), submitting a trend search should create a job record with status "queued" and return a job ID.

Validates: Requirements 1.1

Property 2: Invalid input is rejected

For any Search_Query composed entirely of whitespace characters, or for any Company_Context exceeding 2000 characters, the Trend_Search_Engine should reject the request with a validation error and not create a job record.

Validates: Requirements 1.4, 1.5

Property 3: Category inference produces valid categories

For any valid Search_Query and Company_Context where no Search_Categories are specified, the Query_Planner should return planned queries where every category is a member of the valid SearchCategory enum (fashion, finance, business, tech).

Validates: Requirements 1.2

Property 4: Specified categories are preserved in planned queries

For any set of specified Search_Categories, all planned queries produced by the Query_Planner should only reference categories from the specified set.

Validates: Requirements 1.3

Property 5: Query planner always produces sub-queries

For any valid Search_Query and Company_Context, the Query_Planner should return at least one PlannedQuery.

Validates: Requirements 2.1

Property 6: Every sub-query triggers a search call

For any list of PlannedQueries, the web search executor should invoke the search provider exactly once per sub-query (before retries).

Validates: Requirements 3.1

Property 7: Synthesizer output structure

For any set of at least 5 raw search results, the Content_Synthesizer should produce exactly 5 SearchResult objects, each containing a non-empty sourceUrl, a non-empty summary, and a non-empty description.

Validates: Requirements 4.1, 4.2

Property 8: Source URL traceability

For any output from the Content_Synthesizer, every sourceUrl in the results should be present in the set of URLs from the raw input results.

Validates: Requirements 4.3

Property 9: Persistence round-trip

For any completed trend search, persisting the Result_Set and then retrieving it by job ID should return an equivalent Result_Set, including the original query, company context, categories, and a timestamp.

Validates: Requirements 5.1, 5.2, 5.3

Property 10: Company data isolation

For any two distinct company IDs, trend search results persisted under company A should never appear when querying results for company B.

Validates: Requirements 5.4

Property 11: Successful pipeline sets completed status

For any trend search pipeline that completes all steps without error, the job record status should be "completed" and the completedAt timestamp should be non-null.

Validates: Requirements 6.4

Property 12: Input serialization round-trip

For any valid TrendSearchInput, serializing it to the Inngest event JSON payload and then deserializing it back should produce an object equal to the original input.

Validates: Requirements 8.1, 8.2

Error Handling

Error Scenario	Handling Strategy
Empty/whitespace query	Zod validation rejects at API layer, returns 400 with error details
Company context too long	Zod validation rejects at API layer, returns 400 with error details
Tavily API unavailable	Retry up to 2 times per sub-query (Inngest step retry). Log and continue with remaining sub-queries
Tavily returns 0 results for a sub-query	Log warning, continue with other sub-queries
All sub-queries return 0 results	Synthesizer returns 5 placeholder results with empty sourceUrl and "Insufficient results" summary
LLM (Query Planner) fails	Inngest step retry (up to 3 retries). If all fail, mark job as failed
LLM (Synthesizer) fails	Inngest step retry (up to 3 retries). If all fail, mark job as failed
LLM returns malformed output	Zod validation on LLM output catches structural issues. Retry the step
DB write fails	Inngest step retry. If persistent, mark job as failed with error message
Unauthorized request	Clerk auth middleware returns 401 before reaching the handler
Job not found on poll	Return 404
Job belongs to different company	Return 404 (do not leak existence)

Testing Strategy

Property-Based Testing

Use fast-check as the property-based testing library (already compatible with the Jest setup in this project).

Each correctness property maps to a single property-based test with a minimum of 100 iterations. Tests should be tagged with the property reference.

Tag format: Feature: ai-trend-search-engine, Property {N}: {title}

Key property tests:

• Input validation properties (P1, P2): Generate random valid/invalid inputs and verify acceptance/rejection
• Query planner properties (P3, P4, P5): Mock LLM responses, generate random category combinations, verify structural constraints
• Synthesizer properties (P7, P8): Generate random raw result sets, mock LLM synthesis, verify output structure and URL traceability
• Persistence properties (P9, P10): Use test DB, generate random results, verify round-trip and isolation
• Serialization property (P12): Generate random TrendSearchInput objects, verify serialize/deserialize round-trip

Unit Testing

Unit tests complement property tests for specific examples and edge cases:

• Tavily returning 0 results for one sub-query (edge case from 3.3)
• Tavily failing and retry behavior (edge case from 3.4)
• Fewer than 5 raw results triggering placeholder padding (edge case from 4.5)
• Job failure status when pipeline step fails (edge case from 6.3)
• Zod schema validation for specific malformed inputs

Integration Testing

• End-to-end test with mocked Tavily and LLM: submit search → poll for results → verify output
• DB integration: verify Drizzle schema, migrations, and query scoping

[kien-ship-it]

Implementation Plan: AI Trend Search Engine

Overview

This plan is structured for 3 developers working independently. The work is divided into three tracks:

• Track A (Kaan): Data layer + Inngest pipeline orchestration
• Track B (Tiffany): Core AI modules (Query Planner + Content Synthesizer)
• Track C (Kien): API routes + module entry point + wiring

Each track has minimal dependencies on the others. Track A and B can start in parallel. Track C depends on types from Track A and functions from Track B, but can start with the shared types task.

Tasks

• 1. Shared types and validation schemas (any dev, do first)

  • 1.1 Create src/server/trend-search/types.ts with all shared types

    • Define SearchCategoryEnum, TrendSearchInputSchema, SearchResult, TrendSearchOutput, TrendSearchJobStatus, TrendSearchJobRecord, TrendSearchEventDataSchema and their TypeScript types
    • Define PlannedQuery interface (searchQuery, category, rationale)
    • Define RawSearchResult interface (url, title, content, score, publishedDate?)
    • Use Zod for all runtime validation schemas
    • Requirements: 1.1, 1.4, 1.5, 8.1

  • 1.2 Write property test for input serialization round-trip

    • Property 12: Input serialization round-trip
    • Generate random valid TrendSearchInput objects with fast-check, serialize to JSON, deserialize with TrendSearchEventDataSchema, verify equality
    • Validates: Requirements 8.1, 8.2

  • 1.3 Write property tests for input validation

    • Property 1: Valid input creates a job — generate random valid inputs, verify schema accepts them
    • Property 2: Invalid input is rejected — generate whitespace-only strings and strings >2000 chars, verify schema rejects them
    • Validates: Requirements 1.1, 1.4, 1.5

• 2. Track A: Data layer and Inngest pipeline (Kaan)

  • 2.1 Create Drizzle schema for trend_search_jobs table

    • Create src/server/db/schema/trend-search.ts with the trendSearchJobs table definition
    • Add company_id foreign key, status enum, JSONB columns for categories and results
    • Add indexes: company_id, status, compound company_id+status
    • Export the table and register it in src/server/db/schema.ts
    • Requirements: 5.1, 5.3

  • 2.2 Create DB helper functions for trend search jobs

    • Create src/server/trend-search/db.ts with functions: createJob, updateJobStatus, updateJobResults, getJobById, getJobsByCompanyId
    • All queries scoped by company_id
    • getJobById must verify company_id matches (return null if mismatch)
    • Requirements: 5.1, 5.2, 5.3, 5.4

  • [ ]* 2.3 Write property tests for persistence and company isolation

    • Property 9: Persistence round-trip — generate random result sets, persist via createJob + updateJobResults, retrieve via getJobById, verify equality of results, query, companyContext, categories
    • Property 10: Company data isolation — create jobs for two different company IDs, verify getJobsByCompanyId for company A never returns company B's jobs
    • Validates: Requirements 5.1, 5.2, 5.3, 5.4

  • 2.4 Create Inngest function for trend search pipeline

    • Create src/server/inngest/functions/trendSearch.ts
    • Define event type TrendSearchEvent with name "trend-search/run.requested" and data type TrendSearchEventData
    • Add event type to Events union in src/server/inngest/client.ts
    • Implement trendSearchJob Inngest function with steps:
      • Step 1 "run-pipeline": call runTrendSearch(input), update status to "searching" → "synthesizing" as it progresses
      • Step 2 "persist": call updateJobResults() with the returned TrendSearchOutput, update status to "completed"
    • runTrendSearch() owns the pipeline; Inngest owns persistence and status tracking
    • Add onFailure handler that marks job as failed with error message
    • Configure retries: 3
    • Register the function in src/app/api/inngest/route.ts
    • Requirements: 6.1, 6.2, 6.3, 6.4

  • [ ]* 2.5 Write property test for job completion status

    • Property 11: Successful pipeline sets completed status — mock all pipeline functions to succeed, run the Inngest function, verify job status is "completed" and completedAt is non-null
    • Validates: Requirements 6.4

• 3. Checkpoint - Track A and shared types

  • Ensure all tests pass, ask the user if questions arise.

• 4. Track B: Core AI modules (Tiffany)

  • 4.1 Implement Query Planner

    • Create src/server/trend-search/query-planner.ts
    • Implement planQueries(query, companyContext, categories?) function
    • Use ChatOpenAI from @langchain/openai with structured output
    • System prompt: instruct LLM to generate 3-5 sub-queries focused on recent news/events
    • When categories not provided, instruct LLM to infer from query + company context
    • When categories provided, constrain sub-queries to those categories
    • Parse LLM output with Zod schema for PlannedQuery[]
    • Requirements: 1.2, 1.3, 2.1, 2.2, 2.3, 2.4

  • [ ]* 4.2 Write property tests for Query Planner

    • Property 3: Category inference produces valid categories — mock LLM, generate random queries without categories, verify all returned categories are valid enum members
    • Property 4: Specified categories are preserved — generate random category subsets, verify planned queries only reference specified categories
    • Property 5: Query planner always produces sub-queries — generate random valid inputs, verify at least one PlannedQuery is returned
    • Validates: Requirements 1.2, 1.3, 2.1

  • 4.3 Implement Web Search Executor

    • Create src/server/trend-search/web-search.ts
    • Implement executeSearch(subQueries) function
    • Use Tavily API with search_depth: "advanced" and topic: "news"
    • Read TAVILY_API_KEY from env
    • Execute each sub-query, collect results
    • Deduplicate results by URL
    • Implement retry logic: up to 2 retries per sub-query on failure
    • On zero results for a sub-query, log and continue
    • Return combined RawSearchResult[]
    • Requirements: 3.1, 3.2, 3.3, 3.4

  • [ ]* 4.4 Write property test for search execution and unit tests for edge cases

    • Property 6: Every sub-query triggers a search call — mock Tavily, generate random PlannedQuery arrays, verify mock called once per sub-query
    • Unit test: one sub-query returns 0 results, pipeline continues (edge case 3.3)
    • Unit test: Tavily fails, retries 2 times then marks sub-query failed (edge case 3.4)
    • Validates: Requirements 3.1, 3.3, 3.4

  • 4.5 Implement Content Synthesizer

    • Create src/server/trend-search/synthesizer.ts
    • Implement synthesizeResults(rawResults, query, companyContext, categories) function
    • Use ChatOpenAI with structured output
    • System prompt: select 5 most relevant results, rank by relevance to company context, generate summary + description for each
    • Validate output with Zod schema for SearchResult[]
    • If fewer than 5 raw results, pad with placeholder entries (sourceUrl: "", summary: "Insufficient results", description: "Not enough search results were found...")
    • Requirements: 4.1, 4.2, 4.3, 4.4, 4.5

  • [ ]* 4.6 Write property tests for Content Synthesizer

    • Property 7: Synthesizer output structure — mock LLM, generate random raw result sets (≥5 items), verify output has exactly 5 results each with non-empty sourceUrl, summary, description
    • Property 8: Source URL traceability — verify every output sourceUrl exists in the input raw results URL set
    • Unit test: fewer than 5 raw results triggers placeholder padding (edge case 4.5)
    • Validates: Requirements 4.1, 4.2, 4.3, 4.5

• 5. Checkpoint - Track B core modules

  • Ensure all tests pass, ask the user if questions arise.

• 6. Track C: API routes and wiring (Kien)

  • 6.1 Create POST /api/trend-search route

    • Create src/app/api/trend-search/route.ts
    • Authenticate with Clerk (auth())
    • Validate request body with TrendSearchInputSchema
    • Look up user's company_id from the users table
    • Generate a unique job ID (uuid)
    • Create job record via createJob()
    • Dispatch Inngest event "trend-search/run.requested" with TrendSearchEventData
    • Return 202 { jobId, status: "queued" }
    • Requirements: 1.1, 1.4, 1.5, 6.1, 8.1

  • 6.2 Create GET /api/trend-search/[jobId] route

    • Create src/app/api/trend-search/[jobId]/route.ts
    • Authenticate with Clerk
    • Look up job by ID via getJobById() scoped to user's company_id
    • Return 404 if not found or company mismatch
    • Return job status, results (if completed), and metadata
    • Requirements: 5.2, 5.4

  • 6.3 Create GET /api/trend-search route for listing past searches

    • Add GET handler to src/app/api/trend-search/route.ts
    • Authenticate with Clerk
    • Query getJobsByCompanyId() with user's company_id
    • Return paginated list of past searches (id, status, query, categories, createdAt)
    • Requirements: 5.4

  • 6.4 Create module entry point for programmatic invocation

    • Create src/server/trend-search/index.ts
    • Implement runTrendSearch(input: TrendSearchInput) — accepts only search input, no companyId/userId
    • Calls planQueries() → executeSearch() → synthesizeResults() and returns TrendSearchOutput
    • No DB writes, no side effects — pure pipeline execution
    • The Inngest function is responsible for calling runTrendSearch() and then persisting the result
    • This is the interface for future agent invocation
    • Requirements: 7.1, 7.2, 7.3

  • [ ]* 6.5 Write unit tests for API routes

    • Test POST with valid input returns 202 with jobId
    • Test POST with empty query returns 400
    • Test POST with oversized company context returns 400
    • Test GET /[jobId] returns 404 for wrong company
    • Test GET /[jobId] returns results when job is completed
    • Requirements: 1.1, 1.4, 1.5, 5.2, 5.4

• 7. Final checkpoint - All tracks integrated

  • Ensure all tests pass, ask the user if questions arise.
  • Verify Inngest function is registered in the route handler
  • Verify end-to-end flow: POST → Inngest job → poll GET → results returned

Notes

• Tasks marked with * are optional and can be skipped for faster MVP
• Track A (tasks 2.x) and Track B (tasks 4.x) can be developed in parallel after task 1 is complete
• Track C (tasks 6.x) can start after task 1 is complete for route scaffolding, but needs Track A (DB helpers) and Track B (AI modules) for full wiring
• Task 1 (shared types) should be completed first by any developer and merged before parallel work begins
• Property tests use fast-check library with minimum 100 iterations per test
• Each property test references its design document property number

[kien-ship-it]

Developer Notes — AI Trend Search Engine

Everything here is specific to this codebase. Read this alongside design and tasks document

---

Authentication & Company ID Lookup

Standard pattern used across all routes. Use auth() from Clerk, then query the users table:

import { auth } from "@clerk/nextjs/server";
import { db } from "~/server/db";
import { users } from "~/server/db/schema";
import { eq } from "drizzle-orm";

const { userId } = await auth();
if (!userId) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });

const [userInfo] = await db.select().from(users).where(eq(users.userId, userId));
if (!userInfo) return NextResponse.json({ error: "Invalid user" }, { status: 400 });

const companyId = userInfo.companyId; // bigint

Reference: src/app/api/fetchCompany/route.ts

---

ID Generation

import { randomUUID } from "crypto";
const jobId = randomUUID();

Reference: src/app/api/agents/documentQ&A/AIChat/chats/route.ts

---

Drizzle Schema

All tables use the pgTable helper from src/server/db/schema/helpers.ts — it prefixes every table with pdr_ai_v2_. Never import pgTable from drizzle directly.

import { pgTable } from "./helpers"; // always this, not from drizzle-orm

After creating src/server/db/schema/trend-search.ts, register it in src/server/db/schema.ts:

export * from "./schema/trend-search";

Then run the migration:

pnpm db:push

Reference for schema structure: src/server/db/schema/agent-ai.ts

---

Inngest — Extending the Events Union

Add the new event type to src/server/inngest/client.ts alongside the existing ProcessDocumentEvent:

import type { TrendSearchEventData } from "~/server/trend-search/types";

export type TrendSearchEvent = {
  name: "trend-search/run.requested";
  data: TrendSearchEventData;
};

export type Events = ProcessDocumentEvent | TrendSearchEvent;

---

Inngest — Function Structure

Follow src/server/inngest/functions/processDocument.ts exactly:

import { inngest } from "../client";

export const trendSearchJob = inngest.createFunction(
  {
    id: "trend-search-run",
    name: "AI Trend Search Pipeline",
    retries: 3,
    onFailure: async ({ error, event }) => {
      const { jobId, companyId } = event.data;
      await updateJobStatus(jobId, BigInt(companyId), "failed", error.message);
    },
  },
  { event: "trend-search/run.requested" },
  async ({ event, step }) => {
    const { jobId, companyId, userId, query, companyContext, categories } = event.data;

    await step.run("update-status-planning", () =>
      updateJobStatus(jobId, BigInt(companyId), "planning")
    );

    const output = await step.run("run-pipeline", () =>
      runTrendSearch({ query, companyContext, categories })
    );

    await step.run("persist", async () => {
      await updateJobResults(jobId, BigInt(companyId), output);
      await updateJobStatus(jobId, BigInt(companyId), "completed");
    });
  }
);

Register it in src/app/api/inngest/route.ts:

import { trendSearchJob } from "~/server/inngest/functions/trendSearch";

const handler = serve({
  client: inngest,
  functions: [uploadDocument, trendSearchJob], // add here
});

---

Inngest — Dispatching from an API Route

Call inngest.send() directly — do not go through src/lib/jobs/ (that abstraction is for the document pipeline only):

import { inngest } from "~/server/inngest/client";

await inngest.send({
  name: "trend-search/run.requested",
  data: {
    jobId,
    companyId: companyId.toString(), // bigint must be serialized to string
    userId,
    query,
    companyContext,
    categories,
  },
});

---

Inngest — Local Development

Two terminals:

pnpm dev          # Next.js app
pnpm inngest:dev  # Inngest dev server → proxies to http://localhost:3000/api/inngest

Dashboard at http://localhost:8288 — use it to trigger events manually and inspect step output.

---

OpenAI / LangChain — Structured Output

Use gpt-4o as the model for both the Query Planner and Synthesizer. Access the API key via env.server, never process.env directly:

import { ChatOpenAI } from "@langchain/openai";
import { env } from "~/env";

const model = new ChatOpenAI({
  openAIApiKey: env.server.OPENAI_API_KEY,
  modelName: "gpt-4o",
  temperature: 0.3,
});

const structured = model.withStructuredOutput(YourZodSchema, {
  name: "your_output_name",
});

const result = await structured.invoke([{ role: "user", content: prompt }]);

Reference: src/app/api/agents/predictive-document-analysis/services/referenceExtractor.ts

---

Tavily — Direct Fetch

The project calls Tavily via fetch, not LangChain. TAVILY_API_KEY is optional in src/env.ts — guard against it being unset at the start of executeSearch() and throw early:

import { env } from "~/env";

if (!env.server.TAVILY_API_KEY) throw new Error("TAVILY_API_KEY is not configured");

const response = await fetch("https://api.tavily.com/search", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    api_key: env.server.TAVILY_API_KEY,
    query: subQuery.searchQuery,
    max_results: 5,
    search_depth: "advanced",
    topic: "news",
  }),
});

Reference: src/app/api/agents/documentQ&A/services/tavilySearch.ts

---

Status Updates — Who Owns What

runTrendSearch() is stateless and has no DB access. The Inngest function owns all status transitions:

queued → planning   (before step.run("run-pipeline"))
planning → completed (inside step.run("persist"))
any step failure → failed (onFailure handler)

The searching and synthesizing statuses defined in the schema are available if you want finer granularity, but are not required for MVP. Decide as a team before implementing — just be consistent.

---

Categories — What Gets Persisted

When the user doesn't specify categories, the Query Planner infers them. The inferred categories must be included in TrendSearchOutput.metadata.categories and persisted to the categories JSONB column. The column should never be null in a completed job.

---

Environment Variables

Always use env.server.* from ~/env, never process.env directly:

Variable	Required	Used for
OPENAI_API_KEY	Yes	LLM calls
TAVILY_API_KEY	No (validate at runtime)	Web search
INNGEST_EVENT_KEY	Yes in prod	Inngest dispatch

---

Tests

import * as fc from "fast-check"; // fast-check import

Run tests:

pnpm test

Property tests go in __tests__/api/trendSearch/ following the existing types.pbt.test.ts file already in that directory.

[kien-ship-it]

look Search alternatives: Duck duck go
search for client companies (google maps) around me

[kien-ship-it]

Initial search feature completed

┌─────────────────────────────────────────────────────────────────────────────┐
│  Running `npx tsx scripts/test-trend-search.ts`                             │
│                                                                             │
│  Required env vars:                                                         │
│   - OPENAI_API_KEY                                                          │
│   - TAVILY_API_KEY                                                          │
│                                                                             │
│  Sample output:                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

─── Input ───

{
  "query": "latest AI trends in retail marketing",
  "companyContext": "We are a mid-size fashion retailer focused on Gen Z customers in the US market.",
  "categories": ["fashion", "tech"]
}

Running pipeline (plan → search → synthesize)…

⏳ stage: searching
⏳ stage: synthesizing

─── Output ───

{
  "results": [
    {
      "sourceUrl": "https://www.businessoffashion.com/articles/technology/fashion-retail-synthetic-consumer-research/",
      "summary": "AI-generated focus groups are transforming consumer research in fashion retail.",
      "description": "This article discusses how AI-generated focus groups are being utilized by fashion retailers to gain insights into consumer preferences. For a mid-size fashion retailer targeting Gen Z, understanding consumer sentiment through innovative AI methods can enhance marketing strategies and product offerings."
    },
    {
      "sourceUrl": "https://www.businessoffashion.com/opinions/technology/opinion-online-shopping-could-be-ais-next-victim/",
      "summary": "AI-assisted shopping tools are reshaping online retail experiences.",
      "description": "The introduction of AI tools like OpenAI's Instant Checkout is revolutionizing how consumers shop online, allowing for a seamless purchasing experience. This trend is particularly relevant for fashion retailers aiming to attract Gen Z customers who value convenience and personalization in their shopping experiences."
    },
    {
      "sourceUrl": "https://www.retailtouchpoints.com/features/executive-viewpoints/ai-isnt-the-end-of-in-store",
      "summary": "AI enhances in-store shopping experiences through personalization.",
      "description": "This article highlights how fashion retailers are using AI to create personalized in-store experiences, such as virtual try-ons. For a fashion retailer focused on Gen Z, leveraging AI to enhance the shopping experience can drive engagement and sales."
    },
    {
      "sourceUrl": "https://www.consultancy.eu/news/amp/13240/ecommerce-braces-for-new-era-of-agentic-ai-shopping",
      "summary": "The rise of agentic AI is changing how products are marketed and sold online.",
      "description": "As AI becomes more integral to e-commerce, brands must adapt their marketing strategies to ensure their products are easily understood by AI systems. This trend is crucial for fashion retailers looking to maintain visibility and appeal to tech-savvy Gen Z consumers."
    },
    {
      "sourceUrl": "https://www.businessoffashion.com/briefings/sustainability/fashion-searches-for-a-new-climate-solution-tapestry-carbon-capture-textile-recycling-eu-ban/",
      "summary": "Sustainability in fashion is increasingly driven by technological advancements.",
      "description": "This article discusses the intersection of technology and sustainability in fashion, which is becoming a significant concern for Gen Z consumers. As a fashion retailer, aligning marketing strategies with sustainable practices can enhance brand loyalty among this demographic."
    }
  ],
  "metadata": {
    "query": "latest AI trends in retail marketing",
    "companyContext": "We are a mid-size fashion retailer focused on Gen Z customers in the US market.",
    "categories": ["fashion", "tech"],
    "createdAt": "2026-02-23T08:41:22.609Z"
  }
}