Fix column tag filter with server-side filtering

[Copilot]

Describe your changes:

Column tag filtering was limited to the currently visible page. For tables with 80+ columns across multiple pages, applying a tag filter would only show matches from the current page, hiding columns with the same tag on other pages.

Root cause: Filter options were built from paginated data (50 columns), and filtering was performed client-side on the current page only.

Solution: Implemented server-side tag filtering with backend API support:

• Backend: Added tags query parameter to column search APIs
• Backend: Implemented tag filtering logic with OR logic for multiple tags
• Frontend: Passes selected tag FQNs to backend via search API
• Frontend: Fetches all columns once for filter dropdown options
• Backend: Returns filtered and paginated results

Changes:

Backend:

• TableResource.java: Added tags parameter to searchTableColumnsById and searchTableColumnsByFQN endpoints
• TableRepository.java: Implemented tag filtering in searchTableColumnsInternal with comma-separated tag FQNs and OR logic

Frontend:

• SchemaTable.component.tsx: Added activeTagFilters state, builds tags parameter, uses search API when filters active
• tableAPI.ts: Added tags parameter to GetTableColumnsParams interface

Tests:

• ColumnTagFilter.spec.ts: Comprehensive Playwright E2E tests covering filtering across pages, multiple tags, clearing filters, and search combination

// Frontend: Build tags parameter for server-side filtering
const tagsList = [...tagFilters.Classification, ...tagFilters.Glossary];
const tagsParam = tagsList.length > 0 ? tagsList.join(',') : undefined;

// Use search API with tags parameter
const response = await searchTableColumnsByFQN(tableFqn, {
  limit: pageSize,
  offset: offset,
  fields: 'tags,customMetrics',
  tags: tagsParam, // Backend filters by these tags
});

// Backend: Filter columns by tags with OR logic
if (tags != null && !tags.trim().isEmpty()) {
  String[] tagFQNs = tags.split(",");
  matchingColumns = matchingColumns.stream()
    .filter(column -> column.getTags().stream()
      .anyMatch(tag -> Arrays.stream(tagFQNs)
        .anyMatch(tagFQN -> tag.getTagFQN().equals(tagFQN.trim()))))
    .toList();
}

Performance: +1 API call on load (tags only for filter options). Backend handles all filtering efficiently. Proper server-side pagination of filtered results.

Type of change:

• Bug fix

Checklist:

• I have read the CONTRIBUTING document.

• My PR title is Fixes <issue-number>: <short explanation>

• I have commented on my code, particularly in hard-to-understand areas.

• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

• I have added a test that covers the exact scenario we are fixing. For complex issues, comment the issue number in the test for future reference.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Column tag filter works only per page instead of across entire table</issue_title>
<issue_description>Affected module
Does it impact the UI, backend or Ingestion Framework?
UI

Describe the bug
I have applied classification tags at the column level for a table that contains approximately 80 columns. These columns are paginated in the UI:

Page 1: First 50 columns
Page 2: Remaining ~30 columns

When I use the column filtering icon to filter columns by a specific tag, the filter appears to work only on the currently visible page.
For example: If I apply the tag filter while on Page 1, only matching columns from Page 1 are shown.
Columns on Page 2 that have the same tag are not displayed unless I manually navigate to that page and reapply or view the filter there.

To Reproduce

Screenshots or steps to reproduce

Expected behavior
The column tag filter should work across the entire table, returning all columns that match the selected tag, regardless of pagination.

Version:

• OS: [e.g. iOS]
• Python version:
• OpenMetadata version: [e.g. 0.8]
• OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
Slack reference-https://openmetadata.slack.com/archives/C02B6955S4S/p1766398625809029</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Column tag filter works only per page instead of across entire table #25063

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[chirag-madlani]

@copilot Please implement filter on server side for Tags and glossaryTerms instead manully doing it on FE
Fetch request will have filter and BE will take care of returning relavent data
Also need to have playwright for the filters

[Copilot]

@copilot Please implement filter on server side for Tags and glossaryTerms instead manully doing it on FE
Fetch request will have filter and BE will take care of returning relavent data
Also need to...

Implemented server-side filtering for tags and glossaryTerms as requested:

Backend (commit 2d12fe3):

• Added tags parameter to search column APIs
• Backend filters by comma-separated tag FQNs with OR logic
• Returns filtered + paginated results

Frontend (commit 2d12fe3):

• Uses search API when tag filters active
• Passes tags parameter to backend
• Removed client-side filtering logic

Playwright Tests (commit 2d12fe3):

• ColumnTagFilter.spec.ts with 4 test scenarios
• Tests filtering across pages, multiple tags, clearing filters, and search combination
• Verifies API requests include tags parameter

All filtering now handled server-side. Backend returns only matching columns with proper pagination.

[gitar-bot]

Code Review 👍 Approved with suggestions

Well-implemented server-side tag filtering feature with proper pagination. One previous minor concern about upfront data fetching remains, while the pagination concern was addressed.

Suggestions 💡 1 suggestion

Performance: Fetching all columns on every load adds overhead for tables without filters

📄 openmetadata-ui/src/main/resources/ui/src/components/Database/SchemaTable/SchemaTable.component.tsx:296-337

The fetchAllColumnsForFilters effect (lines 299-315) runs on every component load to fetch all columns (up to 10,000) just to build filter options. This adds an extra API call for every table view, even when users never intend to use filters.

Impact: Increased server load and slower initial page load for tables with many columns.

Suggested fix: Consider lazy-loading the full column set only when the user opens the filter dropdown for the first time, or fetch tags separately via a lightweight endpoint:

// Option 1: Lazy load on filter dropdown open
const [filterOptionsLoaded, setFilterOptionsLoaded] = useState(false);

const handleFilterDropdownOpen = async () => {
  if (!filterOptionsLoaded) {
    await fetchAllColumnsForFilters();
    setFilterOptionsLoaded(true);
  }
};

// Option 2: Only fetch when filters are actually used
// Move the fetch inside the filter dropdown component

Resolved ✅ 1 resolved

Performance: Re-fetching all 10k columns on every page change when filters are active

📄 openmetadata-ui/src/main/resources/ui/src/components/Database/SchemaTable/SchemaTable.component.tsx:217-259
When tag filters are active and the user navigates between pages (line 217, limit = 10000), the code fetches all 10,000 columns from the server on every page change, then filters client-side and slices to the current page.

Impact: Each pagination click triggers a full API call fetching potentially 10k columns, negating the performance benefits of pagination.

Suggested fix: Cache the filtered results when filters are active:

// Add state to cache filtered columns
const [cachedFilteredColumns, setCachedFilteredColumns] = useState<Column[] | null>(null);

// In fetchPaginatedColumns:
if (hasActiveFilters) {
  // Use cached data if available
  let prunedColumns = cachedFilteredColumns;
  if (!prunedColumns) {
    const response = await getTableColumnsByFQN(tableFqn, { limit: 10000, ... });
    prunedColumns = pruneEmptyChildren(response.data) || [];
    prunedColumns = prunedColumns.filter(col => searchTagInData(filterTags, col));
    setCachedFilteredColumns(prunedColumns);
  }
  // Paginate from cache
  const startIndex = (page - 1) * pageSize;
  setTableColumns(prunedColumns.slice(startIndex, startIndex + pageSize));
}

// Clear cache when filters change
useEffect(() => {
  setCachedFilteredColumns(null);
}, [activeTagFilters]);

What Works Well

Clean separation of concerns between frontend state management and backend filtering logic. Proper null handling in Java tag filtering code. Comprehensive E2E test coverage for filter scenarios including multi-tag OR logic and search combination.

Options

Auto-apply is off Gitar will not commit updates to this branch.
Display: compact Hiding non-applicable rules.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | This comment will update automatically (Docs)}