LabelEditor for VSCode

Annotate images directly in VS Code — polygon, rectangle, line, point, and SAM AI-assisted masks — and save in the LabelMe JSON format used by most ML pipelines, or work natively in YOLO format (open a data.yaml, edit, save .txt). No extra app, no context switch.

✨ Features

Annotation Tools

Shape types: polygon, rectangle, line, point, and circle — all saved in LabelMe JSON (circle as center + edge points)
Drawing modes: 👁️ View (browse/select, default) · ⬠ Polygon · ▭ Rectangle · ⟋ Line · • Point · ◯ Circle
Unified edit mode: click a shape to move it or drag its vertices; circle handles translate (center) or resize (edge)
Context menu: right-click any shape to Edit, Rename, Hide, or Delete
Labels & descriptions: assign and edit a label per region, plus an optional description shown in the sidebar (omitted from JSON when empty)
Undo / redo with up to 50 history states

Selecting & Editing Instances

Smart overlap selection: clicking overlapping instances picks the smallest / most specific one (points and lines before filled shapes); click again in place to cycle down the stack with an N / M badge; hovering shows a dashed outline of what a click would select
Multi-select: Ctrl+Click to toggle (canvas or list), Shift+Click to range-select in the list, Ctrl+A for all, drag a box in View mode (Ctrl+Drag to add)
Clear selection: press Esc, or click empty space — on the canvas, or the blank area of the Labels / Instances panels
Batch operations: Rename, Hide/Show, or Delete the whole selection from the context menu or sidebar
Eraser: Shift+Click for a polygon eraser, Shift+Long-press+Drag for a rectangle eraser — boolean-subtracts the area from all overlapping shapes (interior cut-outs become hole-free polygons); right-click / Esc to cancel
Merge (Ctrl+G): union multi-selected overlapping polygons/rectangles (all-rectangles → bounding box; otherwise → polygon); a single undo restores the originals
Draw Over Instances (opt-in): when on, clicking inside an existing instance in a drawing mode starts a new annotation instead of selecting it; right-click still selects/deletes
Per-instance visibility toggle, in-place label editing, and delete

AI-Assisted Annotation (SAM)

Interactive segmentation with the Segment Anything Model — enter via the ✨ button or I
- Left click positive prompt · Shift+Left click negative · Left-drag box prompt · Right click undo last · Double click confirm
- Point and box prompts combine (a new box replaces the previous one); real-time mask preview
Encode mode: Full Image (default) or Local Crop — Local Crop encodes only the zoomed viewport for far better accuracy on small targets (encoded region shown as a yellow dashed box)
Encode source: Original file (default) or Adjusted View — encodes with Brightness/Contrast/CLAHE/Channel baked in, useful for low-contrast / medical / microscopy data
Output shape: Polygon (default) or Rectangle (mask reduced to its bounding box, with a WYSIWYG preview)
Supports SAM1 & SAM2 ONNX models (auto-detected); runs as a standalone Python HTTP server in a VS Code terminal and works over Remote-SSH
Requires Python with onnxruntime, opencv-python, numpy

SAM model setup

Download SAM2 ONNX models from HuggingFace
Put the encoder and decoder .onnx files in one folder (filenames containing "encoder"/"decoder"; otherwise the larger file is assumed to be the encoder)
Press I to enter SAM mode; if the service isn't running, a config modal lets you pick the model folder and Python interpreter and starts it in a terminal tab

ONNX Batch Inference

Tools menu → ONNX Batch Infer: run an ONNX segmentation model across all images or just the current one
Configurable model directory, Python interpreter, CPU/GPU, and RGB/BGR; existing annotations can skip / merge / overwrite; progress shown in the terminal
Requires Python with onnxruntime, opencv-python, numpy, tqdm (outputs LabelMe polygons, or YOLO .txt when launched from a YOLO dataset)

YOLO Format Mode

Work directly in YOLO format — right-click a YOLO data.yaml → "LabelEditor: Open as YOLO Dataset"
Resolves the dataset's path + train / val / test image directories and imports existing .txt labels (Ultralytics images/ → labels/ convention); detection lines (cls cx cy w h) load as rectangles, segmentation lines as polygons
Edits save back as .txt, choosing bbox vs segmentation per shape automatically (one file may mix both)
Drawing is scoped to 👁️ View · ✨ SAM · ⬠ Polygon · ▭ Rectangle
Label dialog matches LabelMe mode: it shows the current image's labels and your history (not the full data.yaml class list); entering a class that isn't in the yaml prompts to add it, appended at the next index and written straight back to data.yaml (its list / dict / block-sequence style is preserved, nc is bumped)

Image Browser & Navigation

Sidebar image list: every workspace image organized by folder, click to jump, current image highlighted, resizable — virtual scrolling handles 8000+ images smoothly; symbolic links to images and folders are followed (with cycle protection)
Quick search: filter the list by filename as you type
Advanced search: the sliders button in the search box opens a condition builder — add Name (substring), Name (regex), and Class (multi-select via a searchable, scrollable picker) conditions; conditions AND together while multiple classes in one condition OR. Results keep the image browser's natural order — filtering only narrows the list, it never reorders it — shown with a clearable banner. Name/regex search reads no annotation files; class search indexes sidecar JSON once (cached, with progress and cancel), so it stays fast on large datasets
Open Folder for Annotation: right-click a folder to work with just its images
Multi-panel: open several images/folders side-by-side; re-opening the same one reveals its existing panel
Zoom & pan with mouse-centered pivot; prev/next buttons and A / D (navigation stays within the active filter's results)
Copy image path: click the filename to copy the absolute path, right-click for the name only
Manual save (Ctrl+S) with an unsaved-changes warning on navigation

Display & Image Adjustment

Theme: ☀️ Light / 🌙 Dark / ◐ Auto (follows VS Code)
Brightness & Contrast sliders and RGB channel isolation (view R/G/B as grayscale), each with reset and a lock that preserves the value across images
CLAHE: brighten low-contrast images without color distortion (luminance-only), with a clip-limit slider
On-canvas class names (opt-in): draw each instance's label as a colour-matched pill at its corner, readable at any zoom
Image info popup (dimensions, file size, DPI, bit depth); border width (1–5px) and fill opacity (0–100%) controls
All adjustments affect display only — never the original file

Dataset Export — COCO / YOLO

Tools menu → Export Dataset: COCO Instances or YOLO, over all images or the current one
YOLO exports a ready-to-train Ultralytics dataset (data.yaml + images/train/ + labels/train/), auto-selecting bbox vs segmentation per shape; COCO writes annotations.json
Copy images option: bundle the images into the dataset for a self-contained copy, or leave them out (the folder structure is still created)
Classes reflect the current dataset (the YOLO class order comes from its data.yaml); reorder / rename / remove rows to control class indices (first row = 0 for YOLO, 1 for COCO)
Output directory defaults to <dataset>/export and is chosen (or created) via a save dialog; a run that writes zero annotations warns (likely a class-name mismatch) instead of silently "succeeding"; filename collisions across nested folders are auto-suffixed

SVG Export

Tools menu → Export SVG: write annotation outlines as SVG — the current image or all annotated images — into a folder you choose
The output-folder picker lets you select an existing folder or type/create a new one (missing folders are created); all-images mode writes one .svg per annotated image, mirroring the source's subfolders

Labels Panel

All label categories with live instance counts; show/hide every instance of a label at once
Click a label to select all its instances (Ctrl/Cmd-click adds the group to the current selection, or toggles it off); the row stays highlighted while every instance is selected
24 preset colors + custom hex, with per-label reset; colors persist globally

Workspace & UX

Rebindable keyboard shortcuts (Settings → Keyboard Shortcuts) with conflict detection and per-row / global reset; Ctrl+Y and Backspace stay as secondary Redo / Delete
Multi-language: English / 简体中文 (Settings → Language) — strings live in media/i18n.js, PRs for more locales welcome
In-webview notifications: status messages appear inline in the toolbar (severity-colored) instead of stacking as native popups; native dialogs are reserved for Save / Discard / Cancel prompts
Rich hover tooltips on every control (title, description, and the live keyboard shortcut)
macOS-style UI: SVG icons, blur-backdrop modals, click-outside-dismiss menus, and segmented controls
All settings persist globally across sessions

📦 Installation

From OpenVSX

Open VS Code
Go to Extensions (Ctrl+Shift+X)
Search for "LabelEditor for VSCode"
Click Install

From Source

Clone this repository
Run npm install
Run npm run compile
Press F5 to open Extension Development Host

🚀 Usage

Basic Workflow

Right-click any image file (.jpg, .jpeg, .png, .bmp) in the Explorer
Select "LabelEditor: Open Image Annotator"
Or right-click a folder and select "LabelEditor: Open Folder for Annotation" to work only with images in that folder
Switch to Polygon Mode (⬠) or Rectangle Mode (▭) as needed
For polygons: Click to add vertices, click near the first point to close
For rectangles: Click to set start point, move and click to set end point
Enter a label name
Save with Ctrl+S

YOLO Dataset Workflow

Right-click a YOLO data.yaml in the Explorer
Select "LabelEditor: Open as YOLO Dataset"
Existing .txt labels load automatically (boxes and polygons); annotate with Polygon / Rectangle / SAM
Pick a recent/current label or type a class name; a class not yet in data.yaml prompts to add it
Save with Ctrl+S — labels are written back as YOLO .txt

Labels Management

Click a label name to select all of its instances (Ctrl/Cmd-click to add the group to the current selection)
Click the color indicator to customize label colors
Click eye icon to toggle visibility for all instances of a label
Click reset icon (↻) to restore default color

Advanced Options

Click the ⚙️ icon to open advanced settings
Choose theme: Light (☀️), Dark (🌙), or Auto (◐)
Adjust brightness and contrast with sliders (display only, lock to preserve across images)
Adjust border width and fill opacity with sliders
Click ↺ on any setting to reset to default

Keyboard Shortcuts

Left Click: Add point / Select shape
Right Click: Undo last point while drawing
Ctrl+Wheel: Zoom in/out
V: Switch to View Mode
P: Switch to Polygon Mode
R: Switch to Rectangle Mode
L: Switch to Line Mode
O: Switch to Point Mode
C: Switch to Circle Mode (New in v1.0.0)
I: Switch to SAM AI Mode
Ctrl+Z (Cmd+Z on Mac): Undo last action
Ctrl+Shift+Z or Ctrl+Y: Redo action
Shift+Click: Start polygon eraser
Shift+Long-press+Drag: Start rectangle eraser
Ctrl+A (Cmd+A on Mac): Select all instances
Ctrl+G (New in v0.15.0): Merge selected polygon/rectangle instances (overlapping ones only)
Ctrl+R (New in v0.15.0): Rename the selected shape (single or batch)
Ctrl+H (New in v0.15.0): Toggle visibility of the selected shape(s)
ESC: Cancel current drawing / Cancel eraser / Clear selection
A: Previous image
D: Next image
Ctrl+S (Cmd+S on Mac): Save annotations
Delete/Backspace: Delete selected shape
Ctrl+D (in label modal, New in v0.14.1): Reveal chip shortcut badges, then press a digit (1-9, or 0 for the 10th) to commit that label

Toolbar Buttons

◀ / ▶: Navigate between images
👁️ / ⬠ / ▭ / ⟋ / • / ✨: Switch between View, Polygon, Rectangle, Line, Point, and SAM modes
⚙️: Open advanced rendering options
Save: Save current annotations

📄 Output Format

Annotations are saved in LabelMe JSON format:

```json { "version": "5.0.1", "flags": {}, "shapes": [ { "label": "person", "points": [[x1, y1], [x2, y2], ...], "group_id": null, "shape_type": "polygon", "flags": {}, "description": "optional annotation note" } ], "imagePath": "image.jpg", "imageData": null, "imageHeight": 1080, "imageWidth": 1920 } ```

Note: visible property is not saved to JSON - it's a session-only UI state. description is only included when non-empty.

🛠️ Known Limitations

This extension is still under active development. Some known limitations include:

No support for ellipse shape type yet (Circle is shipped — points: [[cx, cy], [edgeX, edgeY]], LabelMe-compatible)
No import from other formats (export to COCO and YOLO is supported via the Tools menu)
Performance may degrade with very large images (10000x10000+)
No support for video frame annotation
SAM mode requires a local Python environment with ONNX Runtime

🤖 Development

Most code in this extension was written by AI:

Gemini 3 Pro
Gemini 3.1 Pro
Claude Sonnet 4.5
Claude Opus 4.5
Claude Opus 4.6
Claude Opus 4.7 (1M context)
Claude Opus 4.8 (1M context)

Code review by AI:

GPT 5.3 Codex
GPT 5.4

Community PRs are also welcome — see CHANGELOG for per-release contributor credit.

This project serves as a demonstration of AI-assisted development capabilities.

📝 Roadmap

Planned features for future releases:

🤝 Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

📜 License

MIT License - see LICENSE file for details

🙏 Acknowledgments

Inspired by the LabelMe annotation tool
Built for the VS Code extension ecosystem
Developed primarily by AI language models, with feature contributions from the community — see "Development" above

Enjoy annotating! 🎨

Name		Name	Last commit message	Last commit date
Latest commit History 273 Commits
docs/superpowers		docs/superpowers
media		media
scripts		scripts
src		src
test		test
.gitignore		.gitignore
.vscodeignore		.vscodeignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
icon.png		icon.png
package.json		package.json
tsconfig.json		tsconfig.json
tsconfig.test.json		tsconfig.test.json

Folders and files

Latest commit

History

Repository files navigation

LabelEditor for VSCode

✨ Features

Annotation Tools

Selecting & Editing Instances

AI-Assisted Annotation (SAM)

ONNX Batch Inference

YOLO Format Mode

Image Browser & Navigation

Display & Image Adjustment

Dataset Export — COCO / YOLO

SVG Export

Labels Panel

Workspace & UX

📦 Installation

From OpenVSX

From Source

🚀 Usage

Basic Workflow

YOLO Dataset Workflow

Labels Management

Advanced Options

Keyboard Shortcuts

Toolbar Buttons

📄 Output Format

🛠️ Known Limitations

🤖 Development

📝 Roadmap

🤝 Contributing

📜 License

🙏 Acknowledgments

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages