v4.5.2

Deeplake v4.5.2 Release Notes

• Support binary data in dataset and column metadata.
• Remove dependency on libatomic

v4.5.1

Deeplake v4.5.1 Release Notes

Performance

• mimalloc allocator — Switched core allocations to mimalloc for lower overhead
• simdjson log parsing — Faster deeplog deserialization via simdjson
• LZ4 compressed deeplog logs — Reduced on-disk deeplog size
• String optimizations and SIMD-accelerated code paths
• Reduced nd::array sizeof — Smaller per-array memory footprint
• In-place downloads — Reduced memory copies in clustered index path
• Async prefetcher priority fix

Features

• Null value support — Full null handling in arrays, serialization, and log/replay
• Column statistics — Statistics gathering for datasets
• Link content passthrough — Pass content directly to link types
• Binary catalog datasets — Table-catalog-type dataset creation

Bug Fixes

• BM25 & inverted index — Fixed unindexed search and IN operator queries
• exact_text search — Corrected exact text search
• Fork safety — Fixed deadlock in async queues after fork()
• Credentials refresh timing — Fixed refresh timing bug
• Region detection delay — Fixed slow region auto-detection
• Link[bytes] update — Corrected update path
• Sanitizer fixes — Multiple UBSan/TSan/ASan issues resolved

v4.5.0

Pr review (#3121)

* add claude pr review

* edits

* edits

v4.4.5

• Improved Deeplake support for NULL
• Fixed issue with simultaneous row and column creation

v4.4.4

• Deeplake API now has cmake and pkg-config files for better integration
• Improved storage access performance
• Revisited PG ingestion. Improved batch ingestion

v4.4.3

First C++ library release.

v4.4.1 🌈

Release Notes

🎉 New Features & Enhancements

Storage & API Improvements

• Storage list_dirs API - New API endpoint for listing directories by @khustup2
• Added mesh type support - Extended data type capabilities by @khustup2
• PLY visualization and local file listing - Enhanced node server with visualization tools by @azat-manukyan
• Simple visualizer - New visualization component for data inspection by @khustup2
• Credentials key property - Added property to expose creds key for read-only datasets by @activesoull

Data Management

• Efficient version-locked read-only view - Implemented optimized read-only dataset views by @ashotshakhkyan
• Library preloading - Improved performance with preloaded library support by @vahan-activeloop

⚡ Performance Improvements

• 30% LRU cache insertion runtime improvement - Significant speedup in cache operations by @ashotshakhkyan
• Cache clearing on dataset deletion - Minor performance optimization on cleanup by @ashotshakhkyan
• Refactored async_prefetcher - Simplified batch prefetch logic for better performance by @ashotshakhkyan

🐛 Bug Fixes

Core Functionality

• Fixed progress reporting of rebuild_branch - Corrected progress tracking during branch rebuilds by @khustup2
• Fixed deeplake behavior - Resolved critical deeplake module issues by @activesoull
• Fixed JS build - Resolved JavaScript compilation errors by @ashotshakhkyan

Data Handling

• Summary fix - Corrected summary generation logic by @activesoull
• MP3 header magic bytes - Fixed audio file format detection by @azat-manukyan

Indexing & Storage

• Corrected indexing mode of compactor - Fixed compactor indexing behavior by @ashotshakhkyan
• Duplicate index data handling - Resolved race condition in concurrent index builders by @ashotshakhkyan
• Fixed warning - Addressed compiler warnings by @activesoull

🏗️ Architecture & Refactoring

Code Organization

• Simplified rebuild_branch logic - Eliminated temporary branch creation for cleaner code by @khustup2
• Reduced public headers - Hidden implementation details from public API by @ashotshakhkyan
• Deeplog include cleanup - Organized include statements for better maintainability by @ashotshakhkyan

API Improvements

• Added merge commit message - Enhanced commit metadata by @activesoull

v4.3.4 🌈

🎉 New Features
pg_deeplake Enhancements

CHAR/BPCHAR data type support - pg_deeplake now supports CHAR and BPCHAR data types
Library preloading - Improved performance with preloaded libraries in pg_deeplake
Auto table reload - Tables are now automatically reloaded if not found in storage

Architecture Improvements

Query core module - Separated query engine from execution module for better modularity
Group by execution refactor - Separated group by execution logic for future enhancements

🐛 Bug Fixes

Fixed linked audio type data handling - Resolved issues with linked audio type data
Fixed compilation errors - Resolved compile errors in pg_deeplake

💬 User Experience

Better error messaging - Added clear error messages for missing credentials key

🧹 Code Quality

Cleanup and refactoring - Improved code organization with proper renames and cleanup

🧪 Testing

Enhanced test coverage - Added comprehensive tests for pg_deeplake

v3.9.52 🌈

🧭 What's Changed

• Allow numpy v2. (#3075) @khustup2

⚙️ Who Contributes

@activesoull and @khustup2

v4.4.0

Release Notes

New Features

Indexing Improvements

• Support for JSON data indexing - Added support for indexing JSON data types with proper handling and query optimization
• Generic column index specification - New ability to specify index type on generic columns for more granular control
• Progress bar for index creation - Improved user experience with visual progress indicator during index creation operations
• Index handling unification - Streamlined and unified index handling across the codebase for better consistency and maintainability

Documentation

• Added missing documentation sections - Expanded documentation coverage for previously undocumented features and APIs

Improvements

Performance & Reliability

• Fixed build errors on main branch - Resolved compilation issues affecting the main development branch
• Windows test fixes - Enhanced test reliability on Windows platform with targeted fixes
• Thread-safe cancellation - Improved cancellation mechanism with proper thread safety guarantees
• Rebuild branch internal API - Refactored internal API for better architecture and maintainability
• Fix rebuild indexes during compaction - Corrected index rebuilding behavior during database compaction operations

Platform & Build Support

• Windows Python build support - Added comprehensive support for building Python bindings on Windows
• Updated libtiff dependency - Upgraded libtiff library to latest version for improved compatibility and security

Bug Fixes

• Fixed browser crash - Resolved critical issue causing browser crashes in specific scenarios
• Invalid iterator fix - Fixed invalid iterator handling that could cause runtime errors
• General bug fixes - Various stability improvements and edge case handling

Contributors

Thanks to @activesoull, @vahan-activeloop, @khustup2, and @ashotshakhkyan for their contributions to this release.