Release 1.16.0

Author: jkbonfield

CHANGES

@@ -1,18 +1,19 @@
-Version 1.16.0
---------------
+Version 1.16.0 (5th May 2026)
+==============
 
-This release is not ABI compatible with 1.15.x, and loses some API
-compatibility too.  All BAM and CRAM I/O is now performed via htslib,
-which is a newly added dependency.  This change was made due to recent
-CRAM security fixes in htslib and the desire to no longer support two
-independent CRAM implementations.  However with this comes a loss of
-some functionality.
+INCOMPATIBILITY: this release is *not ABI compatible* with 1.15.x, and
+loses some API compatibility too.  All BAM and CRAM I/O is now
+performed via htslib, which is a newly added dependency.  This change
+was made due to recent CRAM security fixes in htslib and the desire to
+no longer support two independent CRAM implementations.  However with
+this comes a loss of some functionality.
 
 For SAM, BAM and CRAM support we would encourage developers to upgrade to
 using htslib natively as this is the package with significantly better
 support and is better tested.
 
-API visible changes:
+API visible changes
+-------------------
 
 * io_lib/cram_bambam.[ch] and the CRAM_IO_CUSTOM_BUFFERING definitions have
   been removed. This means the libmaus2 / biobambam2 API has been removed.
@@ -74,7 +75,8 @@ API visible changes:
   https://github.com/samtools/htslib/.
 
 
-Internal changes:
+Internal changes
+----------------
 
 * While htslib's header API was derived from the original API in io_lib, it
   does not expose all of the functions that io_lib did.  This means we have
@@ -94,15 +96,11 @@ Internal changes:
   this for some file formats.
 
 
-Benchmarks:
+Benchmarks
+----------
 
 Some benchmarks on 10 million NovaSeq alignments follow.  Values are
 real (elapsed), user CPU and system CPU times in seconds.
-Conversions to uncompressed BAM (BAMu) are demonstrative of read-only
-times, while BAM to ??? format conversions are indicative of write times
-(given a fixed cost of decoding the input BAM which can be deduced from
-BAM->BAMu).
-
 
 Reading:
                       0 threads
@@ -145,6 +143,110 @@ variable in the reading tests), SAM and BAM writing are broadly comparable
 to before (using libdeflate) while CRAM writing is significantly faster.
 
 
+External function names removed
+-------------------------------
+
+Note: most of these should have been declared as static and were
+never intended for public use.
+
+There are many more functions which are absent from this library but
+are now inherited via htslib.  However we have not validated the API
+is the same, so the functionality will exist but the code may still
+needing changing.
+
+BAM:
+    bam_close
+    bam_copy
+    bam_get_seq
+    bam_next_seq
+    bam_open
+    bam_open_block
+    bam_put_seq
+    bam_set_option
+    bam_set_voption
+    bam_write_header
+    bgzf_decode_thread
+    bgzf_encode_thread
+
+CRAM:
+    add_read_names
+    cram_allocate_encoder
+    cram_block_method2str
+    cram_deallocate_encoder
+    cram_decode_TD
+    cram_default_version
+    cram_dependent_data_series
+    cram_encoder_get_fd
+    cram_enque_compression_block
+    cram_index_load_via_callbacks
+    cram_init_varint
+    cram_io_allocate_output_buffer
+    cram_io_close
+    cram_io_deallocate_output_buffer
+    cram_io_flush_output_buffer
+    cram_io_input_buffer_fgets
+    cram_io_input_buffer_read
+    cram_io_input_buffer_seek
+    cram_io_input_buffer_underflow
+    cram_io_open
+    cram_io_open_by_callbacks
+    cram_io_openw_by_callbacks
+    cram_io_output_buffer_putc
+    cram_io_output_buffer_write
+    cram_mem_write_callback
+    cram_nop_decode_reset
+    cram_open_by_callbacks
+    cram_openw_by_callbacks
+    cram_process_work_package
+    cram_stats_dump
+    cram_stats_qual
+    expand_cache_path
+    expected_template_count
+    generate_consensus
+    int32_decode
+    int32_encode
+    int32_put
+    itf8_size
+    load_ref_portion
+    lossy_read_names
+    refs_load_fai
+    safe_itf8_put
+    safe_ltf8_put
+
+Misc:
+    mkdir_prefix
+    paranoid_fclose
+    scram_input_bam_block
+    scram_open_cram_via_callbacks
+
+Thread pool:
+    t_pool_delete_result
+    t_pool_destroy
+    t_pool_dispatch
+    t_pool_dispatch2
+    t_pool_flush
+    t_pool_init
+    t_pool_next_result
+    t_pool_next_result_wait
+    t_pool_results_queue_empty
+    t_pool_results_queue_len
+    t_pool_results_queue_sz
+    t_results_queue_destroy
+    t_results_queue_init
+
+New external functions added
+----------------------------
+
+    bam1_to_bam_seq
+    bam_seq_to_bam1
+    sam_hdr_convert
+    sam_hdr_convert_to_htslib
+    sam_hdr_nref
+    sam_hdr_ref2len
+    sam_hdr_ref2name
+
+
+
 Version 1.15.1 (14th July 2025)
 --------------
 

README.md

@@ -41,15 +41,16 @@ been too time consuming.  Note this does have some small API
 incompatibilities however.
 
 
-Version 1.16.0 (May 2026)
+Version 1.16.0 (5th May 2026)
 --------------
 
-This release is not ABI compatible with 1.15.x, and loses some API
-compatibility too.  All BAM and CRAM I/O is now performed via htslib,
-which is a newly added dependency.  This change was made due to recent
-CRAM security fixes in htslib and the desire to no longer support two
-independent CRAM implementations.  However with this comes a loss of
-some functionality.  See CHANGES for full details.
+INCOMPATIBILITY: this release is not ABI compatible with 1.15.x, and
+loses some API compatibility too.  All BAM and CRAM I/O is now
+performed via HTSlib, which is a newly added dependency.  This change
+was made due to recent CRAM security fixes in htslib and the desire to
+no longer support two independent CRAM implementations.  However with
+this comes a loss of some functionality.  See CHANGES for full
+details.
 
 Note to pass all the CRAM v4 tests this needs htslib 1.24 or above.  It
 still works without this, except for the sign of insert sizes changing
@@ -246,11 +247,11 @@ files).  The appropriate operating system package names and comands
 differ per system.  On Debian Linux derived systems use the command
 below (or build and install your own copies from source):
 
-  sudo apt-get install make zlib1g-dev libbz2-dev liblzma-dev
+  sudo apt-get install make zlib1g-dev libhts-dev libdeflate-dev
 
 On RedHat derived systems the package names differ:
 
-  sudo yum install make zlib-devel bzip2-devel xz-devel
+  sudo yum install make zlib-devel htslib-devel libdeflate-devel
 
 
 Zlib
@@ -287,10 +288,11 @@ with ./configure --disable-own-crc (or CFLAGS=-UIOLIB_CRC).
 Libdeflate
 ----------
 
-The BAM reading and writing also has optional support for the
-libdeflate library (https://github.com/ebiggers/libdeflate).  This can
-be used instead of an optimised zlib (see above), and generally is
-slightly faster.  Build using:
+Libdeflate is used by HTSlib's BAM reading and writing, but io_lib also
+optionally makes use of its CRC32 implementation.
+See https://github.com/ebiggers/libdeflate.
+
+Build using:
 
     ./configure --with-libdeflate=/path
 
@@ -391,25 +393,6 @@ with $DIST being pre-populated with already built and installed 3rd
 party dependencies, some from MSYS mentioned above.
 
 
-Libbsc
-------
-
-This is experimental, just to see what we can get with a high quality
-compression engine in CRAM.  It's hard to build right now, especially
-given it's a C++ library and our code is C.  The hacky solution now
-is (linux) e.g.:
-
-  ../configure \
-    CPPFLAGS=-I$HOME/ftp/compression/libbsc \
-    LDFLAGS="-L$HOME/ftp/compression/libbsc -fopenmp" \
-    LIBS=-lstdc++
-
-Enable it using scramble -J, but note this requires experimental CRAM
-versions 3.1 or 4.0.
-
-** Neither of these should be used for production data. **
-
-
 MacOS X
 -------
 

configure.ac

@@ -1,5 +1,5 @@
 dnl Process this file with autoconf to produce a configure script.
-AC_INIT(io_lib, 1.15.1)
+AC_INIT(io_lib, 1.16.0)
 IOLIB_VERSION=$PACKAGE_VERSION
 IOLIB_VERSION_MAJOR=`expr "$PACKAGE_VERSION" : '\([[0-9]]*\)'`
 IOLIB_VERSION_MINOR=`expr "$PACKAGE_VERSION" : '[[0-9]]*\.\([[0-9]]*\)'`
@@ -59,9 +59,9 @@ AM_MAINTAINER_MODE
 #       libstaden-read.so.1 -> libstaden-read.so.1.1.0
 #       libstaden-read.so.1.1.0
 
-VERS_CURRENT=15
-VERS_REVISION=4
-VERS_AGE=1
+VERS_CURRENT=16
+VERS_REVISION=0
+VERS_AGE=0
 AC_SUBST(VERS_CURRENT)
 AC_SUBST(VERS_REVISION)
 AC_SUBST(VERS_AGE)

progs/scramble.c

@@ -185,7 +185,7 @@ static int filter_tags(bam_seq_t *s, char *aux_filter, int keep) {
 
 static void usage(FILE *fp) {
     fprintf(fp, "  -=- sCRAMble -=-     version %s\n", IOLIB_VERSION);
-    fprintf(fp, "Author: James Bonfield, Wellcome Trust Sanger Institute. 2013-2023\n\n");
+    fprintf(fp, "Author: James Bonfield, Wellcome Trust Sanger Institute. 2013-2026\n\n");
 
     fprintf(fp, "Usage:    scramble [options] [input_file [output_file]]\n");