Streaming API

Multi-threaded, FILE*-based compression and decompression. More...

Functions
int64_t	zxc_stream_compress (FILE *f_in, FILE *f_out, const zxc_compress_opts_t *opts)
	Compresses one file stream into another.
int64_t	zxc_stream_decompress (FILE *f_in, FILE *f_out, const zxc_decompress_opts_t *opts)
	Decompresses one file stream into another.
int64_t	zxc_stream_get_decompressed_size (FILE *f_in)
	Reads the original size from a ZXC file's footers, without decoding.
int	zxc_stream_get_frame_info (FILE *f_in, zxc_frame_info_t *info, size_t info_size)
	Reads a file's frame header and footer, without decoding.
int	zxc_stream_get_last_frame_info (FILE *f_in, uint64_t end, zxc_frame_info_t *info, size_t info_size)
	Reads the frame that ends at offset end of a file, without decoding.
zxc_seekable *	zxc_seekable_open_file (FILE *f)
	Opens a seekable archive from a FILE*.

Detailed Description

Multi-threaded, FILE*-based compression and decompression.

Function Documentation

zxc_seekable_open_file()

zxc_seekable * zxc_seekable_open_file

(

FILE *

f

)

Opens a seekable archive from a FILE*.

Builds a zxc_reader_t doing thread-safe positional reads (pread on POSIX, ReadFile + OVERLAPPED on Windows) on f's descriptor, then delegates to zxc_seekable_open_reader. The current file position is saved and restored, and f must stay open for the lifetime of the handle.

It lives here with the other FILE* entry points rather than in zxc_seekable.h, which stays freestanding (kernel-includable).

Parameters

[in]

f

File opened in "rb" mode (must be seekable, not a pipe).

Returns

Handle on success (free with zxc_seekable_free), or NULL on error.

Opens a seekable archive from a FILE*.

Public API; full contract in zxc_stream.h. Snapshots and restores the file position, measures the file, wraps it in a thread-safe positioned reader (pread on POSIX, ReadFile + OVERLAPPED on Windows), and delegates to zxc_seekable_open_reader. The reader context is heap-owned and handed to the returned handle via zxc_seekable_attach_owned_ctx, so zxc_seekable_free releases it.

Definition at line 1546 of file zxc_driver.c.

References zxc_reader_t::read_at, ZXC_FREE, ZXC_MALLOC, zxc_seekable_attach_owned_ctx(), zxc_seekable_open_reader(), and zxc_stdio_read_at().

zxc_stream_compress()

int64_t zxc_stream_compress	(	FILE *	f_in,
		FILE *	f_out,
		const zxc_compress_opts_t *	opts
	)

Compresses one file stream into another.

Runs the multi-threaded pipeline: a reader thread pulls chunks from f_in, worker threads compress them in parallel (LZ77 + bitpacking), and a writer thread reorders and writes them to f_out.

Parameters

[in]	f_in	Input stream, opened in "rb" mode.
[out]	f_out	Output stream, opened in "wb" mode.
[in]	opts	Compression options, or NULL for defaults.

Returns

Total compressed bytes written, or a negative zxc_error_t (e.g. ZXC_ERROR_IO).

Note

f_out is flushed before returning, so a write error that stdio buffering would otherwise defer (e.g. disk full) is reported as ZXC_ERROR_IO. f_out is left open: still check the result of your own fclose(), which can fail on its own (e.g. on NFS).

Compresses one file stream into another.

Public API; full contract in zxc_stream.h. Resolves the options (threads, level, block size, checksums, seekable, dictionary) with their defaults, then runs the engine over the input.

Definition at line 1226 of file zxc_driver.c.

References zxc_compress_opts_t::checksum_enabled, zxc_compress_opts_t::dict, zxc_compress_opts_t::n_threads, zxc_compress_opts_t::progress_cb, zxc_compress_opts_t::seekable, zxc_compress_opts_t::user_data, ZXC_BLOCK_SIZE_DEFAULT, zxc_dict_id(), ZXC_DICT_SIZE_MAX, ZXC_ERROR_BAD_BLOCK_SIZE, ZXC_ERROR_DICT_TOO_LARGE, ZXC_ERROR_IO, ZXC_ERROR_MEMORY, ZXC_ERROR_NULL_INPUT, ZXC_FILE_HEADER_SIZE, zxc_file_span(), ZXC_FREE, ZXC_LEVEL_DEFAULT, ZXC_MALLOC, ZXC_OK, ZXC_OPTS_BLOCK_SIZE, ZXC_OPTS_DICT_HUF, ZXC_OPTS_DICT_SIZE, ZXC_OPTS_LEVEL, zxc_stream_engine_start(), zxc_stream_engine_stop(), zxc_stream_finish_compress(), zxc_stream_read_loop(), zxc_validate_block_size(), and zxc_write_file_header().

zxc_stream_decompress()

int64_t zxc_stream_decompress	(	FILE *	f_in,
		FILE *	f_out,
		const zxc_decompress_opts_t *	opts
	)

Decompresses one file stream into another.

Same pipeline as compression. After a footer comes the end of the input, another frame or ZXC_ERROR_CORRUPT_DATA.

Parameters

[in]	f_in	Input stream, opened in "rb" mode.
[out]	f_out	Output stream, opened in "wb" mode.
[in]	opts	Decompression options, or NULL for defaults.

Returns

Total decompressed bytes written, or a negative zxc_error_t (e.g. ZXC_ERROR_BAD_HEADER).

Note

f_out is flushed before returning; see zxc_stream_compress.

A regular file is read ahead in batches; a pipe or socket is decoded block by block, so a live stream's output is never held back.

Decompresses one file stream into another.

Public API; see zxc_stream.h. Before each frame, 4 bytes: the end of the input, a magic word or an error. Block size and level come from each frame header, not from opts. Consecutive frames with the same block size and checksum flag share one engine.

Definition at line 1311 of file zxc_driver.c.

References zxc_decompress_opts_t::checksum_enabled, zxc_decompress_opts_t::dict, zxc_decompress_opts_t::n_threads, zxc_decompress_opts_t::progress_cb, zxc_decompress_opts_t::user_data, zxc_dict_id(), ZXC_DICT_SIZE_MAX, ZXC_ERROR_BAD_MAGIC, ZXC_ERROR_CORRUPT_DATA, ZXC_ERROR_DICT_TOO_LARGE, ZXC_ERROR_IO, ZXC_ERROR_NULL_INPUT, ZXC_ERROR_SRC_TOO_SMALL, zxc_le32(), ZXC_MAGIC_WORD, ZXC_OK, ZXC_OPTS_DICT_HUF, ZXC_OPTS_DICT_SIZE, zxc_stream_decode_frame(), zxc_stream_engine_close(), zxc_stream_engine_start(), zxc_stream_read_exact(), and zxc_stream_read_frame_header().

zxc_stream_get_decompressed_size()

int64_t zxc_stream_get_decompressed_size

(

FILE *

f_in

)

Reads the original size from a ZXC file's footers, without decoding.

zxc_get_decompressed_size() on the whole file: concatenated frames add up. The file position is restored afterwards.

Parameters

[in]

f_in

Input stream, opened in "rb" mode.

Returns

Original uncompressed size in bytes, or a negative zxc_error_t code: the header's verdict (e.g. ZXC_ERROR_BAD_MAGIC, ZXC_ERROR_BAD_HEADER), ZXC_ERROR_SRC_TOO_SMALL, ZXC_ERROR_CORRUPT_DATA for an implausible size, or an I/O error.

Reads the original size from a ZXC file's footers, without decoding.

Public API; see zxc_stream.h. zxc_scan_container over the whole file, as zxc_get_decompressed_size; the stream position is restored.

Definition at line 1416 of file zxc_driver.c.

References zxc_container_info_t::dsize, ZXC_ERROR_CORRUPT_DATA, ZXC_ERROR_NULL_INPUT, zxc_file_scan_read(), zxc_file_span(), ZXC_OK, and zxc_scan_container().

zxc_stream_get_frame_info()

int zxc_stream_get_frame_info	(	FILE *	f_in,
		zxc_frame_info_t *	info,
		const size_t	info_size
	)

Reads a file's frame header and footer, without decoding.

Same checks as zxc_get_frame_info(), on the whole file from offset 0. The file position is restored afterwards.

Parameters

[in]	f_in	Input stream, opened in "rb" mode; must be seekable.
[out]	info	Filled on success, untouched otherwise.
[in]	info_size	sizeof(*info) as the caller compiled it.

Returns

ZXC_OK, or a negative zxc_error_t, ZXC_ERROR_IO included.

Public API; see zxc_stream.h.

Definition at line 1437 of file zxc_driver.c.

References ZXC_ERROR_NULL_INPUT, zxc_frame_info_copy(), ZXC_OK, and zxc_stream_read_frame_info().

zxc_stream_get_last_frame_info()

int zxc_stream_get_last_frame_info	(	FILE *	f_in,
		const uint64_t	end,
		zxc_frame_info_t *	info,
		const size_t	info_size
	)

Reads the frame that ends at offset end of a file, without decoding.

zxc_get_last_frame_info() on the first end bytes of f_in: pass the file size, then end - info->compressed_size. The file position is restored.

Parameters

[in]	f_in	Input stream, opened in "rb" mode; must be seekable.
[in]	end	Offset just past the frame; at most the file size.
[out]	info	Filled on success, untouched otherwise.
[in]	info_size	sizeof(*info) as the caller compiled it.

Returns

ZXC_OK, or a negative zxc_error_t (ZXC_ERROR_SRC_TOO_SMALL past the end of the file, ZXC_ERROR_IO included).

Public API; see zxc_stream.h.

Definition at line 1451 of file zxc_driver.c.

References ZXC_ERROR_NULL_INPUT, ZXC_ERROR_SRC_TOO_SMALL, zxc_file_scan_read(), zxc_file_span(), zxc_frame_info_copy(), ZXC_OK, and zxc_scan_frame().