Overview

What is bpack?

The bpack Python package provides tools to describe and encode/decode binary data.

Binary data are assumed to be organized in records, each composed by a sequence of fields. Fields are characterized by a known size, offset (w.r.t. the beginning of the record) and datatype.

The package provides classes and functions that can be used to:

  • describe binary data structures in a declarative way (structures can be specified up to the bit level)

  • automatically generate encoders/decoders for a specified data descriptor

Encoders/decoders (backends) rely on well known Python packages like:

Features

  • declarative description of binary data structures

  • specification of data structures up to bit level

  • automatic codec generation from data descriptors

  • decoding (from binary data to Python objects)

  • encoding (from Python objects to binary data)

  • backend:

  • support for signed/unsigned integer types

  • support for enum.Enum types

  • support for sequence types, i.e. fields with multiple (homogeneous) items

  • both bit and byte order can be specified by the user

  • automatic size determination for some data types

  • record nesting (the field in a record descriptor can be another record)

  • possibility to specify data types using the special type annotation class bpack.typing.T that accepts annotations and string specifiers compatible with the numpy “Array Interface” and dtype

  • comprehensive test suite

Limitations

  • only fixed size binary records are supported by design, the size of the record shall be known at the moment of the record descriptor definition. It should be easy for the user to leverage tools provided by the bpack Python package to support more complex decoding scenarios.

  • currently it is assumed that all fields in a binary record share the same bit/byte order. The management of different byte order in the same binary record is, in principle, possible but not planned at the moment.

  • sequence types can only contain basic numeric types; nested sequences, sequences of enums or sequences of records are not allowed at the moment.

  • record nesting is only possible for records having the same base-units, bits or bytes, and compatible decoder types eventually.

  • currently the bpack.np has a limited (incomplete) support to record nesting and encoding capabilities.

Possible additional features still not implemented

  • user defined converters

  • support for complex and datetime data types

  • conversion to CSV, HDF5 and numpy.dtype()