pycrate v0.6 releases: Python library to ease the development of encoders and decoders

Pycrate is a French word for qualifying bad wine. The present software library has nothing to do with bad wine, it is simply a Python library for manipulating various digital formats in an easy way. It is the glorious successor of libmich, which was started 8 years ago and served well.

Components

Pycrate is actually more a software suite than a single library. It is composed of several subdirectories, each providing specific services.

pycrate_core

The core of the library.

• utils provides basics functions to manipulate integers, bytes and bits
• charpy provides the Charpy class to handle easily the consumption of a bit-stream
• elt and base are providing several classes to help when building complex data structures
• repr provides simple functions to help with the representation of instances from the elt and base modules

Some of the most useful features are provided by the pack_val() functions from the utils module and the Charpy class from the charpy module. They help to deal easily with packing and unpacking bytes and integers (signed / unsigned, little / big endian) in an aligned and unaligned way. All lengths of fields are provided in bits, hence facilitating the handling of unaligned structures.

pycrate_ether

The modules provided here implement Ethernet and IP-oriented protocols and formats.

• Ethernet with structures for Ethernet and VLAN headers
• ARP simply providing the structure for ARP
• IP with structures for IPv4, IPv6, ICMP, UDP and TCP
• PCAP with structures for the PCAP global header and the record header

pycrate_media

The modules here implement various multimedia formats.

• JPEG with detailed structures used in the JPEG format
• GIF with detailed structures used in the GIF format
• TIFF with detailed structures used in the TIFF format
• BMP with structures used in the BMP format
• PNG with the basic structure used in the PNG format
• MPEG4 with the basic structure used in the MPEG4 file format
• MP3 with detailed structures used in the MP3 format, including ID3v1 and ID3v2 tags

Most of the classes here implement a complete recipe to parse all of those formats in a single shot, by using their from_char()method.

pycrate_asn1c

All the modules here serve the sole purpose of compiling ASN.1 specifications. The most important ones are:

• asnobj which is the almighty class when parsing any ASN.1 definition
• generator which provides two distinct generators to produce source files from the ASN.1 objects processed in Python: PycrateGenerator which produces source file to be used with the pycrate ASN.1 runtime (in pycrate_asn1rt), and JSONDepGraphGenerator which produces json file listing object dependencies (which then can be browsed dynamically thanks to D3).
• proc which is the top-level module for the compiler, it contains, for example, the compile_text() function which compiles a series of ASN.1 modules into Python objects

This compiler supports most of the ASN.1 language features, including parameterization and class objects and sets (especially useful when working with table constraints). It has, however, few restrictions, the biggest being the need for the left part of the ASN.1 assignment::= being on a single line. Also, old-school ASN.1 macros are not supported; hence, the compiler cannot parse SNMP MIBs.

pycrate_asn1dir

This subdirectory contains several ASN.1 specifications that are supported and precompiled for pycrate. Very few specifications have been changed in order to work with pycrate :

• Q.775, in which the terrible AllPackagesAS is commented out
• Q.773 and Q.775, in which the TCInvokeIdSet constraint is modified to be easier used as a set of values That’s all !

pycrate_asn1rt

This subdirectory contains the ASN.1 runtime, that is loaded and used by the ASN.1 specifications compiled with the compiler in pycrate_asn1c. It supports the PER encoding rules (aligned and not, canonical also), and the BER, CER and DER encoding rules.

pycrate_csn1

This subdirectory contains a CSN.1 to Python translator in the file trans.py, and a CSN.1 runtime in the file csnobj.py, in order to encode and decode CSN.1 structures translated to Python objects.

pycrate_csn1dir

This subdirectory contains some CSN.1 structures extracted from 3GPP specifications (in the .csn files), and translated into Python objects.

pycrate_mobile

This subdirectory implements most of the 3GPP NAS protocol formats:

• GSMTAP: gsmtap header format
• MCC_MNC: dictionnaries for MCC and MNC look-ups
• NAS: provides two functions to parse any uplink and downlink mobile NAS messages
• NASLTE: provides two functions to parse LTE uplink and downlink NAS messages
• PPP: structures for NCP and LCP protocols used for PPP connection establishment
• SCCP: structures for SCCP user-data and management messages
• SIGTRAN: structures for SIGTRAN (mostly M2PA and M3UA) messages
• TS23038: structures and routines for SMS encoding
• TS23040_SMS: structures for the SMS transport protocol
• TS23041_CBS: structures for the Cell Broadcast Service protocol
• TS24007: basic structures from the TS 24.007 specification, reused in most of the NAS protocols
• TS24008_CC : structures for call control messages from TS 24.008
• TS24008_GMM: structures for GPRS mobility management messages from TS 24.008
• TS24008_IE: structures for many information elements from TS 24.008
• TS24008_MM: structures for mobility management messages from TS 24.008
• TS24008_SM: structures for GPRS session management messages from TS 24.008
• TS24011_PPSMS: structures for the SMS point-to-point protocol
• TS24080_SS: structures for the Supplementary Services protocol, wrapping some MAP ASN.1 objects
• TS24301_EMM: structures for the EPS mobility management messages from TS 24.301
• TS24301_ESM: structures for the EPS mobility management messages from TS 24.301
• TS24301_IE: structures for many information elements from TS 24.301
• TS44018_GTTP: structure for the single GSM GTTP message from TS 44.018
• TS44018_IE: structures for many information elements from TS 44.018
• TS44018_RR: structures for the GSM and GPRS radio resources messages from TS 44.018

pycrate_corenet

This subdirectory implements a signalling server that supports an Iuh interface (including HNBAP and RUA/RANAP) for interfacing with 3G femtocells, and just enough of IuCS and IuPS stacks to support the attachment and mobility of mobiles connecting through the femtocells.

Changelog v0.6

• Dropped compatibility with Python 2. Incorporated many little fixes and improvements.
  3GPP 4G ad 5G RAN ASN.1 bumped to Rel.17.
  Large rework of the GTPv0, v1, v2, GTP-U and PFCP modules.
  NAS 5G updated to Rel.16, as part of the NAS 4G.
  Merged PRs from: mathlag35, mlg35, Jason Kuan.

Install && Use

Copyright (C) 2018 P1sec