Add Python as a supported programming language

[ThibFrgsGmz]

I would be interested in using the serialization protocol source code in my GSE backend implemented in Python (Flask, Streamlit etc...).

Is there any work planned to generate the source code in Python or to autogenerate the python bindings?

[maxime-esa]

There are Python bindings for ASN1SCC developed in 3rd party software and integrated in the TASTE virtual machine (download from https://taste.tools)

The core bindings are generated by the DMT toolset (git clone https://gitrepos.estec.esa.int/taste/dmt ; cd dmt; make install). You may check this article that gives an example to use it: https://www.thanassis.space/asn1.html

The simplest way to generate the Python bindings is to run:

$ asn2dataModel -toPython fileA.asn fileA.acn

The bindings allow to create instances of the ASN.1 types using a Python ctypes interface. You can read and change the values using a simple Get/Set API, and call the uPER and ACN encoding and decoding functions which are implemented in C.

   x = MyType()
   x.field_a.Set(10)
   x.field_b[0].Set (DV.MyType_blue)  # set an enumerated value
   buf = DataStream (DV.MyType_REQUIRED_BYTES_FOR_ENCODING)   # create a buffer to encode x
   x.Encode (buf)
   ...

DMT's asn2dataModel tool can also generate SQL schemas and a full interface to SQL databases (SQLite, Postgres, etc) through the SQLAlchemy Python library.

The ctypes binding is a low-level API and has some limitations (or rather inconvenience) if you want to use it directly ; it was designed to be wrapped into higher level API or to be used in combination with automatic code generators.

For example, if you want to set a single field of a a variable with an instance of another (ctype) variable, you cannot do that. The Get/Set API only works with raw values (integers, boolean, enumerated, and array elements) - not with structured types.

Higher level APIs have been developed in TASTE to overcome this limitation, in the following tools: opengeode (https://github.com/esa/opengeode) and asn1_value_editor (https://gitrepos.estec.esa.int/taste/asn1_value_editor).

These tools allow to set values using the ASN.1 Value Notation - making it easier than calling the low-level _Set() functions for each field one by one.

The asn1_value_editor tool also allows to get graphical editors to set or display the ASN.1 values:

$ cat foo.asn
FOO DEFINITIONS ::=
BEGIN
  MyType ::= SEQUENCE (SIZE(1..10)) OF
      fruit ENUMERATED {apple, orange, banana}
END

$ asn1-value-editor -a foo.asn -t MyType -d "{apple, banana}"

This will open a standalone graphical editor:

This editor exists in the form of a Python widget that can be used in a larger application.

As you can see there are many possibilities to combine the use of asn1scc and Python as all the essential building bricks are there. Let me know if you have any additional or specific questions.

[ThibFrgsGmz]

OK thanks, this sounds promising!

Are there any plans to integrate this python binding functionality into the tool?
This would allow for a fluid user experience and a better standalone tool.

For now I'm more interested in using the tool in standalone mode than integrating other external dependencies. 😊

[maxime-esa]

We don't have a short term plan to move that to the asn1scc repo. It is made of various contributions that have their own dependencies and that cannot be easily isolated.

Alternatively there was an earlier attempt at making a native Python backend from within asn1scc itself - check this PR : https://github.com/ttsiodras/asn1scc/pull/138
I do not know the status of this work. You may be interested in contributing on this?

[ThibFrgsGmz]

Sure, I might be interested in contributing, but I don't like developing in Windows / Visual Studio - and I don't know F#, which discourages me a lot.

[maxime-esa]

The backends of ASN1SCC are based on a templating engine (StringTemplate, https://www.stringtemplate.org/). This means that all the code that is generated (data types in C, encoding and decoding functions) are independent from the parser itself (that is in F#).
If you want to change the layout of the generated code you have to change a template only, and you do not need to recompile ASN1SCC or even look at its code.

To illustrate, you can check this file: https://github.com/ttsiodras/asn1scc/blob/master/StgC/header_c.stg

Inside, you will find for example the template used to generate the code for a SEQUENCE OF type:

Define_new_sequence_of(td/*:FE_SizeableTypeDefinition*/, nMin, nMax, bFixedSize, sChildType, soChildDefintion) ::= <<

<soChildDefintion>

typedef struct {
<if(!bFixedSize)>
    int nCount; 

<endif>    
    <sChildType> arr[<nMax>];
} <td.typeName>;
>>

The first line is the input provided by the parser, here nMin and nMax are the size constraint of your array. You can see that this will emit a typedef struct including the nCount field if the array has a variable size, and the array of size nMax.

All templates are stored in these .stg files.

Creating native Python backends would mostly be about creating a new set of templates that instead of generating C code generate Python code.
This is what was started in this commit: https://github.com/ttsiodras/asn1scc/pull/125
and more specifically here: https://github.com/lewyg/asn1scc/tree/py_lang/StgPython

The main reason this was not merged yet is because even if 95% of the work is about providing the templates, the remaining (and missing) 5% are about adding the support in the tool itself for other languages than C and Ada. The CLI provides the -c and -Ada options only and it looks for templates in hardcoded folders for these languages. @usr3-1415 has started adapting the F# code to be backend-independent, but this is not completed yet.

At the moment I do not know the status of the Python templates - they were done a few years ago and will certainly need to be aligned with the latest updates of ASN1SCC. But if you want to help, that would be what has to be done - no F#, no Windows, just work on the StringTemplate files.