binary
binary copied to clipboard
smee
Conditionals and complex bit handling
Is there currently a way to use "conditional" fields? My use case is for the BACnet protocol, which is somewhat complex. Many fields are included in the spec that only show up if a previous field matches a certain value (or some other more complicated condition is met). Or sometimes, based on an earlier condition, the parsing rules for further segments will change...
I'm not sure how to do that with this library... maybe I'm missing something? I'm investigating header further to see if it can do everything I need, and if I find out I'll close this issue.
Er, I think I can go off of select-codec from https://gist.github.com/stathissideris/8801295
... closing
Er, then again, maybe not... that seems to be read-only.
I guess, could you give me an example that includes read and write, and has codec switching / selection logic?
I'm also not sure how to cleanly handle a case where I have a byte that consists of 6 boolean bits and then 2 bits that combine to make an enum. Any suggestions?
For question 2: This sounds like a job for compile-codec. You could use a :ubyte codec which reads/writes 8bit values, and then define two functions that convert a byte to the object you are refering to (containing 6 flags and an enum) as well as the inverse function that turns such an object into a byte. The you could use (compile-codec :ubyte byte->foo foo->byte). For example, refer to the implementation of the codec enum that uses exactly this pattern.
For question 1: Yes, something employing the header codec is what I would use. The main idea is capturing the pattern: Use something in the stream to determine how the next thing should be parsed. The example you referred to looks interesting. The missing implementation for write-data needs to be added, but that seems straightforward (add a second function that implements the inverse of select-codec). I'm not familar with BACnet, but if you have a specific part you have trouble with I can try to have a look at it.
Also, maybe something like the ID3v2 parser for mp3 tags might be useful as a reference. There is a rather large, hetereogeneous and growing number of ID3 tags, so maybe that usecase is similar to yours?
As for my second question (on complex bit handling), I've written a codec to generically handle this sort of case, like so:
Config:
[:contains-apdu nil :dest-present nil :source-present :reply-expected
{:bits 2
:name :priority
:enum-map {:life-safety 2r11
:critical-equipment 2r10
:urgent 2r01
:normal 2r00}}]
Input: 0x25
Output:
{:contains-apdu false,
:dest-present true,
:source-present false,
:reply-expected true,
:priority :urgent}
Ordering of bits vs enums doesn't matter. If no enum-map is given, it'll just return the sum numeric value of the bits.
Does that sound like something that could belong in binary-dsl? If so, would you like to take a look at a pull request and give me some feedback?
And as for the codec selection in the header, do you think select-codec from the example I referenced looks like another candidate for inclusion in your library? (Assuming write-data was implemented). It seems like a pretty general-purpose solution to a common problem.
I solved all of my problems! I have BACnet encoding and decoding working now.
I ended up implementing an additional generic codec which I think may be useful to others. Sometime soon I'll write up a gist so you can review and see if it's worth incorporating.
I'm also thinking about how to streamline the whole spec into pure EDN, which may also prove useful generally.
That is great to hear! Will you publish the BACnet parser in the future? I would love to link to it as a more complex example.
Regarding your questions: I'm trying to keep the set of predefined codecs as small as possible. Your bit-codec points to a gap in our provided functionality: Codecs are primarily byte-based. Handling bits is not this libraries strongest suit. Maybe in the future we could support generic bit streams, then something like you suggested might fit right in. Also, something like Facebooks Gorilla compression scheme could be expressed, which is not possible right now.
A pure data representation of codecs might be useful. I didn't have the need yet. Some codecs like header which use custom functions might be hard to represent. It would still need a mixed representation (some functions in a namespace and references in the EDN).
So I really really want to publish my BACnet parser as open source, but as I wrote it for my employer, that may or may not happen. I'm going to keep pushing for it.
And yeah I've pretty much discarded the idea of pure-EDN specs. It's working quite well as is.