Serializer

[hildjj]

Starting a serializer feels like a good amount of design work that I'd want to talk to you about first.

[cabo]

For many applications, it is more work to do the integration with buffer management and traversal of application data structures than doing the actual serialization. For those, the best solution may be to simply have a "head writer", something like:

static inline void cbor_encoder_write_head(out_t* out, unsigned int ib,
uint64_t n) {
    if (n < 24) {
        ensure_writable(out, 1);
        write_1(out, ib + (int)n);
    } else if (n < 256) {
        ensure_writable(out, 2);
        write_2(out, ib + 24, n);
    } else if (n < 65536) {
        ensure_writable(out, 3);
        uint16_t be = be16(n);
        write_byte_and_data(out, ib + 25, (const void*)&be, 2);
    } else if (n < 0x100000000LU) {
        ensure_writable(out, 5);
        uint32_t be = be32(n);
        write_byte_and_data(out, ib + 26, (const void*)&be, 4);
    } else {
        ensure_writable(out, 9);
        uint64_t be = be64(n);
        write_byte_and_data(out, ib + 27, (const void*)&be, 8);
    }
}

(again, it is almost more work to handle the output buffer than to write the
actual data. be16/32/64 are byte swappers if your architecture needs them.)

So, in your application, you'd write something like

unsigned measurements[4];
...
write_head(..., IB_ARRAY, 4);
for (i = 0; i < 4; i++)
  write_head(..., IB_UNSIGNED, measurements[i]);

Yes, this leaves the onus of creating valid CBOR on the application, but it's not that hard to do that.
Maybe add a little function to write a (signed) integer.
(Yes, that's the reason why the application sees cbor.h in the current design.)

For applications that operate on a received data structure and send back a modified instance, a full-blown serializer from the internal data structure format is useful as well.

I'd like to make sure we don't put all of this in one bloated .o -- applications should only need to link in the parts they need. (That's why even cn_cbor_error_str isn't part of the basic cn_cbor.c -- you never need it on a constrained node, and it increases the size of the library by one third!)

[hildjj]

If you like, i'll move the errors to another .c file. I like your approach, and will fiddle with it some to see if I can get it to work.

[hildjj]

I'm almost there, just have to debug, but I have to stop for dinner.

[hildjj]

My branch (https://github.com/hildjj/cn-cbor/tree/encode) seems to work.

[cabo]

Right. I'll merge that (encode-2, actually) in a couple of minutes, but this leaves a gap:
The encoder assumes you have filled in cn_cbor structs with what you want to write.
What's missing is a raw interface so you can write right from the application data structures.
(Yes, that will put some onus on the application to ensure writing correct CBOR, but that is trivial.)
write_head really is meant to be used by applications, not only from the visitor.
So, as a next step after merging, I'll try to extract that raw application interface (and put it into a separate file, so implementations that need just that won't need the visitor code).

[hildjj]

I actually need the write_head functionality. I want to have the following CBOR on the wire: {0: h'...'}, where the bytestring is potentially long, and is passed in to my function. I want to set up an iov struct with everything but the bytestring, then the bytestring in a separate iov, so I don't have to copy it.

However, "head" seems really confusing to people. "preamble"? "beginning"? "preface"? "prelude"?

[cabo]

I think I tried quite a few alternatives. RFC 7049 is not using a term for the head because Paul also saw the potential for confusion. But then, I'm still not sure what is confusing about the "head" for a data item. "Header", if it must be.

[hildjj]

Header is better than head.