[optimization] buffer can be dynamically allocated after calculation of required size

[XakerTwo]

main buffer defined as char buffer[1024*1024];
and the only place it accessed is append_to_buffer() - generate-cat-file.c @ 109
however, inside the buffer is accessed only on write pass, else - just check of its size not exceeded
but why? you can achieve more dynamic behaviour in term of buffer size by these few actions

• ommit buffer size check in append_to_buffer() or move it under write=true condition
• declare buffer as char *buffer;
• get required buffer size by unrolling encode_sequence(&s, encode_pkcs7_toplevel, true) - generate-cat-file.c @ 817, that internally performs no-write run
• and finally buffer = (char*)malloc(buffer_sz)

in addition you can add size limitation just before allocation.

also you can optimize encode_sequence() (or all similar functions) to something like

//this code is from previous iteration of my version, based on your
size_t encode_sequence(void *s, size_t a_fn(void*, bool), bool write)
{
	size_t len = a_fn(s, false);

	size_t l2 = encode_tag_and_length(SEQUENCE_TAG, len, write);
	if (write)
		return a_fn(s, write) + l2;

	return len + l2;
}

so encode_sequence(p, f, false) will be faster and cheaper, cuz there will be no useless run on write=false, that anyway would have been checked inside append_to_buffer()
ideally no function should reach append_to_buffer() on write=false

[johannesthoma]

Hi this sounds very interesting. Having a fixed size buffer is probably a bad
idea because one could specify 1000+ files to encode. Are you already working
on a patch? I think it shouldn't be that hard to implement but on the other hand
it is also not that urgent (have to drop now).

[XakerTwo]

not exactly... i altered you work in sligtly different direction, but partially this optimization implemented. Described part should be easy and done soon, but no any ETA, i assume that i'll create a PR when it's done