redefine {.incompleteStruct.} to use as safe guards for wrapped types

[alaviss]

Opening as discussion first to explore whether the idea is solid before making it an issue.

As of writing, incompleteStruct is defined as:

The incompleteStruct pragma tells the compiler to not use the underlying C struct in a sizeof expression

This is strictly an implementation detail for the C backend and is not very useful for defining pure wrappers (ie. wrappers not using .header).

I propose to redefine incompleteStruct as:

The incompleteStruct pragma marks an object as incomplete, meaning:

• The size of the object is undefined.
• The memory layout of the object is not known.

as such, several restrictions are applied on a marked type:

• Use of sizeof, alignof or offsetof is considered a compilation error.
• It is forbidden to create values of the given type, either implicit or explicit.
• It is forbidden to retrieve values of the given type (i.e. via dereferencing, as return value from procedures).
• It is forbidden to have fields of the given type in any NimSkull-defined structure (incl. object and tuple). Indirection via reference types are allowed.
• It is forbidden to create variables of the given type. However, it is allowed to import such variables from external storage (i.e. using importc).
• Marked object types may not inherit from other types.
• Marked object types are final and cannot be inherited from, regardless of whether {.inheritable.} is specified.
• No fields can be defined for any marked object type.
• Values of the given type can only be passed as references.
• The default =copy, =sink implementations are marked as {.error.}.

The goal of the new definition is to enable the creation of safer wrappers, for example:

struct Bar;
struct Bar* new_bar(void);
void do_clone(struct Bar* a, const struct Bar* b);

would be defined as:

type Bar {.incompleteStruct.} = object
proc new_bar(): ptr Bar {.importc.}
proc do_clone(a: ptr Bar, b: ptr Bar) {.importc.}
# Alternatively
# proc do_clone(a: var Bar, b: Bar) {.importc.}

preventing these kinds of misuse (all of which are possible at the moment):

let b: ptr Bar = new_bar()

var a: Bar # Error: local `a` has unknown size
do_clone(a, b)

var global {.importc.}: Bar

var a = global # Error: local `a` has unknown size

var x: seq[Bar] # Error: `Bar` has unknown size
x.add global

EDIT: Updated with inheritance semantics as discussed in #1520 (comment)

[alaviss]

I'll implement this if accepted, it'd make good practice for compiler work.

[saem]

Thanks for putting together a discussion and a definition.

For some reason I had this confused with partial struct definition, whoops.

I like the definition, we might want the following:

• it's considered final and cannot be inherited from

[alaviss]

I totally forgot about inheritance, thanks.

I'd say we can start with these rules:

• Marked objects are final and cannot be inherited from, regardless of whether {.inheritable.} is specified.
• Marked objects may not inherit from any other objects.

---

Inheritance is a bit tricky, as there are two separate sides to it:

• Runtime: Stuff that requires RTTI to function (i.e. of, method, checked conversions, etc.)
• Compile-time:
  • Automatic conversion to parent class: Button -> Widget
  • Participation in generics: T: Widget can match Button, TextField, etc.

The runtime side strictly requires our RTTI implementation, but the CT side can be useful for wrapping things with a hierarchy like Gtk or something as simple as struct sockaddr, so there might be an argument to allow that.

But at the same time there is an argument against this since it might lock us into a particular style of objects. There are also hierarchies that cannot be specified in nimskull terms (i.e. multiple inheritance), and we might not want to set expectations where users would want those to be representable for FFI.

What do you think about this?

[saem]

I like the proposed restrictions as a starting point, let's move forward with those.

---

I think we're going to run into a bunch of issues with inheritance around sub typing/RTTI/etc as you indicated which I don't think we're ready to handle just yet. If we tried to tackle this now, I have a feeling that we'll have to go around and touch large chunks of the compiler in order to account for handling incompleteStruct gotchas.

I suspect, that before supporting inheritance with .incompleteStruct, we should probably explore other related ideas like an unknown type or something like it, where it means we don't know the rest of the type, kinda like this:

type
  Foo = tuple
    a: int
    b: string
    theRest: unknown

Where the type is basically partially known/understood, and I can see that being something where we might mess with inheritance. Right now incompleteStruct feels more like an opaque at runtime type, so inheritance being a non-starter makes sense.

[alaviss]

Moved to #1522 as new feature request.