GIR in Apache Arrow - Error: OutputStream matches conflicting symbols

[kassane]

Hi, 👋

I have been testing that bindgen in Apache-Arrow during the first time after converting the files. I had some conflicts that required manual patching.

some minor fixes:
kassane/arrow@4bcfed1
kassane/arrow@86c9062

But...

Output error

# $PWD = arrow
$ dub test --root=d -f
             Generating test runner configuration 'arrow-d-test-unittest' for 'unittest' (library).
     Pre-gen Running commands for glibd
    Existing package girtod found locally
0 packages fetched, 1 already present, 0 failed
             Building package girtod in /home/kassane/.dub/packages/girtod/0.23.2/girtod/
     Pre-gen Running commands for girtod
    Starting Performing "debug" build using /usr/bin/ldc2 for x86_64.
    Building girtod 0.23.2: building configuration [application]
     Linking girtod
     Running ../../../girtod/0.23.2/girtod/girtod -i src -o generated --use-runtime-linker
copying file [src/gtkd] to [generated/gtkd]
    Starting Performing "unittest" build using /usr/bin/ldc2 for x86_64.
    Building glibd 2.4.3+commit.2.g1546823: building configuration [library]
    Building arrow-d ~master: building configuration [arrow-d-test-unittest]
source/arrow/GIOOutputStream.d(12,8): Error: `OutputStream` matches conflicting symbols:
public class GIOOutputStream : OutputStream
       ^
source/arrow/OutputStream.d(18,8):        class `arrow.OutputStream.OutputStream`
public class OutputStream : ObjectG, FileIF, WritableIF
       ^
../../.dub/packages/glibd/1546823185334c4727d378baf890fa13d9fa4cbd/glibd/generated/gio/OutputStream.d(58,8):        class `gio.OutputStream.OutputStream`
public class OutputStream : ObjectG
       ^
source/arrow/GIOOutputStream.d(55,9): Error: `OutputStream` matches conflicting symbols:
        public this(OutputStream gioOutputStream)
        ^
source/arrow/OutputStream.d(18,8):        class `arrow.OutputStream.OutputStream`
public class OutputStream : ObjectG, FileIF, WritableIF
       ^
../../.dub/packages/glibd/1546823185334c4727d378baf890fa13d9fa4cbd/glibd/generated/gio/OutputStream.d(58,8):        class `gio.OutputStream.OutputStream`
public class OutputStream : ObjectG
       ^
source/arrow/GIOOutputStream.d(76,22): Error: `OutputStream` matches conflicting symbols:
        public OutputStream getRaw()
                     ^
source/arrow/OutputStream.d(18,8):        class `arrow.OutputStream.OutputStream`
public class OutputStream : ObjectG, FileIF, WritableIF
       ^
../../.dub/packages/glibd/1546823185334c4727d378baf890fa13d9fa4cbd/glibd/generated/gio/OutputStream.d(58,8):        class `gio.OutputStream.OutputStream`
public class OutputStream : ObjectG
       ^
source/arrow/LargeListArray.d(138,27): Error: function `DataType arrow.LargeListArray.LargeListArray.getValueType()` does not override any function, did you mean to override `arrow.c.types.GArrowType arrow.Array.Array.getValueType()`?
        public override DataType getValueType()
                          ^
source/arrow/ListArray.d(134,27): Error: function `DataType arrow.ListArray.ListArray.getValueType()` does not override any function, did you mean to override `arrow.c.types.GArrowType arrow.Array.Array.getValueType()`?
        public override DataType getValueType()
                          ^

cc: @MikeWey

Reference

• [D] D programming language Implementation of Arrow apache/arrow#44515

[kassane]

I added APILookup.txt for making fixes. However, certain commands are not sufficiently clear, for example:
How do I fix or overwrite a function that has the wrong parameters?

     Running ../../.dub/packages/girtod/0.23.2/girtod/girtod -i source -o source --use-runtime-linker
source/arrow/ExecutePlan.d(94,38): Error: expression expected, not `,`
                auto __p = garrow_execute_plan_new(, &err);
                                     ^
source/arrow/c/functions.d(2709,30): Error: basic type expected, not `,`
        GArrowExecutePlan* function(, GError** err) c_garrow_execute_plan_new;

[MikeWey]

The problem with "garrow_execute_plan_new" should be fixed with commit 441cf5f.

To override a function you would use noCode for the function in question, and also a noExternal if the external definition is also bad.
Then you can put the manual implementation in a code block, and the new external deceleration in a addFuncts block.

addFuncts: start
    the external definition, and linker command if needed
addFuncts: end

struct: ExecutePlan
noCode: new
noExternal: new
code: start
    Your code here
code: end

[kassane]

The problem with "garrow_execute_plan_new" should be fixed with commit 441cf5f.

Done!

However, back to the main issue.

I tried:

APILookupArrow.txt

wrap: arrow
file: Arrow-1.0.gir

# rename
alias: import import_

# avoid conflicts with gio.InputStream
struct: GArrowInputStream
class: InputStream
extend: GObject.Object
override: isClosed

# avoid conflicts with gio.OutputStream 
struct: GArrowOutputStream
class: OutputStream
extend: GObject.Object

# fix getValue override issues
struct: GArrowDecimal128Array
class: Decimal128Array
override: getValue

struct: GArrowDecimal256Array 
class: Decimal256Array
override: getValue

# fix getValueType override issues
struct: GArrowLargeListArray
class: LargeListArray
override: getValueType

struct: GArrowListArray
class: ListArray 
override: getValueType

Output

  Up-to-date girtod 0.23.3: target for configuration [application] is up to date.
    Finished To force a rebuild of up-to-date targets, run again with --force
     Running ../../.dub/packages/girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/girtod -i source -o source --use-runtime-linker
core.exception.RangeError@../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/gtd/LinkedHasMap.d(122): Range violation
----------------
??:? [0x56f54ab612fe]
??:? [0x56f54ab60f62]
??:? [0x56f54ab881ee]
??:? [0x56f54ab6937f]
??:? [0x56f54ab5faf8]
??:? [0x56f54ab60007]
../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/gtd/LinkedHasMap.d:122 [0x56f54aa77f76]
../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/gtd/GirWrapper.d:493 [0x56f54aa7e680]
../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/gtd/GirWrapper.d:137 [0x56f54aa7bd04]
../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/gtd/GirWrapper.d:80 [0x56f54aa7b474]
../../../girtod/441cf5f62f311c9a21c59b14218f0e912c8addb3/girtod/source/girtod.d:81 [0x56f54aaece5f]
??:? [0x56f54ab6905c]
??:? [0x56f54ab68f56]
??:? [0x56f54ab68dac]
/home/kassane/zig/ldc2-master/bin/../import/core/internal/entrypoint.d:42 [0x56f54aafb191]
??:? [0x7e0b1b66de07]
??:? __libc_start_main [0x7e0b1b66decb]
??:? [0x56f54aa74a44]

[MikeWey]

The APILookup files use the names as they are defined in the gir file.
Also you can skip the class and extends directives if you are not changing what is read from the gir files.
GArrowImputStream doesn;t have a is_closed function.

You would end up with this for the lookup file.

wrap: arrow
file: Arrow-1.0.gir

# rename
alias: import import_

# avoid conflicts with gio.InputStream
struct: InputStream
#override: is_closed

# avoid conflicts with gio.OutputStream 
struct: OutputStream

# fix getValue override issues
struct: Decimal128Array
override: get_value

struct: Decimal256Array
override: get_value

# fix getValueType override issues
struct: LargeListArray
override: get_value_type

struct: ListArray
override: get_value_type

Only we should exit with an error instead of crashing on a undefined function name.

[kassane]

GArrowImputStream doesn't have a is_closed function.

FileT.d has isClosed() in:

public template FileT(TStruct)
{
    // [...]
	public bool isClosed() // fix: missing override 
	{
		return garrow_file_is_closed(getFileStruct()) != 0;
	}
    // [...]
}

You would end up with this for the lookup file.

This change in APILookup get same error!

Anyway, it suggests that it will need to be partially reimplemented, may be use:

code: start
    Your code here
code: end