Custom Functors in C++

[CyrilFMoser]

#include <souffle/SouffleInterface.h>

#include <iostream>

extern "C" {

souffle::RamDomain insert(souffle::SymbolTable* symbolTable, souffle::RecordTable* recordTable,
                          souffle::RamDomain name, souffle::RamDomain type, souffle::RamDomain map) {
    assert(symbolTable && "NULL symbol table");
    assert(recordTable && "NULL record table");

    if (map != 0) {
        const souffle::RamDomain* cur_map = recordTable->unpack(map, 3);
        if (cur_map == 0) {
            return map;
        }
        souffle::RamDomain cur_name = cur_map[0];
        souffle::RamDomain cur_type = cur_map[1];

        souffle::RamDomain next_map_packed = insert(symbolTable, recordTable, name, type, cur_map[2]);
        souffle::RamDomain new_map[3] = {cur_name, type, next_map_packed};  // simply replace the type field in this new map
        return recordTable->pack(new_map, 3);
    }
    return map;
}
}

This is my current implementation of a function that simply replaces the type field with the argument type for every element in this map instance.

.type Type = Int{}
            |Char{}
            |Byte{}
            |Boolean{}

.type Map = [name:symbol,type:Type,next:Map]

.functor insert(name:symbol,type:Type,m:Map):Map stateful

.decl available_map(m:Map)

available_map(["A",$Int(),x]) :- @insert("B",$Char(),["B",$Boolean(),nil]) = x.

.output available_map

This is how I call the function. But when I run this souffle program, I get this error:

souffle: /souffle/src/include/souffle/datastructure/ConcurrentFlyweight.h:290: const Key& souffle::ConcurrentFlyweight<LanesPolicy, Key, Hash, KeyEqual, KeyFactory>::fetch(souffle::ConcurrentFlyweight<LanesPolicy, Key, Hash, KeyEqual, KeyFactory>::lane_id, souffle::ConcurrentFlyweight<LanesPolicy, Key, Hash, KeyEqual, KeyFactory>::index_type) const [with LanesPolicy = souffle::ConcurrentLanes; Key = std::__cxx11::basic_string<char>; Hash = std::hash<std::__cxx11::basic_string<char> >; KeyEqual = std::equal_to<std::__cxx11::basic_string<char> >; KeyFactory = souffle::details::Factory<std::__cxx11::basic_string<char> >; souffle::ConcurrentFlyweight<LanesPolicy, Key, Hash, KeyEqual, KeyFactory>::lane_id = long unsigned int; souffle::ConcurrentFlyweight<LanesPolicy, Key, Hash, KeyEqual, KeyFactory>::index_type = long unsigned int]: Assertion `Idx < SlotCount.load(std::memory_order_relaxed)' failed.

I already tried to figure out why this happens here:
https://github.com/souffle-lang/souffle/blob/master/src/include/souffle/datastructure/ConcurrentFlyweight.h

But to no avail. What am I doing wrong?

[quentin]

Hi,
Which version of Souffle are you using? Can you share the build command used to compile the functor? Can you share the output of "souffle --version" ?

It looks like the sort of error that happens when either Souffle or the functor do not have OpenMP enabled, but it may be something else as well.

[CyrilFMoser]

Yes thank you so much for the quick response!

I compile my functor with this:

g++ -std=c++17 functors.cpp -c -fPIC -fopenmp -o functors.o 
g++ -shared -o libfunctors.so functors.o 
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}`pwd`

This is my output for souffle --version:

----------------------------------------------------------------------------
Version: 2.4
Word size: 64 bits
Options enabled: ffi openmp ncurses sqlite zlib
----------------------------------------------------------------------------
Copyright (c) 2016-22 The Souffle Developers.
Copyright (c) 2013-16 Oracle and/or its affiliates.
All rights reserved.
============================================================================

[quentin]

Did you build Soufflé or did you installed it somehow? Are you using headers from the same sources/build ? I don't see a -I in your compile commands.

Can you share the souffle run command? Do you get any output before the crash?
Have you tried to build the functor with debug symbols (g++ -g) and run with gdb ?

[quentin]

Since souffle is built with Word size: 64 bits, you need to pass -DRAM_DOMAIN_SIZE=64.

[CyrilFMoser]

I installed it. Otherwise it works fine, it also works fine if I change the following in the datalog file to this:

available_map(["A",$Int(),x]) :- @insert("B",$Char(),nil) = x.

As it then simply skips over the functor, but it still is able to call it.

The only header I am using is #include <souffle/SouffleInterface.h>

I don't get any output, but if I print to stderr in the functor this shows up before the crash. So it somehow happens after I return from the functor.

So this means that somehow my functor messes up the record table, so that it only gets noticed after returning from it. Maybe I don't insert the new_map properly?

I have not tried gdb, because I believe that it would be very hard to find where it goes wrong.

[CyrilFMoser]

Since souffle is built with Word size: 64 bits, you need to pass -DRAM_DOMAIN_SIZE=64.

That did it! Thank you so much!