Having troubles on developing and debuging

[ZjuYTW]

Hi there!
I have went through some problems when I wanted to make contribution to c++ source code.

1. (Maybe too trival) I coding on VSCode and when I settled all enviornment and successfully run the program, I found some dependencies marked like, 'Unable to open source file 'arrow/api.h''. Though I've installed apache-arrow by brew. In the previous experience, I could fix this by introducing "./build/compile_commands.json" to VSCode properties, but this time I failed.. So maybe you could help me with it?

2. I want to run the CTest on local enviornment which is macos monterey and chip is M1. But seems all the tests are failed... I have no idea whether I am supposed to run this project on macos native env. By the way, to build the project by sources, I skipped "libgraphlite" by set -DBUILD_VINEYARD_GRAPH=OFF while cmake. And other steps seems all fine to me.

Thx in advance for your reply

[sighingnow]

Hi @ZjuYTW,

Thanks for taking a try on vineyard!

I could fix this by introducing "./build/compile_commands.json" to VSCode properties, but this time I failed.. So maybe you could help me with it?

Which vscode plugin you are working with?

But seems all the tests are failed... I

The test is not executed by make test. Instead, we have a script (./test/runner.py) to setup the required environment for testing. You could find some usage from https://github.com/v6d-io/v6d/blob/main/.github/workflows/build-test.yml#L297.

I will add a doc section about how to run the tests later.

[ZjuYTW]

1. I use c/c++ extension pack (the most commonly used plugin) ?
2. I think there might be abnormal to run on my MacOS so I decide to build it on my linux server... (Ubuntu 2004) (fixed)
   But this time I got

CMake Error at thirdparty/nlohmann-json/CMakeLists.txt:85 (add_library):
  add_library cannot create target "nlohmann_json" because an imported target
  with the same name already exists.


CMake Error at thirdparty/nlohmann-json/CMakeLists.txt:86 (add_library):
  add_library cannot create ALIAS target "nlohmann_json::nlohmann_json"
  because target "nlohmann_json" is imported but not globally visible.


CMake Error at thirdparty/nlohmann-json/CMakeLists.txt:169 (export):
  export given target "nlohmann_json" which is not built by this project.


CMake Error at thirdparty/nlohmann-json/CMakeLists.txt:174 (install):
  install TARGETS given target "nlohmann_json" which does not exist.

when i use cmake .. to generate Makefile T_T. Could you help me with it?

P.S. All fixed, but still looking forward to a local test tutorial🙏

[sighingnow]

Added in #747, will be merged once the CI turns green.

[ZjuYTW]

@sighingnow I have encountered a new problem when I was running ./test/runner.py --with-cpp, and I thought the problem is kind of related to this topic so I decide to place here.

> ./test/runner.py --with-cpp
binary_dir =  /root/Desktop/v6d/v6d/test/../build/bin
Starting /root/Desktop/v6d/v6d/test/../build/bin/vineyardd... with --size 3221225472 --socket /tmp/vineyard.ci.1655647682.622581.sock --rpc_socket_port 2059 --etcd_endpoint http://localhost:2048 --etcd_prefix vineyard_test_1655647682.7583897
I0619 22:08:02.770009 387778 vineyardd.cc:91] Hello vineyard v0.5.2!
I0619 22:08:02.770514 387778 meta_service.h:136] meta service is starting ...
I0619 22:08:02.772357 387778 etcd_launcher.cc:94] Starting the etcd server
I0619 22:08:02.772398 387778 etcd_launcher.cc:102] Found etcd at: /usr/local/bin/etcd
I0619 22:08:02.773375 387778 etcd_launcher.cc:208] Etcd launched: pid = 387822, listen on 2048
{"level":"info","ts":1655647682.808698,"caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_LOG_LEVEL","variable-value":"error"}
running test case -*-*-*-*-*-  array_test  -*-*-*-*-*-*-*-
[error] Assertion failed in "handler != nullptr": Failed to load the vineyard global registry registry: libvineyard_internal_registry.so: cannot open shared object file: No such file or directory, in function 'std::unordered_map<std::__cxx11::basic_string<char>, std::unique_ptr<vineyard::Object> (*)()>* vineyard::detail::__instantize__registry(void*&, void* (*&)())', file /root/Desktop/v6d/v6d/src/client/ds/object_factory.cc, line 147
terminate called after throwing an instance of 'std::runtime_error'
  what():  Assertion failed in "handler != nullptr": Failed to load the vineyard global registry registry: libvineyard_internal_registry.so: cannot open shared object file: No such file or directory, in function 'std::unordered_map<std::__cxx11::basic_string<char>, std::unique_ptr<vineyard::Object> (*)()>* vineyard::detail::__instantize__registry(void*&, void* (*&)())', file /root/Desktop/v6d/v6d/src/client/ds/object_factory.cc, line 147
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
I0619 22:08:04.214208 387778 allocator.cc:69] shmax: 3988230144
I0619 22:08:04.214687 387778 etcd_meta_service.cc:321] start background etcd watch, since 4
I0619 22:08:04.217192 387778 meta_service.h:453] Decide to set rank as 0
I0619 22:08:04.217214 387778 meta_service.h:856] Instance join: 0
I0619 22:08:04.218814 387778 ipc_server.cc:52] Vineyard will listen on "/tmp/vineyard.ci.1655647682.622581.sock" for IPC
I0619 22:08:04.218842 387778 rpc_server.cc:53] Vineyard will listen on 0.0.0.0:2059 for RPC
I0619 22:08:04.218858 387778 meta_service.h:856] Instance join: 0
--------------------------------------------------------------------------
mpiexec noticed that process rank 0 with PID 0 on node iZt4ngs3tx7g8obmliyrymZ exited on signal 6 (Aborted).
--------------------------------------------------------------------------
Terminating /root/Desktop/v6d/v6d/test/../build/bin/vineyardd
I0619 22:08:05.814824 387778 vineyardd.cc:61] SIGTERM Signal received, stop vineyard server...
I0619 22:08:05.814997 387796 etcd_meta_service.cc:343] Watcher stopped, as cancelled: true
I0619 22:08:05.815043 387778 vineyard_runner.cc:116] Delete session : S0000000000000000
Traceback (most recent call last):
  File "./test/runner.py", line 665, in <module>
    main()
  File "./test/runner.py", line 637, in main
    run_single_vineyardd_tests(args.tests)
  File "./test/runner.py", line 301, in run_single_vineyardd_tests
    run_test(tests, 'array_test')
  File "./test/runner.py", line 238, in run_test
    subprocess.check_call(
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['mpiexec', '--allow-run-as-root', '-mca', 'orte_allowed_exit_without_sync', '1', '-mca', 'btl_vader_single_copy_mechanism', 'none', '-n', '1', '--host', 'localhost:1', '/root/Desktop/v6d/v6d/test/../build/bin/array_test', '/tmp/vineyard.ci.1655647682.622581.sock']' returned non-zero exit status 134.

Seems the program can't find the libvineyard_internal_registry.so, but the file actually exists in my build/shared-lib/.

And I run on master, my OS is Ubuntu 20.04

[sighingnow]

You could make install first, then add /usr/local/lib to your LD_LIBRARY_PATH. Then the test script should work.

[sighingnow]

Fixed on main, @ZjuYTW you could pull the latest main branch and it should work.

[ZjuYTW]

Worked, Thx!

[sighingnow]

I have opened an issue to track the problem: #795