samu fails to build libjxl: `samu: file is missing and not created by any action: 'lib/CMakeFiles/jxl_export.dir'`

[mgorny]

To reproduce:

$ git clone -q https://github.com/libjxl/libjxl
$ cd libjxl
$ bash deps.sh
[…]
$ mkdir build
$ cd build
$ cmake .. -G Ninja
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_SYSTEM_PROCESSOR is x86_64
-- Performing Test CXX_FUZZERS_SUPPORTED
-- Performing Test CXX_FUZZERS_SUPPORTED - Failed
-- Performing Test CXX_MACRO_PREFIX_MAP
-- Performing Test CXX_MACRO_PREFIX_MAP - Success
-- Performing Test CXX_NO_RTTI_SUPPORTED
-- Performing Test CXX_NO_RTTI_SUPPORTED - Success
-- Found PkgConfig: /usr/bin/pkg-config (found version "2.1.0") 
-- tcmalloc version  -- tcmalloc 2.8.0 disabled due to https://github.com/gperftools/gperftools/issues/1204
-- Performing Test JXL_HWY_DISABLED_TARGETS_FORCED
-- Performing Test JXL_HWY_DISABLED_TARGETS_FORCED - Failed
-- Compiled IDs C:GNU, C++:GNU
-- Disabled AVX512 (set JPEGXL_ENABLE_AVX512 to enable it)
-- Disabled AVX512_SPR (set JPEGXL_ENABLE_AVX512_SPR to enable it)
-- Disabled AVX512_ZEN4 (set JPEGXL_ENABLE_AVX512_ZEN4 to enable it)
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Performing Test ATOMICS_LOCK_FREE_INSTRUCTIONS
-- Performing Test ATOMICS_LOCK_FREE_INSTRUCTIONS - Success
-- Found Python: /usr/bin/python3.13 (found version "3.13.0") found components: Interpreter 
CMake Deprecation Warning at third_party/highway/CMakeLists.txt:25 (cmake_policy):
  The OLD behavior for policy CMP0111 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- Performing Test HWY_EMSCRIPTEN
-- Performing Test HWY_EMSCRIPTEN - Failed
-- Performing Test HWY_RISCV
-- Performing Test HWY_RISCV - Failed
-- Looking for sys/auxv.h
-- Looking for sys/auxv.h - found
-- Looking for asm/hwcap.h
-- Looking for asm/hwcap.h - not found
-- Setting build type to Release as none was specified.
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Found ZLIB: /usr/lib64/libz.so (found version "1.3.1")  
-- Found PNG: /usr/lib64/libpng.so (found version "1.6.40") 
CMake Deprecation Warning at third_party/sjpeg/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Performing Test SJPEG_HAVE_FLAG___SSE2__
-- Performing Test SJPEG_HAVE_FLAG___SSE2__ - Success
-- Performing Test SJPEG_HAVE_FLAG___ARM_NEON__
-- Performing Test SJPEG_HAVE_FLAG___ARM_NEON__ - Failed
-- Performing Test SJPEG_HAVE_FLAG___ARM_NEON__
-- Performing Test SJPEG_HAVE_FLAG___ARM_NEON__ - Failed
-- Found JPEG: /usr/lib64/libjpeg.so (found version "62") 
-- Found OpenGL: /usr/lib64/libOpenGL.so   
-- Found GLUT: /usr/lib64/libglut.so  
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Performing Test CXX_WPSABI_SUPPORTED
-- Performing Test CXX_WPSABI_SUPPORTED - Success
-- Performing Test LINKER_SUPPORT_EXCLUDE_LIBS
-- Performing Test LINKER_SUPPORT_EXCLUDE_LIBS - Success
-- Found GIF: /usr/lib64/libgif.so (found suitable version "5.2.1", minimum required is "5.1") 
-- Checking for module 'OpenEXR'
--   Found OpenEXR, version 3.1.11
-- Found GTest: $<TARGET_FILE:gtest>  
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Found Python3: /usr/bin/python3.13 (found version "3.13.0") found components: Interpreter 
-- Building with JPEGXL_VERSION=a8d43397 (auto-updated)
-- Checking for module 'libwebp'
--   Found libwebp, version 1.3.2
CMake Warning at tools/CMakeLists.txt:286 (message):
  Using dynamic libwebp


-- Checking for module 'libavif'
--   Found libavif, version 1.0.3
-- Building tools: cjxl;djxl;jxlinfo;cjpegli;djpegli;benchmark_xl
-- Configuring done (3.4s)
-- Generating done (0.3s)
-- Build files have been written to: /tmp/libjxl/build
$ samu
samu: file is missing and not created by any action: 'lib/CMakeFiles/jxl_export.dir'

Reproduce with samurai as of 217059f, ninja 1.11.1 works.

[eli-schwartz]

With ninja:

cd /tmp/libjxl/build && /usr/bin/python3.12 /usr/bin/a2x --format manpage --destination-dir="/tmp/libjxl/build" /tmp/libjxl/doc/man/djxl.txt
Traceback (most recent call last):
  File "/usr/bin/a2x", line 5, in <module>
    from asciidoc.a2x import cli
ModuleNotFoundError: No module named 'asciidoc'

...

The build rule in question:

build cmake_object_order_depends_target_jxl_export: phony || lib/CMakeFiles/jxl_export.dir

Order-only dependency, which seemingly ninja treats as "if it cannot be built then ignore it"? If you manually edit build.ninja and switch it to | (implicit dependency rather than order-only) then ninja would error out too:

ninja: error: 'lib/CMakeFiles/jxl_export.dir', needed by 'cmake_object_order_depends_target_jxl_export', missing and no known rule to make it

I am not sure what the purpose of this rule is. There's no rule to make it, so it shouldn't be an order-only dependency as it cannot be "built" in or out of order.

My gut feeling is that this is samu not being bug-for-bug compatible with ninja, in a case where ninja is technically wrong.

[orbea]

@mgorny Is there a Gentoo issue for this?

I tried reproducing it, but it seems to just work for me...

I tried this with the current samurai commit (217059f), samurai-1.2 and using libjxl-0.8.2, the libjxl-9999 Gentoo ebuild and the git repo (libjxl/libjxl@bf2b765) as shown in the OP reproduction.

Gentoo's app-text/asciidoc can be used to get past the "No module named 'asciidoc'" error.

[mgorny]

It was fixed in libjxl/libjxl@c87291f, so you need to use a commit prior to that.

[orbea]

Ah, reading the discussion in PR libjxl/libjxl#3196 was also helpful for understanding.

There is an argument that projects shouldn't produce invalid build.ninja files and perhaps the upstream fix is all that is needed to be done here?

[vtorri]

note that muon has added some kind of ninja compilation, code forked from samurai with some modifications

[michaelforney]

I tried to make a minimal reproducer:

rule echo
  command = echo $in
build foo: echo a || b

But this failed in the same way as samurai:

ninja: error: 'b', needed by 'foo', missing and no known rule to make it

So it needs to be a phony rule to ignore missing b:

rule echo
  command = echo $in
build foo: phony a || b

and now ninja doesn't care that b is missing:

ninja: no work to do.

This seems to contradict the ninja docs that say

Semantically, the phony rule is equivalent to a plain rule where the command does nothing, but phony rules are handled specially in that they aren’t printed when run, logged (see below), nor do they contribute to the command count printed as part of the build process.

So I think this is a bug in cmake. Nothing in https://cmake.org/cmake/help/latest/command/add_library.html#id3 seems to indicate that object libraries must have at least one source file.

[michaelforney]

note that muon has added some kind of ninja compilation, code forked from samurai with some modifications

Good to know, but what does this have to do with this issue?

[Hello71]

My view was also that this is a CMake bug, so I filed https://gitlab.kitware.com/cmake/cmake/-/issues/25526 in 2023, which was fixed, so I think no samurai change is needed.