Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twister Blackbox tests' paths are too long for CMake on Windows #83229

Open
LukaszMrugala opened this issue Dec 19, 2024 · 0 comments
Open

Twister Blackbox tests' paths are too long for CMake on Windows #83229

LukaszMrugala opened this issue Dec 19, 2024 · 0 comments
Assignees
Labels
area: Twister Twister area: Windows Support Related to building Zephyr on Windows bug The issue is a bug, or the PR is fixing a bug

Comments

@LukaszMrugala
Copy link
Collaborator

LukaszMrugala commented Dec 19, 2024

Describe the bug

When trying to run Twister BlackBox tests on Windows, paths used by CMake are too long, which causes a fatal error on all tests requiring a proper build step.

To Reproduce

  1. Set up Zephyr repo on a Windows Machine (Note to avoid FileNotFound errors caused by placing Zephyr too deep in the filesystem)
  2. Run Twister BlackBox Tests via pytest

Expected behavior
Tests run without a warning about paths, which causes a fatal error further down the line.

Impact
I cannot include BlackBox testing in my CI, which makes me unable to known how my changes impact Windows users.

Logs and console output
Path length warning:

CMake Warning in C:/Windows/Temp/p/test_build_only___build_only_b0/blackbox-out-container/blackbox-out/qemu_x86_atom/scripts/tests/twister_blackbox/test_data/tests/dummy/agnostic/group1/subgroup2/dummy.agnostic.group1.subgroup2/CMakeFiles/CMakeScratch/TryCompile-5tjvtp/CMakeLists.txt:
  The object file directory
    C:/Windows/Temp/p/test_build_only___build_only_b0/blackbox-out-container/blackbox-out/qemu_x86_atom/scripts/tests/twister_blackbox/test_data/tests/dummy/agnostic/group1/subgroup2/dummy.agnostic.group1.subgroup2/CMakeFiles/CMakeScratch/TryCompile-5tjvtp/CMakeFiles/cmTC_12227.dir/./
  has 281 characters.  The maximum full path to an object file is 250
  characters (see CMAKE_OBJECT_PATH_MAX).  Object file
    src.c.obj
  cannot be safely placed under this directory.  The build may not work
  correctly.

Eventual error:

<ZEPHYR_BASE>/arch/x86/core/offsets/offsets.c:31:1: fatal error: opening dependency file zephyr\CMakeFiles\offsets.dir\f70b1b4af56b0ccea97d199dff598841\offsets.c.obj.d: No such file or directory
   31 | GEN_ABS_SYM_END
      | ^~~~~~~~~~~~~~~
compilation terminated.

Environment (please complete the following information):

  • OS: Windows
  • Toolchain: Zephyr SDK 17
  • Commit SHA or Version used

Additional context
Note that I was aware of the restriction on path length in Windows. I have created a junction in the Temp folder (C:\Windows\Temp\d\s) to the parent of the normal ZEPHYR_BASE directory and used the shortened path (C:\Windows\Temp\d\s\zephyr) as the ZEPHYR_BASE.
Additionally, I have used the --basetemp flag in pytest in order to shorten the file path of the tmp_path used to create the outdir to C:\Windows\Temp\p.
All those methods were insufficient.
I'd like to avoid changing registry keys on my machine.
The Twister flag --short-build-paths is not an answer, because it requires the use of ninja, while some blackbox tests require ninja not to be used.

Note that getting just under 250 is insufficient - pytest always appends test name to the path, so it would take just one slightly longer test to break it all again.

One change that would help a bit - shortening the blackbox-out-container/blackbox-out section - it's not often viewed by users, so I could change conftest.py so it creates something like bboc/bbo. It would save 27 characters. It's not enough here, but a nice thought.

I'd like to point out one place in which a change might be warranted to fix this Issue:

@staticmethod
def get_unique(testsuite_root, workdir, name):
canonical_testsuite_root = os.path.realpath(testsuite_root)
if Path(canonical_zephyr_base) in Path(canonical_testsuite_root).parents:
# This is in ZEPHYR_BASE, so include path in name for uniqueness
# FIXME: We should not depend on path of test for unique names.
relative_ts_root = os.path.relpath(canonical_testsuite_root,
start=canonical_zephyr_base)
else:
relative_ts_root = ""
# workdir can be "."
unique = os.path.normpath(
os.path.join(relative_ts_root, workdir, name)
).replace(os.sep, '/')
return unique

The method for getting unique paths/names of TestSuites is already deemed as not the best. In my case, the path appended here "costs" me 79 characters out of possible 250 because blackbox tests use alternative test roots in order to e.g. have tests that always fail without breaking normal CI. Of course, --no-detailed-test-id flag is an option, however, blackbox tests also check that flag (i.a. test_detailed_test_id), so it cannot just be added to all blackbox tests.

@LukaszMrugala LukaszMrugala added bug The issue is a bug, or the PR is fixing a bug area: Twister Twister area: Windows Support Related to building Zephyr on Windows labels Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Twister Twister area: Windows Support Related to building Zephyr on Windows bug The issue is a bug, or the PR is fixing a bug
Projects
None yet
Development

No branches or pull requests

2 participants