[#432] Implement CLI iox2-config #468

brosier01 · 2024-10-14T21:53:07Z

Notes for Reviewer

Create a new CLI for iceoryx2 : iox2-config
iox2-config can show the configuration currently in use and generate a new configuration file at the default location iceoryx2 is looking for.
Remove the print_system_configuration() function in iceoryx2-bb/posix/src/system_configuration.rs file and move it into the CLI iox2-config

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Commits are properly organized and messages are according to the guideline
~~Unit tests have been written for new behavior~~
Public API is documented
PR title describes the changes

Post-review Checklist for the PR Author

All open points are addressed and tracked via issues

References

Relates to #432

elfenpiff · 2024-10-14T22:35:17Z

@brosier01 don't close it - it looked good!

orecham

In general looks pretty good, just one real blocker re: config location and also you'll need to address the static analysis CI findings.

Thanks for getting involved and welcome!

doc/release-notes/iceoryx2-unreleased.md

orecham · 2024-10-14T22:47:15Z

doc/release-notes/iceoryx2-unreleased.md

-   // old
-   let fuu = hello().is_it_me_you_re_looking_for()
+```bash
+   cargo run --bin iox2-config show


Suggested change

cargo run --bin iox2-config show

iox2 config show

doc/release-notes/iceoryx2-unreleased.md

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-14T23:06:31Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn show() -> Result<()> {
+    print_system_configuration();


Does this print the default in-built config if there is no config file being loaded from file?

print_system_configuration() displays the whole system configuration with all limits, features and details to the console. It does not print the built-in config but the system configuration. print_system_configuration() function is independent of the config file.

@elfenpiff Should the defaults be added to the output of iox2 config show, under a different sub-heading ? I would think yes, but what did you have in mind?

@elfenpiff Will need your feedback on this. What was your intention for this command?

@elfenpiff I implemented this to provide me with some debug output of the configuration of the POSIX system, meaning the POSIX configuration that states like how many semaphores can be created or what the maximum value of a semaphore could be.

Windows, Mac Os, FreeBSD, and Linux have different values here, and in the early days of iceoryx2, it was very helpful when analyzing bugs. Now, everything is abstracted away with the iceoryx2_pal and iceoryx2_cal layer but this still makes sense for mission-critical environments where we define the minimum requirements we have for the system (somewhere) and then this would print out the systems requirement and would also highlight the things which would violate our requirements.

A practical example is the iceoryx2_bb_threadsafe::trigger_queue. It uses two semaphores to trigger the producer/consumer when changes happen. But when the semaphores max value is 1024 it means the trigger queue can have a capacity of at most 1024 elements, otherwise the underlying semaphore would overflow and break the queue.

Another example is a gateway. When this subscribes to thousands of services it must be able to memory map thousands of shared memories and also hold thousands of fd's. By default the max number of fd's is 1024 so here it may makes sense to print a warning on gateway startup when the config does not fit.

@elfenpiff Apologies, what is your advice here ?

@orecham My apologies!

Should the defaults be added to the output of iox2 config show, under a different sub-heading ?

Yes, this would make sense.

@elfenpiff My apologies, did you say I had to set up a new subtitle? Like “iox2 config show default” which shows the default configuration? Am I right ?

@brosier01 Yeah, if you could additionally print the contents of the config file under another subtitle, this would be perfect.

Let us know if you hit problems or it turns out more difficult than it seems.

Co-authored-by: orecham <[email protected]>

…file

orecham

I tested the CLI on the different platforms. A few more things to resolve before it can be merged.

orecham · 2024-10-16T13:38:06Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn generate() -> Result<()> {
+    let config_dir = dirs::config_dir().unwrap().join("iceoryx2/");


This results in a directory C:\Users\orecham\AppData\Roaming\iceoryx2/config.toml on Windows (the last slash should be a backslash.) I think you can just remove the / here and rely on .join() adding the proper separators.

On a side note. Windows also supports the forward slash as path separator and if you need to pass a path as string I would recommend to use C:/Windows/foo or you might end up with something like C:\\\\Windows\\\\foo, depending on where the string is defined and how many layers it needs to pass until it reaches the actual Windows call.

orecham · 2024-10-16T13:42:38Z

iceoryx2-cli/iox2-config/src/commands.rs

+
+/// Prints the whole system configuration with all limits, features and details to the console.
+pub fn print_system_configuration() {
+    const HEADER_COLOR: &str = "\x1b[4;92m";


In PowerShell on Windows, these codes don't seem to be parsed correctly:

PS C:\Users\orecham> iox2 config show ←[4;92mposix system configuration←[0m ←[4;92msystem info←[0m ←[0;37mPosixVersion ←[0m ←[0;94m200809←[0m ←[0;37mPageSize ←[0m ←[0;94m4096←[0m ←[0;37mNumberOfClockTicksPerSecond ←[0m ←[0;94m0←[0m ←[0;37mNumberOfCpuCores ←[0m ←[0;94m16←[0m ←[4;92mlimits←[0m ←[0;37mMaxLengthOfHostname ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfLoginName ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSupplementaryGroupIds ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfOpenFiles ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSimultaneousProcessesPerUser ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfArguments ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfOpenStreams ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSymbolicLinks ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxLengthOfTerminalDeviceName ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfBytesInATimezone ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfSemaphores ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxSemaphoreValue ←[0m ←[0;94m2147483646←[0m ←[0;37mMaxNumberOfOpenMessageQueues ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxMessageQueuePriority ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxNumberOfThreads ←[0m ←[0;94m1024←[0m ←[0;37mMaxSizeOfPasswordBuffer ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxSizeOfGroupBuffer ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxPathLength ←[0m ←[0;94m260←[0m ←[0;37mMaxFileNameLength ←[0m ←[0;94m[ unlimited ]←[0m ←[0;37mMaxUnixDomainSocketNameLength ←[0m ←[0;94m108←[0m ←[0;37mMinStackSizeOfThread ←[0m ←[0;94m1048576←[0m ←[4;92moptions←[0m ←[0;90mAdvisoryInfo ←[0m ←[0;90mfalse←[0m ←[0;90mCpuTime ←[0m ←[0;90mfalse←[0m ←[0;90mFsync ←[0m ←[0;90mfalse←[0m ←[0;90mIpv6 ←[0m ←[0;90mfalse←[0m ←[0;90mMemLock ←[0m ←[0;90mfalse←[0m ←[0;90mMemLockRange ←[0m ←[0;90mfalse←[0m ←[0;90mMessagePassing ←[0m ←[0;90mfalse←[0m ←[0;90mPrioritizedIo ←[0m ←[0;90mfalse←[0m ←[0;90mPriorityScheduling ←[0m ←[0;90mfalse←[0m ←[0;90mRegularExpressions ←[0m ←[0;90mfalse←[0m ←[0;90mSpawn ←[0m ←[0;90mfalse←[0m ←[0;90mProcessSporadicServer ←[0m ←[0;90mfalse←[0m ←[0;90mSynchronizedIo ←[0m ←[0;90mfalse←[0m ←[0;90mThreadStackAddress ←[0m ←[0;90mfalse←[0m ←[0;90mThreadStackSize ←[0m ←[0;90mfalse←[0m ←[0;90mThreadCpuTimeClock ←[0m ←[0;90mfalse←[0m ←[0;90mThreadExecutionScheduling ←[0m ←[0;90mfalse←[0m ←[0;90mThreadProcessSharedSynchronization ←[0m ←[0;90mfalse←[0m ←[0;90mMutexPriorityInheritance ←[0m ←[0;90mfalse←[0m ←[0;90mMutexPriorityProtection ←[0m ←[0;90mfalse←[0m ←[0;90mRobustMutexPriorityInhertiance ←[0m ←[0;90mfalse←[0m ←[0;90mRobustMutexPriorityProtection ←[0m ←[0;90mfalse←[0m ←[0;90mThreadSporadicServer ←[0m ←[0;90mfalse←[0m ←[0;90mTrace ←[0m ←[0;90mfalse←[0m ←[0;90mTraceEventFilter ←[0m ←[0;90mfalse←[0m ←[0;90mTraceInherit ←[0m ←[0;90mfalse←[0m ←[0;90mTraceLog ←[0m ←[0;90mfalse←[0m ←[0;90mTypedMemoryObjects ←[0m ←[0;90mfalse←[0m ←[4;92mfeatures←[0m ←[0;37mBarriers ←[0m ←[0;94mtrue←[0m ←[0;90mAsynchronousIo ←[0m ←[0;90mfalse←[0m ←[0;90mClockSelection ←[0m ←[0;90mfalse←[0m ←[0;90mJobControl ←[0m ←[0;90mfalse←[0m ←[0;37mMappedFiles ←[0m ←[0;94mtrue←[0m ←[0;90mMemoryProtection ←[0m ←[0;90mfalse←[0m ←[0;90mMonotonicClock ←[0m ←[0;90mfalse←[0m ←[0;90mRawSockets ←[0m ←[0;90mfalse←[0m ←[0;37mReaderWriterLocks ←[0m ←[0;94mtrue←[0m ←[0;90mRealtimeSignals ←[0m ←[0;90mfalse←[0m ←[0;90mSavedUserAndGroupIds ←[0m ←[0;90mfalse←[0m ←[0;37mSemaphores ←[0m ←[0;94mtrue←[0m ←[0;37mSharedMemoryObjects ←[0m ←[0;94mtrue←[0m ←[0;90mShell ←[0m ←[0;90mfalse←[0m ←[0;37mSpinLocks ←[0m ←[0;94mtrue←[0m ←[0;37mThreadSafeFunctions ←[0m ←[0;94mtrue←[0m ←[0;90mThreads ←[0m ←[0;90mfalse←[0m ←[0;37mTimeouts ←[0m ←[0;94mtrue←[0m ←[0;37mTimers ←[0m ←[0;94mtrue←[0m ←[0;90mOpenRealtimeOptionGroup ←[0m ←[0;90mfalse←[0m ←[0;90mOpenRealtimeThreadsOptionGroup ←[0m ←[0;90mfalse←[0m ←[4;92mprocess resource limits←[0m ←[0;37mMaxCoreFileSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxConsumableCpuTime ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxDataSegmentSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxFileSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxNumberOfOpenFileDescriptors ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxStackSize ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m ←[0;37mMaxSizeOfTotalMemory ←[0m soft: ←[0;94m0 ←[0m hard: ←[0;94m0←[0m

A crate like colored, like used in the rest of the CLI, should be used for cross-platform coloring.

I know this is not your code, but this would need to be updated for UX purposes on Windows.

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-16T13:57:05Z

iceoryx2-cli/iox2-config/src/commands.rs

+use std::io::Write;
+
+/// Prints the whole system configuration with all limits, features and details to the console.
+pub fn print_system_configuration() {


This function has an error on macOS:

0 [F] "ProcessResourceLimit::soft_limit" | This should never happen! Unable to acquire soft limit for MaxSizeOfTotalMemory due to an unknown error (errno { name = "EINVAL", value = 22, details = "Invalid argument" }).

We should make it possible for the CI to catch stuff like this. Could you add a test that executes this function?

As for the reason, it's not immediately obvious to me. For this PR, one workaround could be to update the code to catch errors from the calls it uses to get the information it prints. The function could then continue even if a call has an error, and print a message to the user explaining which information was not retrievable.

cc @elfenpiff

FYI the bug is probably due to: #473

Not for this PR though - handling the error as described above would be sufficient.

iceoryx2-cli/iox2-config/src/commands.rs

…rm coloring + implementation of user confirmation required before deleting current configuration file

…file with the standard configuration path

orecham

The generated configs are now being used and there is a confirmation message for overwriting a pre-existing generated config - nice.

Some more things left to address, see unresolved threads.

orecham · 2024-10-24T16:04:32Z

iceoryx2/src/config.rs

-                &FilePath::new_unchecked(DEFAULT_CONFIG_FILE)
+                &FilePath::new_unchecked(
+                    dirs::config_dir()
+                        .unwrap()


Won't this panic if the directory doesn't exist. The None case should be handled explicitly.

orecham · 2024-10-24T16:12:02Z

iceoryx2/src/config.rs

    pub fn global_config() -> &'static Config {
        if !ICEORYX2_CONFIG.is_initialized()
            && Config::setup_global_config_from_file(unsafe {
-                &FilePath::new_unchecked(DEFAULT_CONFIG_FILE)
+                &FilePath::new_unchecked(


Using new_unchecked will panic if the path is too long: https://github.com/eclipse-iceoryx/iceoryx2/blob/main/iceoryx2-bb/container/src/byte_string.rs#L284

Since the path length depends on the environment it's not a guarantee this won't happen anymore. Would be better to catch this case, print a warning message, then use the default config.

orecham · 2024-10-24T16:15:07Z

iceoryx2/src/config.rs

@@ -69,22 +69,20 @@
 //! # }
 //! ```

+use dirs;


Rather than using this directly, an abstraction should be implemented.

This allows for swapping out the implementation and will make it easier to isolate for safety certification purposes. I haven't looked at the implementation of dirs closely, but it might not be suitable for certification e.g. if it uses heap.

The cleanest way would probably be to create a "concept" in the concept abstraction layer (iceoryx2-cal) for configuration storage, then add one implementation that uses dirs.

This will allow for:

Swapping out an implementation to a safety-certifiable version

Implementation of the concept for arbitrary environments that don't have a file system

cc. @elfenpiff @elBoberido

orecham · 2024-10-24T16:17:23Z

iceoryx2-cli/iox2-config/src/commands.rs

+}
+
+pub fn show() -> Result<()> {
+    print_system_configuration();


@elfenpiff Will need your feedback on this. What was your intention for this command?

iceoryx2-cli/iox2-config/src/commands.rs

orecham · 2024-10-24T16:30:33Z

iceoryx2/src/config.rs

 use iceoryx2_bb_container::semantic_string::SemanticString;
 use iceoryx2_bb_elementary::lazy_singleton::*;
 use iceoryx2_bb_posix::{file::FileBuilder, shared_memory::AccessMode};
 use iceoryx2_bb_system_types::file_name::FileName;
 use iceoryx2_bb_system_types::file_path::FilePath;
 use iceoryx2_bb_system_types::path::Path;
 use serde::{Deserialize, Serialize};
-use std::time::Duration;
+use std::{os::unix::ffi::OsStrExt, time::Duration};


This will not work on Windows (unix-only).

… + add a subcommand to display the contents of the configuration file

brosier01 added 2 commits October 14, 2024 23:36

[eclipse-iceoryx#432] Implement CLI iox2-config

2bcc183

[eclipse-iceoryx#432] Implement CLI iox2-config

0935fe2

brosier01 closed this Oct 14, 2024

brosier01 deleted the iox2-432-CLI-displaying-full-system-configuration branch October 14, 2024 22:27

brosier01 restored the iox2-432-CLI-displaying-full-system-configuration branch October 14, 2024 22:28

brosier01 reopened this Oct 14, 2024

orecham reviewed Oct 14, 2024

View reviewed changes

orecham assigned brosier01 Oct 14, 2024

orecham reviewed Oct 14, 2024

View reviewed changes

brosier01 and others added 2 commits October 15, 2024 19:40

Update doc/release-notes/iceoryx2-unreleased.md

a087102

Co-authored-by: orecham <[email protected]>

Update doc/release-notes/iceoryx2-unreleased.md

ac89a38

Co-authored-by: orecham <[email protected]>

orecham changed the title ~~[eclipse-iceoryx#432] Implement CLI iox2-config~~ [#432] Implement CLI iox2-config Oct 15, 2024

[eclipse-iceoryx#432] Set default location for default configuration …

973c574

…file

brosier01 force-pushed the iox2-432-CLI-displaying-full-system-configuration branch from ff8654f to 973c574 Compare October 15, 2024 22:28

orecham reviewed Oct 16, 2024

View reviewed changes

brosier01 added 2 commits October 17, 2024 23:36

[eclipse-iceoryx#432] Fix print_system_configuration for cross-platfo…

1ba0dcc

…rm coloring + implementation of user confirmation required before deleting current configuration file

[eclipse-iceoryx#432] Replace the relative path of the configuration …

1ec5fed

…file with the standard configuration path

brosier01 requested a review from orecham October 23, 2024 17:26

orecham reviewed Oct 24, 2024

View reviewed changes

Add an abstraction layer to obtain the path to the configuration file…

e7798a3

… + add a subcommand to display the contents of the configuration file

brosier01 requested a review from orecham November 11, 2024 19:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#432] Implement CLI iox2-config #468

[#432] Implement CLI iox2-config #468

brosier01 commented Oct 14, 2024 •

edited

Loading

elfenpiff commented Oct 14, 2024

orecham left a comment •

edited

Loading

orecham Oct 14, 2024

orecham Oct 14, 2024

brosier01 Oct 15, 2024

orecham Oct 16, 2024

orecham Oct 24, 2024

elfenpiff Oct 30, 2024 •

edited

Loading

orecham Oct 30, 2024

elfenpiff Oct 31, 2024

brosier01 Nov 2, 2024

orecham Nov 2, 2024

orecham left a comment

orecham Oct 16, 2024 •

edited

Loading

elBoberido Oct 16, 2024

orecham Oct 16, 2024

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham left a comment

orecham Oct 24, 2024

orecham Oct 24, 2024

orecham Oct 24, 2024

orecham Oct 24, 2024

orecham Oct 24, 2024

[#432] Implement CLI iox2-config #468

Are you sure you want to change the base?

[#432] Implement CLI iox2-config #468

Conversation

brosier01 commented Oct 14, 2024 • edited Loading

Notes for Reviewer

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Post-review Checklist for the PR Author

References

elfenpiff commented Oct 14, 2024

orecham left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elfenpiff Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

orecham left a comment

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

orecham Oct 16, 2024 • edited Loading

Choose a reason for hiding this comment

orecham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brosier01 commented Oct 14, 2024 •

edited

Loading

orecham left a comment •

edited

Loading

elfenpiff Oct 30, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading

orecham Oct 16, 2024 •

edited

Loading