Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Add Logging Support with Optional Output Formats (JSON, XML, Text) #59

Open
dreamer-coding opened this issue Jan 4, 2025 · 0 comments

Comments

@dreamer-coding
Copy link
Collaborator

dreamer-coding commented Jan 4, 2025

Is your feature request related to a problem? Please describe.

Currently, Fossil Test may lack flexibility when it comes to outputting results in various formats. Different use cases, such as integration with CI/CD pipelines or external monitoring systems, may require structured outputs in formats like JSON or XML. Developers may also want human-readable output by default for easier debugging and readability.

Describe the solution you’d like

Introduce a logging system to Fossil Test that supports different output formats, with text being the default and JSON/XML being optional.

Proposed Features:
1. Default Text Output
• By default, Fossil Test will output results in a human-readable text format, showing a summary of test results, including success, failure, skipped tests, and detailed error messages.
2. CLI Flags to Control Output Format
• --log-format=<text|json|xml>: This flag will let users select the output format for test results.
• text: Human-readable output (default).
• json: Structured JSON output for integration with other systems.
• xml: Structured XML output for legacy systems or other integrations.
3. Test Result Logging
• Each test result should be logged with the following fields:
• Test Name
• Status (success/failure/skipped)
• Duration
• Error Message (if any)
• Timestamp (when the test was executed)
4. Detailed Logging for Failures
• In JSON/XML formats, failures should include more detailed information, such as:
• Test Stack Trace: If available.
• Context: Additional context on why the test failed.
5. Log Output Example
• Text Output Example:

[INFO] Running test suite...
[SUCCESS] test_case_1 executed in 2ms
[FAILURE] test_case_2 executed in 5ms
- Error: Null pointer dereference
[INFO] Total Tests: 2, Passed: 1, Failed: 1
•	JSON Output Example:
{
  "test_suite": "MyTestSuite",
  "total_tests": 2,
  "passed": 1,
  "failed": 1,
  "tests": [
    {
      "name": "test_case_1",
      "status": "success",
      "duration": "2ms",
      "timestamp": "2025-01-05T12:00:00Z"
    },
    {
      "name": "test_case_2",
      "status": "failure",
      "duration": "5ms",
      "error": "Null pointer dereference",
      "timestamp": "2025-01-05T12:00:02Z"
    }
  ]
}
•	XML Output Example:
<test_results>
  <test_suite name="MyTestSuite">
    <total_tests>2</total_tests>
    <passed>1</passed>
    <failed>1</failed>
    <test name="test_case_1">
      <status>success</status>
      <duration>2ms</duration>
      <timestamp>2025-01-05T12:00:00Z</timestamp>
    </test>
    <test name="test_case_2">
      <status>failure</status>
      <duration>5ms</duration>
      <error>Null pointer dereference</error>
      <timestamp>2025-01-05T12:00:02Z</timestamp>
    </test>
  </test_suite>
</test_results>
6.	Log File Support (Optional)
•	Add an option to log results to a file with the following flag:
•	--log-file=<file_path>: Specify the file where the log will be written (supports .txt, .json, or .xml).
7.	Log Level Configuration
•	Allow specifying the log level via a configuration flag (e.g., --log-level=<info|debug|error>), controlling the verbosity of the logs.

Describe alternatives you’ve considered
• Only Text Output: Although simpler, relying solely on text output makes it difficult to integrate Fossil Test results into automated systems.
• JSON-only Output: While JSON would be more structured, it might be less readable for developers during initial debugging, making text output more suitable for local development.

Additional Context
• Use Cases:
• CI/CD Integration: Automated pipelines can parse JSON/XML to create detailed reports and trigger further actions based on test results.
• Debugging: During development, text-based logging can help developers quickly identify issues in the test suite.
• Reporting: Generated logs in JSON or XML format can be used in test management systems or for long-term analysis.
• Implementation Details:
• Use a logging library such as spdlog for flexibility in log formatting and management.
• Ensure the output format is customizable through a config file (fossil_test.ini) or CLI flags.
• The CLI should include clear options for logging format, file output, and log level control.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant