Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ava #10

Open
wants to merge 37 commits into
base: feature/pysigma_version_increase
Choose a base branch
from

Conversation

sharrajesh
Copy link

No description provided.

The commit primarily updates the Poetry dependency management version from 1.5.
This update includes upgrading the aiohttp package from version 3.9.3 to 3.9.4. All associated hashes for different systems and platforms have been changed accordingly.
In the pyproject.toml file, the version constraints for several dependencies including 'pysigma', 'pytest', 'pytest-cov', 'black', 'ruff' and 'langchain' were updated to any version ("*"). In the poetry.lock file, specific versions for 'black' and 'langchain' were updated accordingly. The optional setting for the 'llm' group was changed to false.
Dependencies versions in pyproject.toml and poetry.lock files have been updated to the latest versions. This ensures that the project is using the newest and most secure versions of its dependencies. In addition, all specific versions in poetry dependencies have been changed to "*".
Moved 'langchain', 'openai', 'tiktoken', and 'faiss-cpu' from a specific group to the general dependencies in the pyproject.toml file. This change also resulted in an update to the content-hash in the poetry.lock file.
This commit sets specific versions for the project dependencies in the pyproject.toml file. Instead of using wildcard "*", specific version numbers are used following caret "^" versioning. This allows for better control over dependency versions, ensuring compatibility and avoiding potential conflict with other packages.
The commit introduces a mechanism to instantiate SigmaLLM with specific embedding models rather than using default initialization. This provides flexibility to use different OpenAIEmbeddings or other models without modifying the base class. A fallback to the original mechanism is still maintained to ensure backward compatibility. Further refactoring with regard to the redundancy between 'embedding_model' and 'embedding_function' has been noted for future consideration.
Update SigmAIQ version and improve dependencies

The SigmAIQ version was updated to 0.3.1. Also, dependency versions for several packages including certifi, langchain-core, and packaging were updated. Some of the backends and pipelines in `pySigma` were commented out while others were updated or added.
Fix factory to not use the commented out package.

Bring back Stephen's stix2.

Also other backends besides isightidr.

Update pysigma dependencies and disable InsightIDR Backend

The pysigma dependencies were updated to new versions and two new dependencies were added, pysigma-backend-opensearch and pysigma-backend-qradar-aql. Also, the usage of InsightIDR Backend as a dependency in pyproject.toml and sigmaiq_backend_factory.py was temporarily disabled as it's awaiting resolution.
Temporarily disable InsightIDR pipeline

The InsightIDR pipeline has been temporarily disabled within the sigmaiq_pipeline_factory.py file. The necessary import statement and corresponding dictionary entry for the InsightIDR pipeline have been commented out. This is a temporary change until Stephen uncomment this in the pyproject.toml file.
Enable InsightIDR and downgrade pysigma-backend-cortexxdr version

The code has been updated to re-enable the InsightIDR tool in pyproject.toml, sigmaiq_pipeline_factory.py, and sigmaiq_backend_factory.py files. In addition, the pysigma-backend-cortexxdr version has been downgraded from 0.1.3 to 0.1.2 in pyproject.toml and poetry.lock files to avoid exceptions.
Disabled InsightIDR pipeline in sigmaiq_pipeline_factory.py file

The commit involves commenting out the InsightIDR import statement and its associated dictionary entry in the pipeline factory of sigmaiq. These changes are in line with Stephen's previous modifications in the pyproject.toml file. The InsightIDR pipeline is, hence, temporarily turned off.
Rapid7 InsightIDR SIEM has been commented out from the backend factory. This was done in response to Stephen's action of commenting out the corresponding line in the pyproject.toml file.
The Sigma rule creation guidance in the create_sigma_rule.py script has been updated to provide more detailed instruction on how to create Sigma rules based on user questions and returned Sigma rules. The changes highlight the necessity of being as specific as possible while maintaining applicability in various environments, and mention the possibility of returning multiple Sigma rules if necessary.
Update dependencies and remove test release workflow

This commit updates multiple dependencies including "certifi", "openai", "platformdirs", "pysigma-backend-microsoft365defender", and "tiktoken", to their latest versions ensuring compatibility and the latest features. In addition, it removes the GitHub Actions workflow for releasing tests to PyPI as it is no longer required. Plus, some minor tweaks
This commit updates the version of Poetry used, as well as the versions of several packages. Major changes include the update of the 'annotated-types', 'certifi', and 'coverage' packages. The file hashes have also been updated to match the new versions.
The langchain dependency is updated from version 0.0.347 to version 0.2.9. The dataclasses-json dependency is removed from poetry.lock. Additional changes include updating dependencies of the langchain packages and adding the langchain-text-splitters package.
The anyio package version is updated from 3.7.1 to 4.4.0. Along with this change, the Python versions requirement is increased to >=3.8, with updates to the file hash. Adjustments have also been made to package dependencies and extras, including requirement specification for exceptiongroup and typing-extensions, as well as updates to Sphinx, coverage and trio version in extras.
…ain library, particularly in the _create_subset_model function and similar contexts.

Update anyio version and dependencies

The anyio package version is updated from 3.7.1 to 4.4.0. Along with this change, the Python versions requirement is increased to >=3.8, with updates to the file hash. Adjustments have also been made to package dependencies and extras, including requirement specification for exceptiongroup and typing-extensions, as well as updates to Sphinx, coverage and trio version in extras.
Update package versions and dependencies

The openai and pydantic versions have been updated. The 'langchain-core' version has been downgraded. There's also an added dependency 'annotated-types'. This commit adjusts these dependencies to ensure compatibility and stability.
Update aiohttp to 3.10.0 and add aiohappyeyeballs dependency

Upgraded aiohttp from version 3.9.5 to 3.10.0 to ensure compatibility with newer dependencies and improvements. Added aiohappyeyeballs version 2.3.4 to enhance DNS resolution performance when using asyncio.
langchain and few others.
Update dependencies in poetry.lock

Upgraded various packages including anyio, idna, langchain, openai, platformdirs, and pydantic to their latest versions. Adjusted version constraints for Sphinx, trio, and added new extras for idna package.
Upgraded aiohappyeyeballs from version 2.4.0 to 2.4.2, and aiohttp from version 3.10.5 to 3.10.8. These updates improve performance, fix bugs, and ensure compatibility with the latest features.
Updated versions for aiohappyeyeballs, httpcore, langsmith, openai, and tomli to their latest releases. Also added requests-toolbelt as a new dependency for langsmith. These changes ensure compatibility and include various improvements and bug fixes.
Upgraded aiohttp from version 3.10.8 to 3.10.9. This update includes new wheel files for different platforms, ensuring improved compatibility and bug fixes.
This commit introduces a Makefile to enhance project manageability. It includes a 'help' target for displaying available commands, a 'format' target for formatting Python files with Black, and a 'ruff' target for running the Ruff linter.
Refactor multiple tools to incorporate asyncio for asynchronous execution. Removed unnecessary imports and standardized descriptions across files for consistency and clarity. Updated Makefile to include examples directory.
Refactor multiple tools to incorporate asyncio for asynchronous execution. Removed unnecessary imports and standardized descriptions across files for consistency and clarity. Updated Makefile to include examples directory.
Moved logic into an async main function to handle various rule translations and queries. This change ensures that asynchronous invocations are properly awaited, improving efficiency and readability. Added asyncio.run(main()) entry point for script execution.
Update dependencies in poetry.lock

Upgraded "anyio", "black", and "charset-normalizer" to their latest versions. Adjusted Python version requirements for "anyio" and "black" to >= 3.9. Simplified extras for "black" by adjusting aiohttp dependency.
This update includes a minor version bump from 3.10.9 to 3.10.10 for the aiohttp package. The change primarily involves new file hashes and ensures compatibility and improvements found in the latest release.
Upgraded versions for several packages including langchain-core, langsmith, openai, pytest-cov, tqdm, and yarl. These changes improve compatibility and may include bug fixes and new features. Adjusted dependencies where necessary to support these updates.
Reorganized import statements for better readability and fixed verbose attribute declarations in various tools. Updated dependencies in `poetry.lock` including adding `httpx-sse` and upgrading `jiter` to version `0.7.0`.
Improved code readability by restructuring imports and reformatting function definitions. Updated package versions for `langchain-openai` and `langsmith` to ensure compatibility and improved functionality.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant