Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Threat-274 OCSF data model, CloudTrail #1238

Draft
wants to merge 33 commits into
base: develop
Choose a base branch
from
Draft

Threat-274 OCSF data model, CloudTrail #1238

wants to merge 33 commits into from

Conversation

akozlovets098
Copy link
Contributor

@akozlovets098 akozlovets098 commented May 15, 2024 •
edited
Loading

Background

The Open Cybersecurity Schema Framework (OCSF) is a collaborative, open-source effort by AWS and leading partners in the cybersecurity industry. OCSF provides a standard schema for common security events, defines versioning criteria to facilitate schema evolution, and includes a self-governance process for security log producers and consumers. Having our AWS rules work with OCSF log could have a great value for customers who will use OCSF.

Changes

  • Added data models for OCSF.AccountChange, OCSF.ApiActivity, OCSF.Authentication log types
  • Updated data model for AWS.CloudTrail
  • Updated AWS CloudTrail rules to work with data models
  • Added unit tests for OCSF logs
  • Reformatted unit tests yaml -> json

Testing

See unit tests updates

melenevskyi and others added 29 commits April 11, 2024 14:33
@melenevskyi @arielkr256
Deprecate GreyNoise detections (#1205)
8b4be20 
* Deprecate GreyNoise detections

* Update rules/aws_cloudtrail_rules/aws_s3_activity_greynoise.yml

* Update rules/cloudflare_rules/cloudflare_firewall_suspicious_event_greynoise.yml

* Update cloudflare_httpreq_bot_high_volume_greynoise.yml

---------

Co-authored-by: Ariel Ropek <[email protected]>
@akozlovets098
fix - Notion Login From New Location - NoneType error (#1206)
319fd15 
* fix - Notion Login From New Location - NoneType error

* fix - Notion Login From New Location - NoneType error - linter fix
@le4ker
remove codeowners (#1208)
3fde677
@akozlovets098
fix - GCP rules - AttributeError (#1210)
9239b29 
* fix - GCP rules - AttributeError

* fix - GCP rules - AttributeError - linter fix
@ben-githubs @arielkr256
MITRE ATT&CK Mappings for MS Rules (#1209)
f1180d4 
* added MITRE mappings for microsoft rules

* fixed formatting on some helper files

---------

Co-authored-by: Ariel Ropek <[email protected]>
@arielkr256 @le4ker @melenevskyi
traildiscover enrichment with managed schema (#1177)
6eeb515 
* traildiscover enrichment with managed schema

* Add npm install in dockerfile (#1172)

* add npm install in dockerfile

* Remove Python optimizations; add prettier to PATH

---------

Co-authored-by: egibs <[email protected]>

* schema name: TrailDiscover.CloudTrail

* Fix Dockerfile; add Workflow to test image

* updated data set

* Add MongoDB.2FA.Disabled rule (#1190)

Co-authored-by: Ariel Ropek <[email protected]>

* lint and fmt

* fmt

* add OCSF selector

* additional OCSF mappings

* Fix Pipfile

* Rebase changes

---------

Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Oleh Melenevskyi <[email protected]>
Update PAT to 0.46.0 (#1216)
e2e9b4f
Replace panther_analysis_tool import with updated import (#1230)
cd042d2
Update Action versions; use SHAs (#1231)
5e5f196 
* Update Action versions; use SHAs

* Add dependabot.yml to keep Actions updated

* Update PAT to 0.49.0
@arielkr256
migrates the gcp_storage_hmac_keys_create rule to (#1233)
0f28285 
python from sdyaml
@arielkr256
move scheduled rules to the queries directory (#1234)
83e6d74
@kjihso
consistency nit fixes (#1235)
575cf47 
* consistency nit fixes

* - somethings -> some things
@jzandona @melenevskyi
@arielkr256 @akozlovets098 @le4ker @ben-githubs @nhakmiller
AppOmni Alert passthrough (#1211)
c8b6ad9 
* alert passthrough

* Deprecate GreyNoise detections (#1205)

* Deprecate GreyNoise detections

* Update rules/aws_cloudtrail_rules/aws_s3_activity_greynoise.yml

* Update rules/cloudflare_rules/cloudflare_firewall_suspicious_event_greynoise.yml

* Update cloudflare_httpreq_bot_high_volume_greynoise.yml

---------

Co-authored-by: Ariel Ropek <[email protected]>

* fix - Notion Login From New Location - NoneType error (#1206)

* fix - Notion Login From New Location - NoneType error

* fix - Notion Login From New Location - NoneType error - linter fix

* remove codeowners (#1208)

* linting

* fix - GCP rules - AttributeError (#1210)

* fix - GCP rules - AttributeError

* fix - GCP rules - AttributeError - linter fix

* MITRE ATT&CK Mappings for MS Rules (#1209)

* added MITRE mappings for microsoft rules

* fixed formatting on some helper files

---------

Co-authored-by: Ariel Ropek <[email protected]>

* traildiscover enrichment with managed schema (#1177)

* traildiscover enrichment with managed schema

* Add npm install in dockerfile (#1172)

* add npm install in dockerfile

* Remove Python optimizations; add prettier to PATH

---------

Co-authored-by: egibs <[email protected]>

* schema name: TrailDiscover.CloudTrail

* Fix Dockerfile; add Workflow to test image

* updated data set

* Add MongoDB.2FA.Disabled rule (#1190)

Co-authored-by: Ariel Ropek <[email protected]>

* lint and fmt

* fmt

* add OCSF selector

* additional OCSF mappings

* Fix Pipfile

* Rebase changes

---------

Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Oleh Melenevskyi <[email protected]>

* Update PAT to 0.46.0 (#1216)

* add file/host state to msft graph alert context (#1220)

* fix timestamps (#1219)

* Update PAT to 0.46.1 (#1222)

* pack for traildiscover LUT (#1221)

* use event.deep_get and remove InlineFilters

* add pack

---------

Co-authored-by: Oleh Melenevskyi <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
Co-authored-by: akozlovets098 <[email protected]>
Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: ben-githubs <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Evan Gibler <[email protected]>
Co-authored-by: Nick Hakmiller <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
@akozlovets098
Merge remote-tracking branch 'origin/release' into release
1cb6695 
# Conflicts:
#	.github/CODEOWNERS
#	Pipfile
#	Pipfile.lock
@jstanulis-push @melenevskyi
@arielkr256 @akozlovets098 @le4ker @ben-githubs @nhakmiller
Push Security rules (#1207)
8012f11 
* Deprecate GreyNoise detections (#1205)

* Deprecate GreyNoise detections

* Update rules/aws_cloudtrail_rules/aws_s3_activity_greynoise.yml

* Update rules/cloudflare_rules/cloudflare_firewall_suspicious_event_greynoise.yml

* Update cloudflare_httpreq_bot_high_volume_greynoise.yml

---------

Co-authored-by: Ariel Ropek <[email protected]>

* fix - Notion Login From New Location - NoneType error (#1206)

* fix - Notion Login From New Location - NoneType error

* fix - Notion Login From New Location - NoneType error - linter fix

* Push Security rules

* remove codeowners (#1208)

* fix - GCP rules - AttributeError (#1210)

* fix - GCP rules - AttributeError

* fix - GCP rules - AttributeError - linter fix

* MITRE ATT&CK Mappings for MS Rules (#1209)

* added MITRE mappings for microsoft rules

* fixed formatting on some helper files

---------

Co-authored-by: Ariel Ropek <[email protected]>

* traildiscover enrichment with managed schema (#1177)

* traildiscover enrichment with managed schema

* Add npm install in dockerfile (#1172)

* add npm install in dockerfile

* Remove Python optimizations; add prettier to PATH

---------

Co-authored-by: egibs <[email protected]>

* schema name: TrailDiscover.CloudTrail

* Fix Dockerfile; add Workflow to test image

* updated data set

* Add MongoDB.2FA.Disabled rule (#1190)

Co-authored-by: Ariel Ropek <[email protected]>

* lint and fmt

* fmt

* add OCSF selector

* additional OCSF mappings

* Fix Pipfile

* Rebase changes

---------

Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Oleh Melenevskyi <[email protected]>

* Update PAT to 0.46.0 (#1216)

* add file/host state to msft graph alert context (#1220)

* fix timestamps (#1219)

* Update PAT to 0.46.1 (#1222)

* pack for traildiscover LUT (#1221)

* pack, fmt lint, event.deep_get

* pack update

---------

Co-authored-by: Oleh Melenevskyi <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
Co-authored-by: akozlovets098 <[email protected]>
Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: ben-githubs <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Evan Gibler <[email protected]>
Co-authored-by: Nick Hakmiller <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
@arielkr256
created pack and updated event.deep_get (#1239)
1252a70
@arielkr256
Push logtype update (#1240)
63db6ce 
* created pack and updated event.deep_get

* update logtype
@egibs
Remove Node/NPM/Prettier (#1241)
442849c 
* Remove Node/NPM/Prettier

Signed-off-by: egibs <[email protected]>

* Update README; add removal notes

Signed-off-by: egibs <[email protected]>

---------

Signed-off-by: egibs <[email protected]>
@akozlovets098
Merge remote-tracking branch 'origin/release' into release
309c401
@egibs
Small Workflow tweaks (#1243)
c8b23bd 
Signed-off-by: egibs <[email protected]>
@egibs
Use harden-runner Action for all Workflows (#1244)
dc7070c 
* Use harden-runner Action for all Workflows

Signed-off-by: egibs <[email protected]>

* Run Docker Workflow

Signed-off-by: egibs <[email protected]>

* Add blocking policy for docker.yml

Signed-off-by: egibs <[email protected]>

* Add permissions to Workflow

Signed-off-by: egibs <[email protected]>

* More permissions

Signed-off-by: egibs <[email protected]>

---------

Signed-off-by: egibs <[email protected]>
@akozlovets098 @melenevskyi
@arielkr256 @le4ker @ben-githubs @egibs
Threat 319 Replace geoinfo_from_ip with new version (#1242)
736c250 
* Deprecate GreyNoise detections (#1205)

* Deprecate GreyNoise detections

* Update rules/aws_cloudtrail_rules/aws_s3_activity_greynoise.yml

* Update rules/cloudflare_rules/cloudflare_firewall_suspicious_event_greynoise.yml

* Update cloudflare_httpreq_bot_high_volume_greynoise.yml

---------

Co-authored-by: Ariel Ropek <[email protected]>

* fix - Notion Login From New Location - NoneType error (#1206)

* fix - Notion Login From New Location - NoneType error

* fix - Notion Login From New Location - NoneType error - linter fix

* remove codeowners (#1208)

* fix - GCP rules - AttributeError (#1210)

* fix - GCP rules - AttributeError

* fix - GCP rules - AttributeError - linter fix

* MITRE ATT&CK Mappings for MS Rules (#1209)

* added MITRE mappings for microsoft rules

* fixed formatting on some helper files

---------

Co-authored-by: Ariel Ropek <[email protected]>

* traildiscover enrichment with managed schema (#1177)

* traildiscover enrichment with managed schema

* Add npm install in dockerfile (#1172)

* add npm install in dockerfile

* Remove Python optimizations; add prettier to PATH

---------

Co-authored-by: egibs <[email protected]>

* schema name: TrailDiscover.CloudTrail

* Fix Dockerfile; add Workflow to test image

* updated data set

* Add MongoDB.2FA.Disabled rule (#1190)

Co-authored-by: Ariel Ropek <[email protected]>

* lint and fmt

* fmt

* add OCSF selector

* additional OCSF mappings

* Fix Pipfile

* Rebase changes

---------

Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Oleh Melenevskyi <[email protected]>

* Update PAT to 0.46.0 (#1216)

* THREAT-319 Replace geoinfo_from_ip with new version

---------

Co-authored-by: Oleh Melenevskyi <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
Co-authored-by: Panos Sakkos <[email protected]>
Co-authored-by: ben-githubs <[email protected]>
Co-authored-by: egibs <[email protected]>
Co-authored-by: Evan Gibler <[email protected]>
Co-authored-by: Evan Gibler <[email protected]>
@egibs
Use full Action SHAs rather than versioned releases (#1245)
cec5c8c 
Signed-off-by: egibs <[email protected]>
@arielkr256
auth0-cic-credential-stuffing rule and query (#1246)
ca6f7de
@egibs
Merge branch 'main' into release
7eed675 
Signed-off-by: egibs <[email protected]>
@egibs
Update panther-core to 0.10.1 via PAT (#1249)
12ff27b 
Signed-off-by: egibs <[email protected]>
@akozlovets098
Merge remote-tracking branch 'origin/release' into release
c331931
@akozlovets098
THREAT-278 OCSF data model, CloudTrail
88becd1
@akozlovets098
THREAT-278 OCSF data model, CloudTrail - update deps, fix errors
63439fc
@akozlovets098 akozlovets098 force-pushed the THREAT-274-OCSF-data-model,-CloudTrail branch from 51d8db3 to 63439fc Compare June 4, 2024 12:28
@akozlovets098 akozlovets098 force-pushed the THREAT-274-OCSF-data-model,-CloudTrail branch from 7b1f499 to 40e9e64 Compare June 4, 2024 12:52
akozlovets098 and others added 3 commits June 4, 2024 16:02
@akozlovets098
THREAT-278 OCSF data model, CloudTrail - added auth0 rule to pack
1ddcd0e
@egibs @arielkr256
Tweak Snowflake queries (#1250)
aa5ae8b 
* Tweak Snowflake queries

Signed-off-by: egibs <[email protected]>

* Remove configuration drift query from Pack

Signed-off-by: egibs <[email protected]>

* Threat Hunting queries are okay

Signed-off-by: egibs <[email protected]>

* Fix comment Workflow

Signed-off-by: egibs <[email protected]>

* 12 hours -> 1 day

Signed-off-by: egibs <[email protected]>

* Update queries/snowflake_queries/snowflake_0108977_configuration_drift.yml

---------

Signed-off-by: egibs <[email protected]>
Co-authored-by: Ariel Ropek <[email protected]>
@egibs
Merge branch 'release' into THREAT-274-OCSF-data-model,-CloudTrail
a854c16
@akozlovets098 akozlovets098 marked this pull request as ready for review June 5, 2024 08:15
@akozlovets098 akozlovets098 requested a review from a team as a code owner June 5, 2024 08:15
@arielkr256 arielkr256 added the enhancement New feature or request label Sep 4, 2024
@arielkr256 arielkr256 added enhancement New feature or request and removed enhancement New feature or request labels Sep 11, 2024
@arielkr256 arielkr256 marked this pull request as draft September 25, 2024 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@akozlovets098 @arielkr256 @melenevskyi @le4ker @ben-githubs @kjihso @jzandona @jstanulis-push @egibs