Audit Log Collector for PRTG use

[SWA84]

Hello,

I created the Azure App for die Audit Log and the sensor for PRTG.
If i start the sensor I get following error:

XML: Das zurückgelieferte XML entspricht nicht dem erwarteten Schema. (Code: PE233) -- JSON: Das zurückgelieferte JSON entspricht nicht der erwarteten Struktur (Invalid JSON.). (Code: PE231)

XML: The returned XML does not conform to the expected schema. (Code: PE233) -- JSON: The returned JSON does not match the expected structure (Invalid JSON.). (Code: PE231)

Following is done like in the description:

Make sure Auditing is turned on for your tenant!
Use these instructions: https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off?view=o365-worldwide
If you had to turn it on, it may take a few hours to process
Create App registration:
Azure AD > 'App registrations' > 'New registration':
Choose any name for the registration
Choose "Accounts in this organizational directory only (xyz only - Single tenant)"
Hit 'register'
Save 'Tenant ID' and 'Application (Client) ID' from the overview page of the new registration, you will need it to run the collector
Create app secret:
Azure AD > 'App registrations' > Click your new app registration > 'Certificates and secrets' > 'New client secret':
Choose any name and expire date and hit 'add'
Actual key is only shown once upon creation, store it somewhere safe. You will need it to run the collector.
Grant your new app registration 'application' permissions to read the Office API's:
Azure AD > 'App registrations' > Click your new app registration > 'API permissions' > 'Add permissions' > 'Office 365 Management APIs' > 'Application permissions':
Check 'ActivityFeed.Read'
Check 'ActivityFeed.ReadDlp'
Hit 'Add permissions'

But where I have to do following:
Subscribe to audit log feeds of your choice
Set 'autoSubscribe: True' in YAML config file to automate this.
OR Use the '--interactive-subscriber' parameter when executing the collector to manually subscribe to the audit API's of your choice

Its necessery to running the collector if I use it only for PRTG?

In PRTG I did following:

optional) Creating a PRTG sensor
To run with PRTG you must create a sensor:

Copy the OfficeAuditLogCollector.exe executable to the "\Custom Sensors\EXE" sub folder of your PRTG installation

Create a device in PRTG with any host name (e.g. "Office Audit Logs")

Create a 'EXE/Script Advanced Sensor' on that device and choose the executable you just copied

Enter parameters, e.g.: "tenant_id client_key secret_key --config full/path/to/config.yaml" (use full path, because PRTG will execute the script from a different working directory) I used the local path from the PRTG probe... correct?

Copy the prtg.config from ConfigExamples and modify at least the channel names and filters for your needs. (I used the prtg example file)

Set the timeout of the script to something generous that suits the amount of logs you will retrieve. Probably at least 300 seconds. Run the script manually first to check how long it takes.

Match the interval of the sensor to the amount of hours of logs to retrieve. If your interval is 1 hour, hoursToCollect in the config file should also be set to one hour.

I tried also to run the officeAuditLogCollector-V2.3.exe with cmd (admin rights) with
that path

C:\Skripte\officeAuditLogCollector-V2.3.exe xxxxxx-xxxxx-xxxxx-xxxxx-xxxxxxxxxxx xxxxxxxx-xxxx-xxxxx-xxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxx --config C:\Skripte\PRTG_365_Audit_Logs.yaml

A collector.log will be created with following text:
Starting run @ 2022-08-22 08:56:12.615785. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.SharePoint']).

The file is 0KB. After 4 hours the same. Nothing happens?!?

[ddbnl]

Heya,

The steps look good, perhaps it is something in the config. Could you paste the content of the config yaml file here? I will try to reproduce the issue.

[SWA84]

Hello, its the default PRTG config file from your examples: ***@***.*** Mit freundlichen Grüßen Stefan Wachsmuth staatl. geprüfter IT-Systemelektroniker IT-Systemingenieur Bitte senden Sie allgemeine IT-Service-Anfragen immer an ***@***.*** oder nutzen Sie die IT-Service-Hotline: +49 (561) 8792-190. HERMANNS HTI-Bau GmbH u. Co. KG Wilhelm-Speck-Str. 17 34125 Kassel Deutschland Telefon +49 (561) 8792-189 Mobil +49 (151) 61588137 E-Mail ***@***.*** ***@***.***> Internet www.hermanns.de Kontaktieren Sie mich über ***@***.***> Kommanditgesellschaft HRA 7346 Kassel pers. haft. Gesellschafterin: HERMANNS HTI-Bau und Geschäftsführungs GmbH Amtsgericht Kassel HRB 3468, USt.-Id.-Nr.: DE 113024495 Geschäftsführer: Hans-Ulrich Hujer, Swen Haar Ein Unternehmen der HERMANNS-Gruppe. Sofern nicht anders angegeben, ist diese E-Mail und alle Anlagen vertraulich und nur für den Empfänger bestimmt. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhaltes, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Sparen Sie pro nicht gedruckter Seite 250 ml Wasser, 5 g CO2, 15 g Holz und 50 Wh Energie. Von: ddbnl ***@***.***> Gesendet: Samstag, 27. August 2022 10:24 An: ddbnl/office365-audit-log-collector ***@***.***> Cc: Wachsmuth, Stefan ***@***.***>; Author ***@***.***> Betreff: Re: [ddbnl/office365-audit-log-collector] Audit Log Collector for PRTG use (Discussion #35) Heya, The steps look good, perhaps it is something in the config. Could you paste the content of the config yaml file here? I will try to reproduce the issue. — Reply to this email directly, view it on GitHub<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL2RkYm5sL29mZmljZTM2NS1hdWRpdC1sb2ctY29sbGVjdG9yL2Rpc2N1c3Npb25zLzM1I2Rpc2N1c3Npb25jb21tZW50LTM0ODg0NDA=&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=TkNERk5ud29KL3hxV1NwVTRvaVNjaG1PZ1ViZ3pQZ0JNOFpyRDhramttST0=&h=2c636a0e99ef4edb9401d9f858c2a3a7>, or unsubscribe<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL25vdGlmaWNhdGlvbnMvdW5zdWJzY3JpYmUtYXV0aC9BMlVPNTdQVzZVUDNIVzNCRVBUT1lKVFYzSEdESkFOQ05GU001N0hLSFREQQ==&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=eEZRUUhQWHNrNHNRa0J5cmR0U2NxamhOM2xoTktmQXdpQjNhNUV2aXlEZz0=&h=2c636a0e99ef4edb9401d9f858c2a3a7>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ddbnl]

Sorry for the late reply, life got in the way.

I believe you may indeed not be subscribed to the audit log feeds yet. This is done automatically if the 'autoSubscribe' parameter is set to true. This is something I should have included in the default PRTG example; I will make a new commit to fix that mistake. I will add a new default config for you below, could you let me know if that works for you? The below config also has the log setting "debug: False"; if you still experience issues, please set this to "True" and provide the log so I can debug further. I think it'll work with auto subsribe however.

log:  # Log settings. Debug will severely decrease performance
  path: 'collector.log'
  debug: False
collect:
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.SharePoint: True
  skipKnownLogs: False  # Take all logs each time to count the number of active filter hits each interval
  resume: False  # Take all logs each time to count the number of active filter hits each interval
  hoursToCollect: 1  # Period over which to alert, e.g. failed AAD logins over the last hour
  autoSubscribe: True
# The PRTG output defines channels which have filters associated to them. The output of the channel will be
# the number of hits on the filter. E.g. filter for failed AAD logins on a "Failed AAD logins" channel.
output:
  prtg:
    enabled: True
    channels:
    - name: Deleted Sharepoint files
      filters:
        Audit.SharePoint:
          Operation: FileDeleted
    - name: Failed Azure AD logins
      filters:
        Audit.AzureActiveDirectory:
          Operation: UserLoginFailed
    - name: Spoof attempts prevented
      filters:
        Audit.General:
          Policy: Spoof

[SWA84]

Thanks for your reply! When I start the officeAuditLogCollector-V2.3.exe with the following command officeAuditLogCollector-V2.3.exe TenantID ClientID Secret --config C:\Skripte\PRTG_365_Audit_Logs.yaml It works. This command I set up as Parameter in PRTG I got the same error like before: XML: Das zurückgelieferte XML entspricht nicht dem erwarteten Schema. (Code: PE233) -- JSON: Das zurückgelieferte JSON entspricht nicht der erwarteten Struktur (Invalid JSON.). (Code: PE231) How I have to set the path of the config file in the PRTG command? I used the local path oft he PRTG probe server. Best regards. Stefan Wachsmuth Von: ddbnl ***@***.***> Gesendet: Sonntag, 4. September 2022 13:28 An: ddbnl/office365-audit-log-collector ***@***.***> Cc: Wachsmuth, Stefan ***@***.***>; Author ***@***.***> Betreff: Re: [ddbnl/office365-audit-log-collector] Audit Log Collector for PRTG use (Discussion #35) Sorry for the late reply, life got in the way. I believe you may indeed not be subscribed to the audit log feeds yet. This is done automatically if the 'autoSubscribe' parameter is set to true. This is something I should have included in the default PRTG example; I will make a new commit to fix that mistake. I will add a new default config for you below, could you let me know if that works for you? The below config also has the log setting "debug: False"; if you still experience issues, please set this to "True" and provide the log so I can debug further. I think it'll work with auto subsribe however. log: # Log settings. Debug will severely decrease performance path: 'collector.log' debug: False collect: contentTypes: Audit.General: True Audit.AzureActiveDirectory: True Audit.SharePoint: True skipKnownLogs: False # Take all logs each time to count the number of active filter hits each interval resume: False # Take all logs each time to count the number of active filter hits each interval hoursToCollect: 1 # Period over which to alert, e.g. failed AAD logins over the last hour autoSubscribe: True # The PRTG output defines channels which have filters associated to them. The output of the channel will be # the number of hits on the filter. E.g. filter for failed AAD logins on a "Failed AAD logins" channel. output: prtg: enabled: True channels: - name: Deleted Sharepoint files filters: Audit.SharePoint: Operation: FileDeleted - name: Failed Azure AD logins filters: Audit.AzureActiveDirectory: Operation: UserLoginFailed - name: Spoof attempts prevented filters: Audit.General: Policy: Spoof — Reply to this email directly, view it on GitHub<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL2RkYm5sL29mZmljZTM2NS1hdWRpdC1sb2ctY29sbGVjdG9yL2Rpc2N1c3Npb25zLzM1I2Rpc2N1c3Npb25jb21tZW50LTM1NDczNDE=&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=YlRNdlpDRXE1Y0NybWxlblFDV2lqdjg0Mm9Oam1sMHl5bHBBNS92ZmVLdz0=&h=10cb1275712443609ec05ecf08ac96ad>, or unsubscribe<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL25vdGlmaWNhdGlvbnMvdW5zdWJzY3JpYmUtYXV0aC9BMlVPNTdQWU1aWkk3RkgyMkRQVTZHVFY0U0JWWEFOQ05GU001N0hLSFREQQ==&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=dm1QL2hzNXd2NjZJRFVISXZ5R0hYaWd3OTJZR2RWNkhxV3E5Sk1vRHJTcz0=&h=10cb1275712443609ec05ecf08ac96ad>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[ddbnl]

Hi Stefan,

I've tried to reproduce the issue but haven't been able to thus far. The path of the config file should be a full local path, based on the server that is executing the script. So if the script is executed on the probe is should look like e.g. "C:\path\to\config.yaml", and that path is on the probe.

If this is already configured correctly, could you run the script manually where it functions correctly, and copy paste the output here? The output should contain no personal data, and it is so I can verify that the output is in the correct format.

Finally: does the "log" section under the sensor details give any additional information perhaps?

[SWA84]

Hello, I run that skript with cmd with that command: (I changed the IDs and the secret in that mail) C:\Skripte>officeAuditLogCollector-V2.3.exe TentantID CleintID AppSecret --config C:\Skripte\PRTG_365_Audit_Logs.yaml {"prtg": {"text": "OK", "result": [{"Channel": "Failed Azure AD logins", "Value": 4, "Unit": "Count", "SpeedSize": "One", "VolumeSize": "One", "SpeedTime": "Second", "Mode": "Absolute"}, {"Channel": "Deleted Sharepoint files", "Value": 2, "Unit": "Count", "SpeedSize": "One", "VolumeSize": "One", "SpeedTime": "Second", "Mode": "Absolute"}, {"Channel": "Spoof attempts prevented", "Value": 0, "Unit": "Count", "SpeedSize": "One", "VolumeSize": "One", "SpeedTime": "Second", "Mode": "Absolute"}]}} Ist created a collector.log Making API request using URL: https://manage.office.com/api/v1.0/xxxxxxxxxxxxxxxxxxxxxxxxxxxxx/activity/feed/subscriptions/list Starting new HTTPS connection (1): login.microsoftonline.com:443 https://login.microsoftonline.com:443 "POST /xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/oauth2/token HTTP/1.1" 200 1482 Logged in Starting new HTTPS connection (1): manage.office.com:443 https://manage.office.com:443 "GET /api/v1.0/cab63b61-be32-4087-86aa-299cb30147b7/activity/feed/subscriptions/list HTTP/1.1" 200 215 Starting run @ 2022-09-08 13:29:41.512109. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.SharePoint']). Rust engine finished receiving all content Finished. Total logs retrieved: 8145. Total retries: 0. Total logs with errors: 0. Run time: 0:00:03.343781. PRTGInterface reports: 0 successfully sent, 0 errors Von: ddbnl ***@***.***> Gesendet: Donnerstag, 8. September 2022 12:31 An: ddbnl/office365-audit-log-collector ***@***.***> Cc: Wachsmuth, Stefan ***@***.***>; Author ***@***.***> Betreff: Re: [ddbnl/office365-audit-log-collector] Audit Log Collector for PRTG use (Discussion #35) Hi Stefan, I've tried to reproduce the issue but haven't been able to thus far. The path of the config file should be a full local path, based on the server that is executing the script. So if the script is executed on the probe is should look like e.g. "C:\path\to\config.yaml", and that path is on the probe. If this is already configured correctly, could you run the script manually where it functions correctly, and copy paste the output here? The output should contain no personal data, and it is so I can verify that the output is in the correct format. Finally: does the "log" section under the sensor details give any additional information perhaps? — Reply to this email directly, view it on GitHub<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL2RkYm5sL29mZmljZTM2NS1hdWRpdC1sb2ctY29sbGVjdG9yL2Rpc2N1c3Npb25zLzM1I2Rpc2N1c3Npb25jb21tZW50LTM1OTMzOTM=&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=TzR5aHM3cEFtR2pyTC9zVlNaMHMyYjN1L3Z1YWx6cjJwelBlcTM4UFFOOD0=&h=0639d8a85b624598a7bf2ce134502086>, or unsubscribe<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL25vdGlmaWNhdGlvbnMvdW5zdWJzY3JpYmUtYXV0aC9BMlVPNTdKNTdLNjY1UUdZUEFDRzRBRFY1RzU0M0FOQ05GU001N0hLSFREQQ==&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=aGh6cmhic1VHUHJsQVRWZE5zU2dTSTkvVG5Bb3MybHpBV01DcXBYdlozRT0=&h=0639d8a85b624598a7bf2ce134502086>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[SWA84]

Hello, do you have any new informations? Mit freundlichen Grüßen Stefan Wachsmuth Von: ddbnl ***@***.***> Gesendet: Donnerstag, 8. September 2022 12:31 An: ddbnl/office365-audit-log-collector ***@***.***> Cc: Wachsmuth, Stefan ***@***.***>; Author ***@***.***> Betreff: Re: [ddbnl/office365-audit-log-collector] Audit Log Collector for PRTG use (Discussion #35) Hi Stefan, I've tried to reproduce the issue but haven't been able to thus far. The path of the config file should be a full local path, based on the server that is executing the script. So if the script is executed on the probe is should look like e.g. "C:\path\to\config.yaml", and that path is on the probe. If this is already configured correctly, could you run the script manually where it functions correctly, and copy paste the output here? The output should contain no personal data, and it is so I can verify that the output is in the correct format. Finally: does the "log" section under the sensor details give any additional information perhaps? — Reply to this email directly, view it on GitHub<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL2RkYm5sL29mZmljZTM2NS1hdWRpdC1sb2ctY29sbGVjdG9yL2Rpc2N1c3Npb25zLzM1I2Rpc2N1c3Npb25jb21tZW50LTM1OTMzOTM=&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=TzR5aHM3cEFtR2pyTC9zVlNaMHMyYjN1L3Z1YWx6cjJwelBlcTM4UFFOOD0=&h=0639d8a85b624598a7bf2ce134502086>, or unsubscribe<https://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL25vdGlmaWNhdGlvbnMvdW5zdWJzY3JpYmUtYXV0aC9BMlVPNTdKNTdLNjY1UUdZUEFDRzRBRFY1RzU0M0FOQ05GU001N0hLSFREQQ==&i=NjJjNmRmY2E3MGYyYWIxMDIxMjQ0NDA0&t=aGh6cmhic1VHUHJsQVRWZE5zU2dTSTkvVG5Bb3MybHpBV01DcXBYdlozRT0=&h=0639d8a85b624598a7bf2ce134502086>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[SWA84]

Hello,

the sensor in PRTG still doesnt work. Is it correct, that in the collector.log file are PRTG Interface reports are 0?

Finished. Total logs retrieved: 9215. Total retries: 0. Total logs with errors: 0. Run time: 0:00:03.250010.
PRTGInterface reports: 0 successfully sent, 0 errors