Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question regarding RetrievIR configs file #3

Open
snolroy opened this issue Aug 21, 2024 · 40 comments
Open

Question regarding RetrievIR configs file #3

snolroy opened this issue Aug 21, 2024 · 40 comments
Assignees
Labels
bug Something isn't working question Further information is requested

Comments

@snolroy
Copy link

snolroy commented Aug 21, 2024

Hi @joeavanzato
I have a question. Do you have any instructions on how to edit the configs.json files to exclude some of the items that are collected lets say by commands/objective or categories ?
Somehow when I try using the -category argument on a remote system I get an error saying that
"A command that prompts the user failed because the host program or the command type does not support user interaction. The host was attempting to request confirmation with the following message:" Then I dont even see the message.

@joeavanzato
Copy link
Owner

joeavanzato commented Aug 21, 2024

Can you provide the full command-line you are attempting to launch it with/what you are trying to achieve with more details so I can try to replicate?

Are you trying to add new commands?

If you simply want to exclude things, I would just delete them - or you could add a tag to each objective that you DO want and just invoke with that tag.

@joeavanzato joeavanzato self-assigned this Aug 21, 2024
@joeavanzato joeavanzato added the question Further information is requested label Aug 21, 2024
@snolroy
Copy link
Author

snolroy commented Aug 21, 2024

Okay I think the tag makes sense. What file would I need to add the tag to ?

Also, another quick question. I am getting this error when I try to run the script on a remote host

Error: Get-Content : Cannot find path 'C:\configs\Export-MFT.ps1' because it does not exist.At C:\TmpRT\RetrievIR-1.0\RetrievIR.ps1:1126 char:28+

Command Ran via EDR:
powershell.exe -ExecutionPolicy Bypass -File "C:\TmpRT\RetrievIR-1.0\RetrievIR.ps1" -config "C:\TmpRT\RetrievIR-1.0\configs\*" -evidence_dir "C:\TmpRT\RetrievIR-1.0\evidence\" -tags "sans_triage"

@joeavanzato
Copy link
Owner

RetrievIR is powered by the JSON configuration files - each of those files contains 1 or more 'directives' that specify files to collect, commands to execute or registry values to collect. Each 'directive' can contain one or more 'tags' that allow for this type of filtering of directive at run-time - see https://github.com/joeavanzato/RetrievIR/blob/main/configs/antivirus.json for example and look at the 'tags' key in each directive - these can be edited to add or remove different 'tags' to directives.

RE that error - this is because the directory 'configs' is currently hard-coded into

"command": "file:configs\\Export-MFT.ps1",
- I will adjust this with a new version shortly that allows for dynamic configuration directory replacement in the configuration itself, so this is a real 'bug'.

@joeavanzato
Copy link
Owner

@snolroy I just pushed an update to RetrievIR.ps1 and the configuration file 'windows.json' - please replace both of these and attempt to execute again and the problem should be resolved.

I would also recommend modifying your command to the following:
powershell.exe -ExecutionPolicy Bypass -File "C:\TmpRT\RetrievIR-1.0\RetrievIR.ps1" -config "C:\TmpRT\RetrievIR-1.0\configs" -evidence_dir "C:\TmpRT\RetrievIR-1.0\evidence" -tags "sans_triage"

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

Thank you very much. I will try and report findings.

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

Hi @joeavanzato

With the change you made the RetrievIR script now works.

Now I am having a similar issue running the parser remotely. I am getting an error and it stops running but the logs do not even show any signs of errors..
See details below:
Command ran:
powershell.exe -ExecutionPolicy Bypass -File "C:\TmpRT\RetrievIR-1.1\ParseIR.ps1" -config "C:\TmpRT\RetrievIR-1.1\parsing_config.json" -evidence_dir "C:\TmpRT\RetrievIR-1.1\evidence" -utilities_dir "C:\TmpRT\RetrievIR-1.1\utilities" -ignoremissing

Error thrown in my terminal:
Invoke-WebRequest : { "code": "not_found", "message": "File with such name does not exist.", "status": 404 }At C:\TmpRT\RetrievIR-1.1\ParseIR.ps1:144 char:9+ Invoke-WebRequest "$url" -OutFile $zip_path+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc eption + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Details from the logs
2024/08/22 10:06:45 - [!] Starting Evidence Parsing...
2024/08/22 10:06:45 - [+] Using Configuration: C:\TmpRT\RetrievIR-1.1\parsing_config.json
2024/08/22 10:06:45 - [+] Using Evidence Directory: C:\TmpRT\RetrievIR-1.1\evidence
2024/08/22 10:06:45 - [+] Reading Configuration Data...
2024/08/22 10:06:45 - [!] Configuration Validated!
2024/08/22 10:06:45 - [!] Reading File Copy CSV: C:\TmpRT\RetrievIR-1.1\evidence\successful_file_copies.csv
2024/08/22 10:06:45 - [!] Found Binary: AmcacheParser.exe
2024/08/22 10:06:45 - [!] Found Binary: AppCompatCacheParser.exe
2024/08/22 10:06:45 - [!] Found Binary: JLECmd.exe
2024/08/22 10:06:45 - [!] Found Binary: LECmd.exe
2024/08/22 10:06:45 - [+] Downloading PECmd.exe from https://f001.backblazeb2.com/file/EricZimmermanTools/PECmd.zip
2024/08/22 10:06:46 - [!] Found Binary: RBCmd.exe
2024/08/22 10:06:46 - [!] Found Binary: RecentFileCacheParser.exe
2024/08/22 10:06:46 - [!] Found Binary: SBECmd.exe
2024/08/22 10:06:46 - [!] Found Binary: SrumECmd.exe
2024/08/22 10:06:46 - [!] Found Binary: SumECmd.exe
2024/08/22 10:06:46 - [!] Found Binary: WxTCmd.exe
2024/08/22 10:06:46 - [+] Downloading EvtxECmd.exe from https://f001.backblazeb2.com/file/EricZimmermanTools/EvtxECmd.zip
2024/08/22 10:06:47 - [!] Found Binary: MFTECmd.exe
2024/08/22 10:06:47 - [!] Found Binary: SQLECmd.exe
2024/08/22 10:06:47 - [!] Found Binary: SQLECmd.exe
2024/08/22 10:06:47 - [!] Found Binary: powershell.exe
2024/08/22 10:06:47 - [!] Found Binary: ParseBrowserData.ps1
2024/08/22 10:06:47 - [!] Found Binary: System.Data.SQLite.dll

Below is the snippet of code I think is responsible for that error:
ParserCode

Also, see below my utilities dir structure which might explain the reason behind the errors:
Utilities dir

@joeavanzato
Copy link
Owner

joeavanzato commented Aug 22, 2024

It appears the stable hosting links for Zimmerman's tools have all been updated - I will update the parsing configuration to point towards the new URLs.

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

I think my issue isn't with the URLs since I already have the utilities in the utilities folder..

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

Or Why is it even trying to pull the file if it is already in the utilities folder ?

@joeavanzato
Copy link
Owner

Ah - I see the problem you are describing now. So it is attempting to download PECmd and EvtxECmd even though they already exist, correct?

I will investigate as it is not immediately obvious to me why that is occurring.

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

Yes you got it and somehow it is expecting a .ZIP file while in my directory the files are already unzipped with the binary included.
Refer to the 2 screenshots I attached above

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

FYI, what you see in my utilities directory structure is the folder and the binaries are in the folder shows in that screenshot

@snolroy
Copy link
Author

snolroy commented Aug 22, 2024

See my directory structure again below:
Utilities dir struct

@joeavanzato
Copy link
Owner

Thanks for this - I am investigating - I think this is partially due to how cmd.exe interprets certain things vs powershell.exe as I am seeing different behavior when launching the script through your method vs being directly within powershell terrminal - I will work on resolving these and appreciate the feedback.

@joeavanzato
Copy link
Owner

joeavanzato commented Aug 23, 2024

@snolroy So I'm not actually sure why your above command for ParseIR is not working - I am launching similar as below:

powershell.exe -ExecutionPolicy Bypass -File "C:\Users\Joe\Documents\GitHub\PowerPull\ParseIR.ps1" -config "C:\Users\Joe\Documents\GitHub\PowerPull\parsing_config.json" -evidence_dir "C:\Users\Joe\Documents\GitHub\PowerPull\evidence" -utilities_dir "C:\Users\Joe\Documents\GitHub\PowerPull\utilities" -ignoremissing

This is functioning as expected for me and 'finding' all of the binaries inside utilities directory, whether they are at top-level or within folders recursively.

Did you potentially modify the parsing configuration file? It is very odd that it would only cause an issue on those two files since it is identical logic.

I would recommend trying completely locally outside of your EDR to see if it works as expected first.

@snolroy
Copy link
Author

snolroy commented Aug 23, 2024

Yes I did try outside of the EDR and it worked..

@joeavanzato
Copy link
Owner

joeavanzato commented Aug 23, 2024

Then I think this is not a ParseIR/RetrievIR issue but something specific to the EDR. Which EDR?

@joeavanzato joeavanzato added the bug Something isn't working label Aug 23, 2024
@snolroy
Copy link
Author

snolroy commented Aug 23, 2024

That was Crowdstrike

@joeavanzato
Copy link
Owner

Ok - I can try to test against a CrowdStrike deployment and see if I can resolve that way.

@snolroy
Copy link
Author

snolroy commented Aug 23, 2024

Hi @joeavanzato Disregard my previous message. I ran it again a couple of times and found the culprit. I actually went through all the utility subfolders and found that only 1 of them was empty and when I ran the query again, it failed when it was trying to pull that specific utility:

[+] Downloading EvtxECmd.exe from https://f001.backblazeb2.com/file/EricZimmermanTools/EvtxECmd.zip
Invoke-WebRequest : { "code": "not_found", "message": "File with such name does not exist.", "status": 404 }At 

@joeavanzato
Copy link
Owner

joeavanzato commented Aug 23, 2024

@snolroy This is happening due to outdated URLs in parsing_config.json because Eriz Zimmerman changed the hosting location of the tools - please check the new version, this should work as expected.

https://github.com/joeavanzato/RetrievIR/blob/main/parsing_config.json

@snolroy
Copy link
Author

snolroy commented Aug 27, 2024

Hi @joeavanzato
Apologies for the late reply. I finally added all the correct folders but now Im getting another error and not sure what could really be the culprit for this one..

Last command before the error in the log line was:
2024/08/26 21:31:59 - [+] Executing: "C:\TmpRT\RetrievIR-1.1\utilities\SQLECmd.exe\SQLECmd\SQLECmd.exe" -f "C:\TmpRT\RetrievIR-1.1\evidence\XXXXXX\Browsers\ChromeUserData\manager\User Data\Default\Favicons" --csv "C:\TmpRT\RetrievIR-1.1\parsed_evidence\XXXXXX\ChromeMetadata\manager"

Error I got in the terminal was:
Unhandled exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.SQLite.SQLiteException: attempt to write a readonly databaseattempt to write a readonly database at System.Data.SQLite.SQLite3.Prepare(SQLiteConnection cnn, String strSql, SQLiteStatement previous, UInt32 timeoutMS, String& strRemain) at System.Data.SQLite.SQLiteCommand.BuildNextCommand() at System.Data.SQLite.SQLiteDataReader.NextResult() at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave) at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior) at System.Data.SQLite.SQLiteCommand.ExecuteScalar(CommandBehavior behavior) at ServiceStack.OrmLite.Dapper.SqlMapper.ExecuteScalarImpl[T](IDbConnection cnn, CommandDefinition& command) at ServiceStack.OrmLite.Dapper.SqlMapper.ExecuteScalar[T](IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Nullable1 commandTimeout, Nullable1 commandType) at SQLECmd.Program.ProcessFile(String fileName, Boolean hunt, Boolean dedupe, String csv) at SQLECmd.Program.DoWork(String f, String d, String csv, String json, Boolean dedupe, Boolean hunt, String maps, Boolean sync, Boolean debug, Boolean trace) --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at System.Delegate.DynamicInvokeImpl(Object[] args) at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.<InvokeAsync>d__11.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseParseErrorReporting>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass15_0.<<UseHelp>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass26_0.<<UseVersionOption>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass24_0.<<UseTypoCorrections>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__23_0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass21_0.<<UseParseDirective>b__0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseDebugDirective>b__8_0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__7_0>d.MoveNext()--- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseExceptionHandler>b__0>d.MoveNext()

Any idea what could be causing this now ?

@joeavanzato
Copy link
Owner

Were there previous uses of SqlECmd.exe in the logs that appear successful? Is there any amount of parsed evidence from Chrome or Edge present in the parsed_evidence directory?

I've not encountered this issue personally - I will have to investigate further.

@joeavanzato joeavanzato reopened this Aug 27, 2024
@snolroy
Copy link
Author

snolroy commented Aug 27, 2024

Yes there were successful parsing attempts:

2024/08/26 21:31:53 - [+] Executing: "C:\TmpRT\RetrievIR-1.1\utilities\SQLECmd.exe\SQLECmd\SQLECmd.exe" -f "C:\TmpRT\RetrievIR-1.1\evidence\XXXXXXX\Browsers\ChromeUserData\XXXXXXX.XXXXXXX\User Data\Snapshots\XXXXXXX\Default\Favicons" --csv "C:\TmpRT\RetrievIR-1.1\parsed_evidence\XXXXXXX\ChromeMetadata\XXXXXXX.XXXXXXX"

2024/08/26 21:31:55 - [+] Parsing: C:\TmpRT\RetrievIR-1.1\evidence\XXXXXXX\Browsers\ChromeUserData\XXXXXXX.XXXXXXX\User Data\Snapshots\XXXXXXX\Default\Favicons-journal

2024/08/26 21:31:55 - [+] Executing: "C:\TmpRT\RetrievIR-1.1\utilities\SQLECmd.exe\SQLECmd\SQLECmd.exe" -f "C:\TmpRT\RetrievIR-1.1\evidence\XXXXXXX\Browsers\ChromeUserData\XXXXXXX.XXXXXXX\User Data\Snapshots\XXXXXXX\Default\Favicons-journal" --csv "C:\TmpRT\RetrievIR-1.1\parsed_evidence\XXXXXXX\ChromeMetadata\XXXXXXX.XXXXXXX"

2024/08/26 21:31:56 - [+] Parsing: C:\TmpRT\RetrievIR-1.1\evidence\XXXXXXX\Browsers\ChromeUserData\XXXXXXX.XXXXXXX\User Data\System Profile\Favicons

Below is all the parsed evidence I got back:
Parsed_evidence

@snolroy
Copy link
Author

snolroy commented Sep 9, 2024

Greetings @joeavanzato
Update regarding this issue:
After reviewing the errors and the parser configs files, I removed the following from my config file and was able to get through the errors shown above. See what I changed below in my parser_config file:

"ChromeMetadata":{
"name": "ChromeMetadata",
"evidence_type": "ChromeMetadata",
"executable": "SQLECmd.exe",
"cmdline": "#PARSER# -f #SOURCE_FILE# --csv #DESTINATION_DIR#",
"url": "https://download.mikestammer.com/SQLECmd.zip",
"dl_type": "zip",
"if_missing": "download",
"operates_on": "file",
"file_filter": ["Bookmarks*","Cookies*","Session*","DownloadMetadata","Extension Cookies","Last Session", "Last Tabs", "Media History*", "SyncData.sqlite3"],
"type": "inline"
},
"EdgeMetadata":{
"name": "EdgeMetadata",
"evidence_type": "EdgeMetadata",
"executable": "SQLECmd.exe",
"cmdline": "#PARSER# -f #SOURCE_FILE# --csv #DESTINATION_DIR#",
"url": "https://download.mikestammer.com/SQLECmd.zip",
"dl_type": "zip",
"if_missing": "download",
"operates_on": "file",
"file_filter": ["Bookmarks*","Cookies*","Session*","DownloadMetadata","Extension Cookies","Last Session", "Last Tabs", "Media History*", "SyncData.sqlite3"],
"type": "inline"
},

However, after I got passed the error above, I am now getting another error associated with the ParseBrowserData.ps1 script...

Below are the last lines printed in the logs prior to the error being thrown:
2024/09/08 23:15:52 - [+] Executing: "C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1" -base_evidence_dir "C:\TmpRT\RetrievIR-1.1\evidence" -parsed_evidence_dir "C:\TmpRT\RetrievIR-1.1\parsed_evidence"

Below is the actual error being thrown in the terminal
New-Object : Exception calling ".ctor" with "0" argument(s): "An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)"At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:219 char:29+ ... $dbConnection = New-Object System.Data.SQLite.SQLiteConnection+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [New-Object], MethodInvocationException + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommandNotSpecified: ( :String) [], RemoteExceptionThe property 'ConnectionString' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:220 char:13+ $dbConnection.ConnectionString = $dbString+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:221 char:13+ $dbConnection.open()+ ~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:271 char:5+ $dbCmd = $dbConnection.CreateCommand()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionThe property 'Commandtext' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:272 char:5+ $dbCmd.Commandtext = "SELECT * from urls"+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionThe property 'CommandType' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:273 char:5+ $dbCmd.CommandType = [System.Data.CommandType]::Text+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:274 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:297 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:331 char:5+ $dbCmd = $dbConnection.CreateCommand()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionThe property 'Commandtext' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:332 char:5+ $dbCmd.Commandtext = "SELECT * from downloads"+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionThe property 'CommandType' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:333 char:5+ $dbCmd.CommandType = [System.Data.CommandType]::Text+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:334 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:365 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:301 char:5+ $dbCmd = $dbConnection.CreateCommand()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionThe property 'Commandtext' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:302 char:5+ $dbCmd.Commandtext = "SELECT keyword_id, url_id, term, url, title ...+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionThe property 'CommandType' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:303 char:5+ $dbCmd.CommandType = [System.Data.CommandType]::Text+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:304 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:327 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:239 char:5+ $dbCmd = $dbConnection.CreateCommand()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionThe property 'Commandtext' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:241 char:5+ $dbCmd.Commandtext = "SELECT visits.id,urls.url,title,visit_time, ...+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionThe property 'CommandType' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:242 char:5+ $dbCmd.CommandType = [System.Data.CommandType]::Text+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:243 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:267 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:228 char:13+ $dbConnection.Close()+ ~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionException calling "Open" with "0" argument(s): "An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)"At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:221 char:13+ $dbConnection.open()+ ~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : BadImageFormatExceptionNotSpecified: ( :String) [], RemoteExceptionException calling "ExecuteReader" with "0" argument(s): "Database is not open"At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:274 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : InvalidOperationExceptionNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:297 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNullNotSpecified: ( :String) [], RemoteExceptionException calling "ExecuteReader" with "0" argument(s): "Database is not open"At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:334 char:5+ $dbReader = $dbCmd.ExecuteReader()+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : InvalidOperationExceptionNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:365 char:5+ $dbReader.Close()+ ~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException

Any idea why the above is being thrown now? I tried troubleshooting and attempted modifying the ParseBrowserData.ps 1 script to see if it'll work but that change ended up generating another error so I gave up and decided to post the original error here to see if you can help troubleshoot/find out why I am getting this.

@joeavanzato
Copy link
Owner

joeavanzato commented Sep 9, 2024

I have a feeling this may be caused by an issue loading the relevant Sqlite DLL on line 196 - please edit the ParseBrowserData.ps1 file line 196 to remove the "| Out-Null" at the end of the line as below:

Original:

[Reflection.Assembly]::LoadFile($sqlite_dll_location) | Out-Null

New:

Write-Host $sqlite_dll_location
[Reflection.Assembly]::LoadFile($sqlite_dll_location)

Then try re-running and see if there are any other errors that appear. Make sure that there is an actual DLL location printed out as well.

@joeavanzato
Copy link
Owner

joeavanzato commented Sep 9, 2024

@snolroy On further thought, I'm wondering if this is an issue with the type of SQLite DLL being downloaded - please review the file downloaded at

"url": "https://system.data.sqlite.org/blobs/1.0.118.0/sqlite-netFx45-binary-x64-2012-1.0.118.0.zip",
to determine if this is appropriate for your target system and adjust as needed since it's possible this is not the correct DLL for your processing system.

This DLL does rely on both system version (32 vs 64) as well as the appropriate .NET installation being downloaded.

@snolroy
Copy link
Author

snolroy commented Sep 10, 2024

@joeavanzato When I look at the file, how do I determine if it's 32 vs 64 bit ?

@joeavanzato
Copy link
Owner

The default file in the configuration is designed to run on:

  • x64 CPU (64 bit)
  • .NET Framework 4.5 installed

If your processing computer does not meet those requirements, you may need to replace that with another one available at https://system.data.sqlite.org/index.html/doc/trunk/www/downloads.wiki

@snolroy
Copy link
Author

snolroy commented Sep 10, 2024

@joeavanzato I made adjustments to the ParseBrowserData.ps1 script on line 196 you provided and ran it again and it now printed the location of the DLL before throwing the SAME error as above...

Last log line before the error was:
2024/09/10 12:32:30 - [+] Executing: "C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1" -base_evidence_dir "C:\TmpRT\RetrievIR-1.1\evidence" -parsed_evidence_dir "C:\TmpRT\RetrievIR-1.1\parsed_evidence"

Last terminal output before the error was the following:

[+] Found 1 Targets...
[+] Processing: Hostname
C:\TmpRT\RetrievIR-1.1\utilities\standalone\System.Data.SQLite.dll\System.Data.SQLite.dll

GAC    Version        Location                                                                                         
---    -------        --------                                                                                         
False  v4.0.30319     C:\TmpRT\RetrievIR-1.1\utilities\standalone\System.Data.SQLite.dll\System.Data.SQLite.dll

Error Thrown: New-Object : Exception calling ".ctor" with "0" argument(s): "An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)"At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:220 char:29+ ...        $dbConnection = New-Object System.Data.SQLite.SQLiteConnection+                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommandNotSpecified: ( :String) [], RemoteExceptionThe property 'ConnectionString' cannot be found on this object. Verify that the property exists and can be set.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:221 char:13+             $dbConnection.ConnectionString = $dbString+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException    + FullyQualifiedErrorId : PropertyNotFoundNotSpecified: ( :String) [], RemoteExceptionYou cannot call a method on a null-valued expression.At C:\TmpRT\RetrievIR-1.1\utilities\standalone\ParseBrowserData.ps1:222 char:13+             $dbConnection.open()+         ............................................................

@snolroy
Copy link
Author

snolroy commented Sep 10, 2024

I also checked if the target system was 64 bit and confirmed that it is:
64bit check

@joeavanzato
Copy link
Owner

Just to clarify, the system running ParseIR is the one that should be 64 bit with .NET 4.5 installed if using the default SQLite DLL embedded in the parsing_config.json. Can you verify .NET 4.5 is installed?

@snolroy
Copy link
Author

snolroy commented Sep 10, 2024

Yes---The command above was ran on the target system showing its a 64 bit system. Also it is the same system that has the SQLite DLL installed as shown in the above terminal output...

is .NET 4.5 installed while installing SQLite or is it already installed on the system. If the latter is there an easy way to verify that ?

@snolroy
Copy link
Author

snolroy commented Sep 10, 2024

After checking the .NET release version using((Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full').Release) it was 528372 which corresponds to .NET Framework 4.8.

@joeavanzato
Copy link
Owner

@snolroy - this may be caused by the relevant "System.Data.SQLite.dll.config" file missing from the directory - can you please list the contents of C:\TmpRT\RetrievIR-1.1\utilities\standalone\System.Data.SQLite.dll?

@snolroy
Copy link
Author

snolroy commented Sep 11, 2024

@joeavanzato See below:
Systemdta directory

@joeavanzato
Copy link
Owner

I would try running this locally - use RetrievIR to gather the evidence from the target and bring it back to another host then parse it locally using ParseIR and see if that makes as difference in operations.

@snolroy
Copy link
Author

snolroy commented Sep 16, 2024

Hi @joeavanzato...I ran it previously locally and it worked.. But then I wanted to try it via EDR as well. Does that mean every time I'd have to just pull the evidence folder from the remote system and then run the parser locally with the evidence folder downloaded ?

@joeavanzato
Copy link
Owner

joeavanzato commented Sep 17, 2024

@snolroy I think I would recommend doing this in either one of two ways.

  1. Running RetrievIR through EDR, collecting evidence back to EDR cloud, downloading to analysis machine and using ParseIR against the data on your analysis machine.
  2. Running RetrievIR on Host A and targeting Host B through command-line properties (.\RetrievIR.ps1 -targets HOSTB), enabling remote evidence collection - then you can immediately pivot to parsing without involving the EDR directly on HOSTA.

I think I will add an option to ZIP the results as well to make this easier.

@snolroy
Copy link
Author

snolroy commented Sep 18, 2024

Hi @joeavanzato Thanks for your feedback...
I think the only option possible for me would be option 1...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants