Update: Metadata is included by default, Add: exclude_data arg to exc…

…lude meta_data (#10) * update: made metadata to be always included by default * Update README.md Signed-off-by: pk-zipstack <[email protected]> --------- Signed-off-by: pk-zipstack <[email protected]>
Zipstack · Nov 19, 2024 · 582b0d1 · 582b0d1
1 parent 11c46b5
commit 582b0d1
Show file tree

Hide file tree

Showing 2 changed files with 10 additions and 6 deletions.
diff --git a/README.md b/README.md
@@ -30,6 +30,10 @@ The script uses a local SQLite database (`file_processing.db`) with the followin
   - `time_taken` (REAL): Time taken to process the file
   - `status_code` (INTEGER): API status code
   - `status_api_endpoint` (TEXT): API endpoint for checking status
+  - `total_embedding_cost` (REAL): Total cost incurred for embeddings.
+  - `total_embedding_tokens` (INTEGER): Total tokens used for embeddings.
+  - `total_llm_cost` (REAL): Total cost incurred for LLM operations.
+  - `total_llm_tokens` (INTEGER): Total tokens used for LLM operations.
   - `updated_at` (TEXT): Last updated timestamp
   - `created_at` (TEXT): Creation timestamp
 
@@ -60,7 +64,7 @@ This will display detailed usage information.
 - `--skip_unprocessed`: Skip unprocessed files when retrying failed files.
 - `--log_level`: Log level (default: `INFO`).
 - `--print_report`: Print a detailed report of all processed files at the end.
-- `--include_metadata`: Include metadata on tokens consumed and the context passed to LLMs for prompt studio exported tools in the result for each file.
+- `--exclude_metadata`: Exclude metadata on tokens consumed and the context passed to LLMs for prompt studio exported tools in the result for each file.
 - `--no_verify`: Disable SSL certificate verification. (By default, SSL verification is enabled.)
 
 ## Usage Examples

diff --git a/main.py b/main.py
@@ -35,7 +35,7 @@ class Arguments:
     skip_unprocessed: bool = False
     log_level: str = "INFO"
     print_report: bool = False
-    include_metadata: bool = False
+    include_metadata: bool = True
     verify: bool = True
 
 
@@ -510,10 +510,10 @@ def main():
     )
 
     parser.add_argument(
-        "--include_metadata",
+        "--exclude_metadata",
         dest="include_metadata",
-        action="store_true",
-        help="Include metadata on tokens consumed and the context passed to LLMs for prompt studio exported tools in the result for each file.",
+        action="store_false",
+        help="Exclude metadata on tokens consumed and the context passed to LLMs for prompt studio exported tools in the result for each file.",
     )
 
     parser.add_argument(
@@ -546,4 +546,4 @@ def main():
 
 
 if __name__ == "__main__":
-    main()
+    main()