Update README and add evals

Author: salzubi401

README.md

@@ -1,52 +1,120 @@
-# OpenDeepSearch
+# OpenDeepSearch 🚀🔍
 
-## Core Components
+OpenDeepSearch is a lightweight yet powerful search tool designed for seamless integration with AI agents. It enables deep web search and retrieval, optimized for use with Hugging Face's **[SmolAgents](https://github.com/huggingface/smolagents)** ecosystem.
 
-### 1. Web Scraping
-The WebScraper class supports multiple extraction strategies:
-- LLM-based extraction
-- CSS selectors
-- XPath queries
-- No-extraction (raw content)
-- Cosine similarity-based extraction
+## Features ✨
 
-Reference: 
+- **Semantic Search** 🧠: Leverages Infinity Embedding API for high-quality search results.
+- **Two Modes of Operation** ⚡:
+  - **Default Mode**: Quick and efficient search with minimal latency.
+  - **Pro Mode (Deep Search)**: More in-depth and accurate results at the cost of additional processing time.
+- **Optimized for AI Agents** 🤖: Works seamlessly with **SmolAgents** like `CodeAgent`.
+- **Fast and Lightweight** ⚡: Designed for speed and efficiency with minimal setup.
+- **Extensible** 🔌: Easily configurable to work with different models and APIs.
 
-### 2. Content Processing
-Includes advanced content processing features:
-- Chunk-based text splitting
-- Quality content filtering
-- Educational value assessment
-- Smart paragraph processing
+## Installation 📦
 
-Reference:
+To install OpenDeepSearch, run:
 
-### 3. Semantic Search
-Implements semantic search capabilities:
-- Document reranking
-- Query-document similarity scoring
-- Customizable normalization methods
+```bash
+pip install opendeepsearch
+```
 
-Reference:
+## Usage 🏗️
 
-## Environment Variables
+You can use OpenDeepSearch independently or integrate it with **SmolAgents** for enhanced reasoning and code generation capabilities.
 
-Required environment variables:
-- `SERPER_API_KEY`: For web search functionality
-- `OPENROUTER_API_KEY`: For LLM-based extraction
+### Using OpenDeepSearch Standalone 🔍
 
-## Dependencies
+```python
+from opendeepsearch import OpenDeepSearchTool
+import os
 
-Key dependencies include:
+search_agent = OpenDeepSearchTool(model_name="openrouter/google/gemini-2.0-flash-001", pro_mode=True)  # Set pro_mode for deep search
+query = "Fastest land animal?"
+result = search_agent.search(query)
+print(result)
+```
 
-Full dependencies list can be found in the pyproject.toml file.
+### Integrating with SmolAgents & LiteLLM 🤖⚙️
 
-## Contributing
+```python
+from opendeepsearch import OpenDeepSearchTool
+from smolagents import CodeAgent, LiteLLMModel
+import os
 
-Contributions are welcome! Please feel free to submit a Pull Request.
+search_agent = OpenDeepSearchTool(model_name="openrouter/google/gemini-2.0-flash-001", pro_mode=True)
+model = LiteLLMModel(
+    "openrouter/google/gemini-2.0-flash-001",
+    temperature=0.2,
+    api_key=os.environ["OPENROUTER_API_KEY"]
+)
 
-## Acknowledgments
+code_agent = CodeAgent(tools=[search_agent], model=model)
+query = "How long would a cheetah at full speed take to run the length of Pont Alexandre III?"
+result = code_agent.run(query)
+
+print(result)
+```
+
+## LiteLLM Setup & Usage 🔥
+
+[LiteLLM](https://github.com/BerriAI/litellm) is a lightweight and efficient wrapper that enables seamless integration with multiple LLM APIs. OpenDeepSearch leverages LiteLLM, meaning you can use **any LLM from any provider** that LiteLLM supports. This includes OpenAI, Anthropic, Cohere, and others. **OpenRouter** is a great example of a provider that gives access to multiple models through a single API.
+
+### Installing LiteLLM
+
+[LiteLLM](https://github.com/BerriAI/litellm) is a lightweight and efficient wrapper that enables seamless integration with multiple LLM APIs. OpenDeepSearch leverages LiteLLM for model inference.
+
+### Installing LiteLLM
+
+To install LiteLLM, run:
+
+```bash
+pip install litellm
+```
+
+### Using LiteLLM with OpenDeepSearch
+
+You need to set up your API key in your environment variables before using LiteLLM:
+
+```bash
+export OPENROUTER_API_KEY='your-api-key-here'
+```
+
+Then, you can use it as shown in the SmolAgents integration example above.
+
+## Configuration ⚙️
+
+You can configure OpenDeepSearch with environment variables or parameters:
+
+- `OPENROUTER_API_KEY`: API key for accessing OpenRouter models.
+- `MODEL_NAME`: Model used for search (default: `openrouter/google/gemini-2.0-flash-001`).
+- `PRO_MODE`: Set to `True` to enable deep search for more accurate results.
+
+## Acknowledgments 💡
+
+OpenDeepSearch is built on the shoulders of great open-source projects:
+
+- **[Crawl4AI](https://github.com/crawl4ai)** 🕷️ – Provides data crawling support.
+- **[Infinity Embedding API](https://infinity.ai)** 🌍 – Powers semantic search capabilities.
+- **LiteLLM** 🔥 – Used for efficient AI model integration.
+- **Various Open-Source Libraries** 📚 – Enhancing search and retrieval functionalities.
+
+## License 📜
+
+This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
+
+## Contributing 🤝
+
+We welcome contributions! If you'd like to improve OpenDeepSearch, please:
+
+1. Fork the repository.
+2. Create a new branch (`feature-xyz`).
+3. Submit a pull request.
+
+For major changes, open an issue to discuss your ideas first.
+
+## Contact 📬
+
+For questions or collaborations, open an issue or reach out to the maintainers.
 
-- Built with [Crawl4AI](https://github.com/crawl4ai)
-- Uses [Infinity Embedding API](https://infinity.ai) for semantic search
-- Powered by various open-source libraries and tools