HBIT Project

HBIT is a project aimed at analyzing and aggregating security information about mobile devices. It consists of several components including scrapers, a web API, and a CLI application. Below is an overview of the project's structure and functionality.

📁 Project Structure

🕵️‍♂️ Scrapers

Tip

More on scrapers in hbit_data/README.md file!

The scrapers gather data from various sources and store it in a structured format. The main scrapers include:

• Security Updates Scraper: Collects security updates.
• iPhones Scraper: Gathers information about iPhone models.
• CWEs Scraper: Collects Common Weakness Enumerations.
• CAPECs Scraper: Collects Common Attack Pattern Enumeration and Classification.
• Patches Scraper: Collects patch information.

🌐 REST API

Tip

More on REST API in hbit_api/README.md file!

The REST API provides access to the collected data. It is built using FastAPI and includes endpoints for:

• Login: User authentication.
• Users: User management.
• Devices: Device information.
• Vulnerabilities: Vulnerability information.
• Evaluations: Security evaluations of devices and patches.

Tip

More on REST API in hbit_api/README.md file!

The REST API provides access to the collected data. It is built using FastAPI and includes endpoints for:

• Login: User authentication.
• Users: User management.
• Devices: Device information.
• Vulnerabilities: Vulnerability information.
• Evaluations: Security evaluations of devices and patches.

🤖 AI Streamlit UI

Warning

Running Streamlit UI requires having HBIT API running.

To use the CLI application, run:

PYTHONPATH=. uv run streamlit run hbit/ui/security_agent.py

Tip

More on AI, agents and their UI in hbit/README.md file!

TBD

🚀 Next Steps

• Add More Scrapers: Include additional scrapers to gather more data.
• Add Email Security Information: Maybe for this case I can use some retriever from HIBP and save data using scrapers.

🚀 Installation

To install the project, follow these steps:

1. Clone the repository:

   git clone https://github.com/TomasJani/hbit.git
   cd hbit

2. Install the dependencies:

   uv sync

📖 Usage

📄 Creating .env file

The project uses environment variables for configuration. Create a .env file in the root directory. You can use .env.example as an example and fill every change-this.

Some env variables are set to change-one-of. You will be required to fill at least one env variable in a group of these variables.

🚀 Starting the REST API

To start the REST API, run:

uvicorn hbit_api.api.main:api_router --reload

🛠️ Running the Scrapers

Warning

Running scrapers requires having HBIT API running.

You can run all scrapers in the correct order by calling the bash script:

bash hbit_data/scripts/run_scrapers.sh

To run individual scrapers, use the CLI commands provided in hbit_data/cli.py. For example, to scrape security updates:

PYTHONPATH=. uv run hbit_data/cli.py scrape_security_updates

💻 AI Streamlit UI

Warning

Running Streamlit UI requires having HBIT API running.

To use the CLI application, run:

PYTHONPATH=. uv run streamlit run hbit/ui/security_agent.py