Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance PM2 Process Management with Error Handling and Graceful Shutdown #30

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

Enhance PM2 Process Management with Error Handling and Graceful Shutdown #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
71a8f11
Update pm2.js
rickybharti Jun 14, 2024
614270d
Update README.md
rickybharti Jun 14, 2024
dad94aa
Update README.md
rickybharti Jun 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 169 additions & 65 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,132 +1,236 @@
# Overview
## Overview

The Shardeum JSON-RPC Server enables developers to interact with the Shardeum blockchain network. It allows dapps to post requests, retrieve information, and perform other related operations using JSON-RPC over HTTP. Additionally, the Shardeum JSON-RPC Server comes with an added REST API for debugging and monitoring purposes.

For running existing dapps on Shardeum, refer to the [EVM JSON-RPC API](https://ethereum.org/en/developers/docs/apis/json-rpc/).

## Table of Contents

- [Project Structure](#project-structure)
- [Docker Setup](#docker-setup)
- [Start JSON-RPC Server](#start-json-rpc-server)
- [Check the Logs](#check-the-logs)
- [Clean the Setup](#clean-the-setup)
- [Developer Environment Setup](#developer-environment-setup)
- [Requirements](#requirements)
- [Installing Project Source Code](#installing-project-source-code)
- [Starting JSON-RPC Server](#starting-json-rpc-server)
- [Cleanup](#cleanup)
- [Usage](#usage)
- [REST API Endpoints](#rest-api-endpoints)
- [Debug Endpoints](#debug-endpoints)
- [Configuration](#configuration)
- [Notable Modules and Functions](#notable-modules-and-functions)
- [api.ts](#apits)
- [clients.ts](#clientsts)
- [index.ts](#indexts)
- [log_server.ts](#log_serverts)
- [Contributing](#contributing)
- [License](#license)
- [Contact](#contact)

## Project Structure

The project is organized as follows:

The Shardeum JSON-RPC Server enables developers to interact with the Shardeum blockchain network. It allows dapps to post request, retrieve information, and other related operations, using JSON-RPC over HTTP. Additionally, the Shardeum JSON-RPC Server comes with an added REST API for debugging and monitoring purposes.

For running existing dapps on Shardeum, refer to EVM [JSON-RPC API](https://ethereum.org/en/developers/docs/apis/json-rpc/)

## Docker setup
```
json-rpc-server-dev/
├── .appsec/ # Security baseline configurations
├── src/ # Source code
│ ├── api.ts # Main API implementation
│ ├── config.ts # Configuration settings
│ ├── logger.ts # Logging utility
│ ├── server.ts # Server setup and initialization
│ ├── types.ts # Type definitions
│ ├── utils/ # Utility functions
│ ├── cache/ # Cache management
│ ├── external/ # External service integrations
│ ├── middlewares/ # Middleware functions
│ ├── routes/ # API routes
│ ├── service/ # Core service implementations
│ ├── storage/ # Storage solutions
│ ├── websocket/ # WebSocket implementations
├── Dockerfile # Docker configuration
├── docker-compose.yml # Docker Compose configuration
├── README.md # Project documentation
├── package.json # Project metadata and dependencies
└── tsconfig.json # TypeScript configuration
```

For developers deploying to production environments, simply use `docker compose` command.
## Docker Setup

env `NO_OF_RPC_SERVERS` creates replicas of rpc servers using pm2. default is 1. Default port is 8080, port for each replicas will increment by 1 on default port. i.e 8081, 8082, 8083, etc.
For developers deploying to production environments, use the `docker compose` command. The environment variable `NO_OF_RPC_SERVERS` creates replicas of RPC servers using PM2. The default is 1. The default port is 8080, and ports for each replica will increment by 1 (e.g., 8081, 8082, 8083).

### Start json-rpc-server
### Start JSON-RPC Server

```shell
# Run services in detach mode
docker compose up -d
```

### Check the logs
### Check the Logs

```shell
docker compose logs -f
```

### Clean the setup
### Clean the Setup

```shell
docker compose down
```

# Developer Environment Setup
## Developer Environment Setup

For end users, such as exchanges and large decentralized applications (dApps), seeking to deploy their own RPC server, it is recommended to run the Shardeum JSON-RPC server using Docker. It ensures all dependencies are installed and the server is running in a consistent environment. For developers who want to contribute to this project, running the server from source is recommended. You can use `npm` for installing the server locally.

## Requirements
### Requirements

If you are using `Docker`, in order to run the Shardeum JSON-RPC server, you must have the [Docker](https://docs.docker.com/get-docker/) daemon installed.
To run the Shardeum JSON-RPC server, you must have the [Docker](https://docs.docker.com/get-docker/) daemon installed.

## Installing project source code
### Installing Project Source Code

Let’s install the project source code, switch to `dev` branch and follow the below instructions:
Follow these steps to install the project source code and switch to the `dev` branch:

```bash
git clone https://github.com/shardeum/json-rpc-server.git
cd shardeum-json-rpc
git switch dev
```

If you are building using `Docker`, run the below command:
### Building the Project

```bash
make build
```
- **Using Docker**:

This command will build Docker images for the service.
```bash
make build
```

If you are building using `NPM`, run the below command, it will install all the required dependencies
This command will build Docker images for the service.

```bash
npm install
```
- **Using NPM**:

## Starting JSON-RPC Server
```bash
npm install
```

If you are building using `Docker`, you can start the JSON-RPC server by running the following command:
This command will install all the required dependencies.

```bash
make run
```
### Starting JSON-RPC Server

This will start a container running the `shardeum-json-rpc` server image, available on port `8080`. The servers configuration fields can be viewed and edited in the `src/config.ts` file. Additionally, you can manage the server's access control lists by editing the `whitelist.json`, `blacklist.json` and `spammerlist.json`.
- **Using Docker**:

But if you are using NPM, use the below command to run the server:
```bash
make run
```

```bash
npm run start
```
This will start a container running the `shardeum-json-rpc` server image, available on port `8080`. Configuration fields can be viewed and edited in the `src/config.ts` file. Manage the server's access control lists by editing the `whitelist.json`, `blacklist.json`, and `spammerlist.json`.

If you want to modify the chainId or the port number, go to `src/config.ts` file:
- **Using NPM**:

```bash
chainId: 8082
port: 8080
```
```bash
npm run start
```

The RPC URL for using Metamask with Remix IDE and for running scripts is <http://localhost:port> (default: <http://localhost:8080>)
Modify the `chainId` or the `port` number in the `src/config.ts` file:

If you are contributing to this project, use Shardeum server to create the network from within the [validator repo](https://gitlab.com/shardus/archive/archive-server). You can find more details [here](https://github.com/shardeum/shardeum)
```typescript
chainId: 8082
port: 8080
```

## Cleanup
The RPC URL for using Metamask with Remix IDE and for running scripts is <http://localhost:port> (default: <http://localhost:8080>).

If you are using `Docker`, you can stop the server by running:
For contributing to this project, use the Shardeum server to create the network from within the [validator repo](https://gitlab.com/shardus/archive/archive-server). More details can be found [here](https://github.com/shardeum/shardeum).

```bash
make stop
```
### Cleanup

Additionally, you can remove the docker images by running:
- **Using Docker**:

```bash
make clean
```
```bash
make stop
```

Remove the Docker images by running:

```bash
make clean
```

This will remove all Docker images created by the server during the build process.

## Usage

### REST API Endpoints

- **GET `/authenticate/:passphrase`**: Authenticate using the `passphrase` set in `config.ts` or system environment variable.
- **GET `/log/api-stats`**: Emit RPC interface call counts, average TPS, and other information. Supports querying by time range (`/log/api-stats?start={x}&end={x}`).
- **GET `/log/txs`**: Return the transactions made through the RPC server. Supports dynamic pagination (`/log/txs?max=30&page=9`).
- **GET `/log/status`**: Return the status of logging.
- **GET `/log/startTxCapture`**: Enable capturing of incoming transactions.
- **GET `/log/stopTxCapture`**: Disable capturing of incoming transactions.
- **GET `/log/startRPCCapture`**: Enable capturing of RPC interface call stats.
- **GET `/log/stopRPCCapture`**: Disable capturing of RPC interface call stats.
- **GET `/cleanStatTable`**: Trigger purging of the stats table.
- **GET `/cleanTxTable`**: Trigger purging of the transactions table.

This will remove all docker images created by the server during the build process.
### Debug Endpoints

# DEBUG Endpoints
These APIs are protected, preventing the general public from wiping out debug data. To authenticate, use `/authenticate/:passphrase`. The `passphrase` is set in `config.ts` or within the system environment variable.

These api are protected preventing general public to wiping out debug data to authenticate use `/authenticate/:passphrase`. `passphrase` is set in `config.ts` config file or within the system env variable.
- **GET `/log/api-stats`**: Emits RPC interface call counts and average TPS along with other information. Supports query by time range (e.g., `/log/api-stats?start={x}&end={x}`). The parameter value can be either `yyyy-mm-dd` or Unix epoch in milliseconds.
- **GET `/log/txs`**: Returns the transactions made through the RPC server. Supports dynamic pagination (e.g., `/log/txs?max=30&page=9`). Default values are `1000` for `max` and `0` for `page`.
- **GET `/log/status`**: Returns the status of logging, such as the date of recording start and whether or not recording is enabled.
- **GET `/log/startTxCapture`**: Sets the config value to true, enabling the capture of incoming transactions.
- **GET `/log/stopTxCapture`**: Sets the config value to false, disabling the capture of incoming transactions.
- **GET `/log/startRPCCapture`**: Sets the config value to true, enabling the capture of RPC interface call stats.
- **GET `/log/stopRPCCapture`**: Sets the config value to false, disabling the capture of RPC interface call stats.
- **GET `/cleanStatTable`**: Triggers purging of the table that stores interface stats.
- **GET `/cleanTxTable`**: Triggers purging of the table that stores transaction logging.

GET `/log/api-stats` this endpint emit the rpc interface call counts and avg tps along with a few a other information. This endpoint support query by time range. i.e `/log/api-stats?start={x}&end={x}`. The parameter value can be either `yyyy-mm-dd` or unix epoch in millisecond. (NOTE standard unix epoch is in seconds which does not work, it has to be in millisecond accuracy). Not setting any timestamp parameter will returns paginated json of all the entry in db.
## Configuration

GET `/log/txs` this endpoint return the txs it has been made through rpc server. This endpoint support dynmaic pagination. i.e `/log/txs?max=30&page=9`.
Default values are `1000` for `max` and `0` for page.
Important configuration files and parameters:

GET `/log/status` this endpint return status of logging such as date of recording start and whether or not recording is enabled.
- **`config.ts`**: Contains various configuration settings for the server.
- **Environment Variables**:
- `NO_OF_RPC_SERVERS`: Number of RPC server replicas (default is 1).
- `PORT`: Port number for the server (default is 8080).

GET `/log/startTxCapture` this endpoint set the config value to true which control whether to capture incoming txs and store in database.
## Notable Modules and Functions

GET `/log/stopRPCCapture` this endpoint set the config value to false which control whether to capture incoming rpc interface call stat and store in database
## `api.ts`

GET `/log/startRPCCapture` this endpoint set the config value to true which control whether to capture rpc interface call stat and store in database.
- **Core RPC Methods**: Handles various JSON-RPC methods such as `eth_call`, `eth_sendTransaction`, and more.
- **Utilities**: Includes utility functions for validating addresses, handling transactions, and interacting with the Ethereum ecosystem.
- **Error Handling**: Uses `serializeError` to format and return errors in a standardized way.

GET `/log/stopTxCapture` this endpoint set the config value to false which control whether to capture incoming txs and store in database
## `clients.ts`

GET `/cleanStatTable` this endpoint trigger purging of table that store interface stats
- **ClientList Class**: Manages active WebSocket clients and their subscription details.
- **Subscription Management**: Provides methods to add, remove, and manage client subscriptions.
- **Mapping**: Maps subscription IDs to request IDs and vice versa for efficient lookup and management.

GET `/cleanTxTable` this endpoint trigger purging of table that store transaction logging
## `index.ts`

# Contributing
- **WebSocket Server Setup**: Initializes and manages WebSocket server connections.
- **Event Handling**: Manages events for incoming connections, message handling, and disconnections.
- **Subscription Integration**: Uses `ClientList` to manage subscriptions and emit data to clients.

## `log_server.ts`

- **Log Server Connection**: Sets up and maintains WebSocket connection to the log server.
- **Event Handling**: Manages events for log-related messages.
- **Configuration Integration**: Uses configuration settings to manage log stream setup and maintenance.

## Contributing

Contributions are very welcome! Everyone interacting in our codebases, issue trackers, and any other form of communication, including chat rooms and mailing lists, is expected to follow our [code of conduct](CODE_OF_CONDUCT.md) so we can all enjoy the effort we put into this project.

## License

This project is licensed under the terms of the [MIT license](LICENSE).

## Contact

For any questions or suggestions, feel free to reach out to us at [[email protected]](mailto:[email protected]).
Loading