are you think how a web search engine such as google work? it seem very complex work? dont worry we are create piston engine It behaves like it

Piston Engine Demo

Piston.Engine.mp4

🤩🤩🤩 .Wow we are implement lightweight web search engine same as Google

---

general architecture and overview of piston engine at a glance

we implemented web search engine with elastic . 😎😎 in next version we want to implement key functionality of elastic instead of using it.

---

.

get started with project

it's very easy to run that

1. install Docker from official page

2. use following command for start in development mode . open powershell or git-bash

   chmod +x start-development-services.sh
   bash start-development-services.sh
   

3. its amazing all services and dependency installed and configured properly

---

after run all service & dependency for project to work correctly we need two thing

1. add domain with crawler-service to crawler that domain and related page .and automatically index that in search-service from crawled webpage

we do it manually from swagger . but we can generate ui for that and only admin user can do that
it accessible at http://localhost:8080/swagger-ui/index.html

2. you access to all document generated from start point of crawler-service in kibana dashboard at address(http://localhost:5601) by username -> elastic & password -> 123456 .
   

it has amazing visualization dashboard like bellow

3. also using search-service endpoint at (http://localhost:8083/swagger-ui/index.html) to retrieve clear result for suggestion & search that read from elastic index
   

we fortunately create ui for show suggestion and search from search-service same as Google .
it accessible at http://localhost:3000

i generate ui for autocomplete in search-box and select that. like picture bellow

---

and after select input we search base on query to find most related document and rank them. like picture bellow

over time our dataset is larger and our suggestion it can be more precise and search result is more