Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lxdex #500

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Lxdex #500

wants to merge 4 commits into from

Conversation

turtle0x1
Copy link
Owner

@turtle0x1 turtle0x1 commented May 15, 2022

The first public LXD indexer? Maybe if we ignore general purpose JSON indexers 🥳

When you have have more than 2 or 3 LXD hosts searching becomes slow because to search "fully" you have to hit multiple endpoints. This branch add supports for a multistage process to create a searchable index that should be snappy to search.

There is still quite a bit todo;

  • Filter search results based on user access
  • Scale test (beyond copying the array)
  • Grab more paramaters
  • Stop assuming user.user-data is YAML
  • Concurrency on cache builder
  • Document how it works
  • Get scripts into cron jobs
  • Adding settings to disable / enable / configure search
    • Scanning 30+ hosts to recursion level 2 every 5 mins probably isn't a good idea

Example search for "ubuntu run" (as in to mean, showing instances in Running state with a ubuntu image)
lxdex

We do this because we lots of hosts it could be very slow to gather
all the data.

Areas for improvement

  - Concurrency
    - `go doSomething()` has spoiled me, i miss go :(
The index is a simple tree that looks something like this

[
   "any_string_found_in_a_property"=>[
      // A collection of entities matching
   ]

   // I.E Running instances
   "Running"=> [0, 1, 2]
]

This allows for non-recursive searches later

Currently we dont support indexing an entire array, you have to specify
key/s within the array using regex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant