Skip to content

Commit

Permalink
Fix cluster selection (#182)
Browse files Browse the repository at this point in the history
  • Loading branch information
MaartenGr authored Oct 5, 2023
1 parent e02f83b commit f21062a
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 3 deletions.
15 changes: 15 additions & 0 deletions docs/guides/keyllm.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ This data was chosen to show the different use cases and techniques. As you migh

Let's start with `KeyLLM` only.


# Use Cases

If you want the full performance and easiest method, you can skip the use cases below and go straight to number 5 where you will combine `KeyBERT` with `KeyLLM`.
Expand Down Expand Up @@ -180,6 +181,13 @@ If you have embeddings of your documents, you could use those to find documents
--8<-- "docs/images/efficient.svg"
</div>

!!! Tip
Before you get started, it might be worthwhile to uninstall sentence-transformers and re-install it from the main branch.
There is an issue with community detection (cluster) that might make the model run without finishing. It is as straightforward as:
`pip uninstall sentence-transformers`
`pip install --upgrade git+https://github.com/UKPLab/sentence-transformers`


```python
import openai
from keybert.llm import OpenAI
Expand Down Expand Up @@ -224,6 +232,13 @@ This is the best of both worlds. We use `KeyBERT` to generate a first pass of ke
--8<-- "docs/images/keybert_keyllm.svg"
</div>

!!! Tip
Before you get started, it might be worthwhile to uninstall sentence-transformers and re-install it from the main branch.
There is an issue with community detection (cluster) that might make the model run without finishing. It is as straightforward as:
`pip uninstall sentence-transformers`
`pip install --upgrade git+https://github.com/UKPLab/sentence-transformers`


```python
import openai
from keybert.llm import OpenAI
Expand Down
2 changes: 1 addition & 1 deletion keybert/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from keybert._llm import KeyLLM
from keybert._model import KeyBERT

__version__ = "0.8.2"
__version__ = "0.8.3"
2 changes: 1 addition & 1 deletion keybert/_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def extract_keywords(
if in_cluster:
selected_docs = [docs[cluster[0]] for cluster in clusters]
if candidate_keywords is not None:
selected_keywords = [candidate_keywords[cluster[0]] for cluster in in_cluster]
selected_keywords = [candidate_keywords[cluster[0]] for cluster in clusters]
else:
selected_keywords = None
in_cluster_keywords = self.llm.extract_keywords(
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
setup(
name="keybert",
packages=find_packages(exclude=["notebooks", "docs"]),
version="0.8.2",
version="0.8.3",
author="Maarten Grootendorst",
author_email="[email protected]",
description="KeyBERT performs keyword extraction with state-of-the-art transformer models.",
Expand Down

0 comments on commit f21062a

Please sign in to comment.