How do I use the summary module?

[cvieiraneto]

I'm trying to use the summary pipe, but I haven't found any references for how to use it. Does anyone know?

nlp = spacy.blank("pt")
llm = nlp.add_pipe("llm_summarization")
nlp("""The United Nations, referred to informally as the UN, is an
    intergovernmental organization whose stated purposes are  to maintain
    international peace and security, develop friendly relations among nations,
    achieve international cooperation, and serve as a centre for harmonizing the
    actions of nations. It is the world's largest international organization.
    The UN is headquartered on international territory in New York City, and the
    organization has other offices in Geneva, Nairobi, Vienna, and The Hague,
    where the International Court of Justice is headquartered.\n\n The UN was
    established after World War II with the aim of preventing future world wars,
    and succeeded the League of  Nations, which was characterized as
    ineffective.""")

output: same text (without resume)

[tleers]

I ran into the same issue. For whatever reason, the output of the summariser is nested somewhere in a dictionary under the attribute "user_data".

E.g.:

doc = nlp("text to summarize")
doc.user_data

[rmitsch]

To expand on this: spaCy pipeline components don't modify the text in place. Doing that for this particular spacy-llm task would have run counter to this convention.

By default the summary is stored in doc.user_data["summary"]. It can be stored in an arbitrary field in user_data by setting the field argument in the config.

The documentation mentions this, but perhaps we should highlight this more/add some explanation as to why the Doc instance's text isn't modified directly.

[tleers]

In the documentation you linked, I see no mention of this. On top of that, when I use the following code:

doc = nlp("Summarize this text please.")
doc.user_data["summary"]

just results inKeyError: 'summary'.

The user_data object looks like this:
{('._.', 'summary', None, None): 'The text is too short to be summarized.'}

Am I doing something wrong, @rmitsch?

[rmitsch]

See here:

[rmitsch]

I. e. the summary is, by default, available in doc._.summary.

[rmitsch]

I'll convert this to a discussion, as this is not an issue.