NLP

nlp.analyze()

Analyze text into typed Token objects via the embedded UDPipe model.

Usage

Source

nlp.analyze(text)

Tokenizes, tags, and lemmatizes text, resolves each token’s codepoint span in the source, looks up its surface polarity, and flags clause-local negation. Negation is scoped forward within a sentence: a negator (lemma in not, no, never, none, nothing, without) flags every following content token until a clause boundary (punctuation or a coordinating/subordinating conjunction). A negator never flags itself. This is a coarse approximation — it does not handle not only, litotes, or idioms — which is acceptable for short feedback text.

Example

>>> [(t.form, t.upos) for t in analyze("this isn't broken")]

[(‘this’, ‘PRON’), (‘is’, ‘AUX’), (“n’t”, ‘PART’), (‘broken’, ‘VERB’)]

nlp.Token

One analyzed token: its surface, linguistics, source span, and sentiment.

Usage

Source

nlp.Token(form, lower, lemma, upos, start, end, polarity, negated)

Attributes

form: str

The surface text exactly as it appears in the source.

lower: str

form lowercased — the key surface polarity is looked up on.

lemma: str

The dictionary form UDPipe assigns (n't and cannot lemmatize to not).

upos: str

The universal part-of-speech tag (ADJ, VERB, PUNCT, …).

start: int

Codepoint offset of form in the source text.

end: int

Codepoint offset one past form in the source text.

polarity: int

Surface-keyed sentiment polarity, 0 when the token is neutral.

negated: bool

Whether a preceding clause-local negator scopes this token.