Filtering

FilterSpec

An ordered list of Clause rules applied to an event stream.

Usage

Source

FilterSpec(clauses)

Parameter Attributes

clauses: tuple[Clause, …]

filterspec.Clause

One filter rule: when predicate holds, apply action.

Usage

Source

filterspec.Clause(
    predicate,
    action=Action.DROP,
    applies_to=frozenset(),
    negate=False,
    label=None
)

Attributes

predicate: Predicate

The condition tested against an event.

action: Action

DROP removes the event (first drop wins, stops evaluation); TAG records label on a surviving event and continues.

applies_to: frozenset[EventKind]

When non-empty, the clause is only evaluated for these kinds.

negate: bool

Invert the predicate match.

label: str | None

The label recorded by a TAG clause.

filterspec.spec_to_json()

Serializes spec to the JSON contract consumed by the Rust backend.

Usage

Source

filterspec.spec_to_json(spec)

apply_spec()

Yields the already-materialized events that survive every DROP clause of spec.

Usage

Source

apply_spec(events, spec)

Streams events lazily: the spec compiles once, then each event is tested as it is pulled, so consumers can stop early without materializing the tail.

annotate_spec()

Yields (event, labels) for events surviving spec, with TAG labels, streaming lazily.

Usage

Source

annotate_spec(events, spec)

keep()

Returns whether event survives every DROP clause of spec.

Usage

Source

keep(event, spec)

Filters an already-materialized ~cc_transcript.models.TranscriptEvent; parse-time filtering runs in Rust before materialization via spec_to_json().

labels_for()

Returns the TAG labels spec records for event, in clause order.

Usage

Source

labels_for(event, spec)

build_spec()

Flattens Clause / tuple[Clause, ...] fragments into a FilterSpec.

Usage

Source

build_spec(*fragments)

keep_only()

Drops every event whose kind is not in kinds.

Usage

Source

keep_only(*kinds)

drop_synthetic()

Drops assistant events with the <synthetic> model.

Usage

Source

drop_synthetic()

drop_empty()

Drops blank events of one kind.

Usage

Source

drop_empty(*, only_from)

Assistant turns with a tool-use block are not blank; user turns have no such rescue, so consider_tool_use is keyed off only_from. Defined for the single-kind USERS / ASSISTANTS sets.

drop_sidechain()

Drops sidechain events; except_assistants keeps assistant sidechains.

Usage

Source

drop_sidechain(*, except_assistants=False)

drop_meta_flag()

Drops events whose EntryMeta boolean flag is set.

Usage

Source

drop_meta_flag(flag, *, only_from=frozenset())

drop_compacted()

Drops compaction-summary and transcript-only entries.

Usage

Source

drop_compacted()

drop_entrypoints()

Drops events whose meta.entrypoint is in entrypoints.

Usage

Source

drop_entrypoints(entrypoints)

drop_junk()

Drops events matching any group in the named JUNK_CATEGORIES.

Usage

Source

drop_junk(*categories, only_from=USERS)

drop_phrases()

Drops events whose normalized text is one of phrases.

Usage

Source

drop_phrases(phrases, *, only_from=USERS)

drop_short()

Drops events with at most max_words whitespace-split words.

Usage

Source

drop_short(max_words, *, only_from=USERS)

NOISE_SPEC

NOISE_SPEC=build_spec(drop_junk("structural"))


filterspec.JUNK_CATEGORIES

filterspec.JUNK_CATEGORIES: dict[str, tuple[tuple[str, str], …]] = {
    "structural": STRUCTURAL_GROUPS,
    "agent_injection": AGENT_INJECTION_GROUPS,
    "interrupt": INTERRUPT_MARKER_GROUPS,
    "stop_hook": STOP_HOOK_GROUPS,
    "continuation": CONTINUATION_GROUPS,
    "command_echo": COMMAND_ECHO_GROUPS,
}


USERS

USERS: frozenset[EventKind] = frozenset({"user"})


ASSISTANTS

ASSISTANTS: frozenset[EventKind] = frozenset({"assistant"})


RESUME_PHRASE_SET

RESUME_PHRASE_SET: frozenset[str] = frozenset(
    {
        "continue",
        "continues",
        "continue please",
        "please continue",
        "resume",
        "resume please",
        "please resume",
        "go ahead",
        "keep going",
        "carry on",
        "proceed",
        "go on",
        "keep at it",
        "ok continue",
        "ok resume",
        "ok go ahead",
        "okay continue",
        "okay resume",
        "okay go ahead",
        "continue from where you left off",
        "continue where you left off",
        "continue your previous task",
        "[context restored] resume",
        "[context restored] continue",
        "[context restored] pick up the conversation",
    }
)


TRIVIAL_ACK_SET

TRIVIAL_ACK_SET: frozenset[str] = frozenset(
    {
        "ok",
        "okay",
        "k",
        "kk",
        "yes",
        "yep",
        "yeah",
        "yup",
        "sure",
        "thanks",
        "thank you",
        "thx",
        "ty",
        "great",
        "perfect",
        "nice",
        "cool",
        "awesome",
        "done",
        "good",
        "lgtm",
        "got it",
        "sounds good",
        "looks good",
        "makes sense",
        "approved",
    }
)


STRUCTURAL_NOISE_RE

STRUCTURAL_NOISE_RE=compile_groups(STRUCTURAL_NOISE_GROUPS, True)


JUNK_USER_MESSAGE_RE

JUNK_USER_MESSAGE_RE=compile_groups(SENTIMENT_JUNK_GROUPS, True)