Sources, filtering, and clusters
Control the pipeline input and understand why an item passed or disappeared.
Supported sources
SemanticHub currently supports:
- websites publishing RSS, Atom, or XML sitemaps;
- public Telegram channels;
- public Bluesky profiles.
Facebook URLs are rejected. Other social platforms are not currently supported as source types.
Adding a website
You add sources in the goal wizard, or later in Goal → Sources → Add source — by searching the library, pasting a site address, or giving a feed address directly. SemanticHub normalizes the domain, discovers feeds and sitemaps, reads basic metadata, and checks whether the source already exists. The address must resolve to a publicly routable host.
Separately, the goal's research adds portals it finds on its own; they appear under Suggested portals for you to accept or dismiss.
Reading source health
- Active means that a goal uses the source and it is scheduled for crawling.
- Last crawl shows when SemanticHub last attempted to check it.
- 24-hour traffic shows recent item throughput.
- Broken indicates that the source returns no usable content or repeatedly fails.
Do not assume a fixed time to first data. The source schedule and its own publishing cadence both matter.
Relevance filtering
By default, the model receives a title and a short excerpt. Full-content evaluation supplies more context but processes substantially more tokens.
Filter decisions and their reasoning live in the Filter tab; what the filter let through is in the Feed, behind the Before filter / After filter switch.
A useful filter instruction contains:
- the subject and relevant geography;
- event types that should pass;
- concrete rejection cases;
- exceptions where a title alone may be misleading.
Disabling filtering or leaving the instruction empty accepts every new item for grouping.
Clustering controls
| Setting | What it controls |
|---|---|
| Similarity threshold | How close two items must be. A higher value creates narrower topics. |
| Minimum articles | How many items a topic needs before it can proceed. |
| Immediate threshold | How many items let processing begin without waiting for holdback. |
| Holdback | The maximum collection window after a topic reaches its minimum. |
| Debounce | A short window that batches new items before regrouping. |
| Article lifetime | How long an item remains active for this goal. |
Tune with evidence
- Too many irrelevant items: add negative examples to the relevance instruction.
- Relevant items are rejected: enable full content or broaden the instruction.
- One event splits into many clusters: lower the similarity threshold.
- Different events merge together: raise the similarity threshold.
- Topics take too long to proceed: lower the minimum or shorten holdback.