> ## Documentation Index
> Fetch the complete documentation index at: https://gladlabs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Tap builtin topic source

# Handler: `tap.builtin_topic_source`

Adapter that brings the existing in-repo topic\_source scrapers (hackernews, devto, web\_search, knowledge, codebase) into the declarative `external_taps` model.

## What it does

Delegates to `services.topic_sources.runner.run_all()` (the same runner the scheduler already uses) and filters the summary to the source name specified in `row.tap_type`. The actual topic persistence (dedup, similarity scoring, `content_task` row creation) happens inside the existing plugin — this handler is purely a shape adapter.

## Row configuration

```
name:            operator-chosen slug, usually == tap_type
handler_name:    builtin_topic_source
tap_type:        name of the in-repo topic_source plugin
                 (hackernews | devto | web_search | knowledge | codebase)
target_table:    content_tasks  (advisory — the plugin already knows)
schedule:        how often to fire (e.g. "every 1 hour")
config:          forwarded to the plugin via the topic_sources runner's
                 PluginConfig.load path. Free-form per-source config.
enabled:         false until operator flips on
```

## Seeded rows (all disabled)

The baseline migration seeds one row per current topic\_source plugin with sensible schedules:

| Name         | Schedule       |
| ------------ | -------------- |
| `hackernews` | every 1 hour   |
| `devto`      | every 2 hours  |
| `web_search` | every 6 hours  |
| `knowledge`  | every 12 hours |
| `codebase`   | every 1 day    |

## Operator runbook

### Enabling a seeded topic source

```
poindexter taps enable hackernews
poindexter taps run hackernews      # optional — fires immediately
```

Wait for the next scheduler tick and verify `last_run_at` populates:

```
poindexter taps list
```

### Adding a new built-in source

If you write a new `services/topic_sources/*.py` plugin and register it under `plugins.topic_source`, seed it as a tap row:

```sql theme={null}
INSERT INTO external_taps
  (name, handler_name, tap_type, target_table, schedule, enabled, metadata)
VALUES (
  'reddit',
  'builtin_topic_source',
  'reddit',
  'content_tasks',
  'every 2 hours',
  FALSE,
  jsonb_build_object('description', 'Reddit submissions from operator-defined subreddits')
);
```

### Disabling

```
poindexter taps disable hackernews
```

Immediately effective — no restart.

### Why this adapter exists

The existing topic\_source plugin architecture is already declarative-ish (via `plugin.topic_source.<name>` app\_settings rows). This adapter just brings the enablement into one table (`external_taps`) so:

1. Grafana can show tap state alongside webhook / retention state.
2. Operator CLI is uniform (`poindexter taps …` mirrors `webhooks …` and `retention …`).
3. Future Singer-protocol taps live in the same table — one list of "what is this system ingesting from?".

## Caveats

* **Full-runner invocation:** each handler call invokes `run_all()` and filters to the tap's name. When multiple built-in taps are enabled, each fires its own `run_all()`. The existing runner is idempotent via content\_hash dedup, so this is safe but slightly wasteful. Fixing this would require the topic\_sources runner to accept a single-source filter — tracked as a follow-up.
* **No per-source config isolation in v1:** the old `plugin.topic_source.<name>.config` app\_settings rows still drive per-source behavior. Moving that config into `external_taps.config` is a follow-up migration.

## Related

* Framework overview: [Integrations](/docs/integrations/index)
* Sibling handler: `tap.singer_subprocess` (stub — full external Singer tap support)
* Existing runner: `services/topic_sources/runner.py`
* GH-103 (external taps issue)