> ## Documentation Index
> Fetch the complete documentation index at: https://gladlabs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Quality service

# Quality Service (Unified)

**File:** `src/cofounder_agent/modules/content/quality_service.py`
**Tested by:** `src/cofounder_agent/tests/unit/services/test_quality_service.py`
**Last reviewed:** 2026-04-30

## What it does

`UnifiedQualityService.evaluate(content, context, method)` runs a
single pass of the seven-criteria quality framework — clarity,
accuracy, completeness, relevance, SEO, readability, engagement —
against generated content and returns a `QualityAssessment` with the
per-dimension scores, an overall score (0-100), a pass/fail flag, and
a list of refinement suggestions.

This service is the *quality scoring* layer (was content quality
score in the legacy pipeline). It complements `MultiModelQA`
(adversarial reviewer fan-out) and `content_validator` (programmatic
hard rules) — quality\_service produces a numeric grade with feedback,
the others produce pass/fail decisions.

Three evaluation modes:

* **`PATTERN_BASED`** (default) — fast, deterministic. Heuristics
  delegated to `services.quality_scorers` (per-dimension functions),
  plus in-service artifact detection (photo metadata, leaked image-gen prompt
  prompts, raw HTML), LLM pattern detection (cliché openers,
  buzzwords, filler phrases, generic transitions, repetitive starters,
  listicle titles, hedging, exclamation spam, formulaic structure),
  and Flesch-Kincaid grade-level scoring. No LLM calls.
* **`LLM_BASED`** — sends content + dimension JSON schema to the
  injected `llm_client`, parses the response. Falls back to
  `PATTERN_BASED` if no client is configured, the response has no
  parseable JSON, or the call errors.
* **`HYBRID`** — runs both, averages the dimension scores 50/50.
  Falls back to pattern-only if LLM is unavailable or itself fell
  back to pattern (avoids double-counting).

The service also tracks running statistics (`total_evaluations`,
`passing_count`, `failing_count`, `average_score`) for the lifetime
of the instance.

## Public API

* `UnifiedQualityService(database_service=None, qa_agent=None, llm_client=None, *, site_config)` —
  constructor. `site_config` is required (keyword-only, DI'd); the rest
  are optional — only `database_service` is needed for persistence and
  only `llm_client` for `LLM_BASED` / `HYBRID`.
* `await qs.evaluate(content, context=None, method=EvaluationMethod.PATTERN_BASED, store_result=True) -> QualityAssessment` —
  main entry point. `context` may include `topic`, `keywords`,
  `audience`, `target_length`, `task_id`, `content_id`. When
  `store_result=True` and `database_service` is set, persists to the
  `quality_evaluations` table via
  `database_service.create_quality_evaluation(...)`.
* `qs.detect_truncation(content) -> bool` — static helper; returns
  True if the LLM appears to have hit its output token limit.
  Truncated content cannot pass regardless of overall score.
* `qs.flesch_kincaid_grade_level(text) -> float` — static helper;
  delegates to `quality_scorers.flesch_kincaid_grade_level`.
* `qs.get_statistics() -> dict` — running counters and pass-rate.
* Module factories:
  * `get_quality_service(database_service=None, llm_client=None, *, site_config)`
  * `get_content_quality_service(...)` — backward-compat alias.
* Backward-compat aliases:
  * `ContentQualityService = UnifiedQualityService` (class alias).
* Re-exported types (so callers don't need to import from
  `quality_models`):
  * `EvaluationMethod` (`PATTERN_BASED`, `LLM_BASED`, `HYBRID`)
  * `QualityAssessment`, `QualityDimensions`, `QualityScore`,
    `RefinementType`

`QualityAssessment` shape (from `services.quality_models`):

* `dimensions: QualityDimensions` — seven 0-100 scores
* `overall_score: float` — 0-100 (after artifact + LLM-pattern penalties)
* `passing: bool` — `overall_score >= qa_pass_threshold` AND not truncated
* `feedback: str` — human-readable summary
* `suggestions: list[str]` — refinement hints
* `evaluation_method: EvaluationMethod`
* `content_length: int`, `word_count: int`
* `flesch_kincaid_grade_level: float`
* `truncation_detected: bool`

## Configuration

Pipeline-wide thresholds are loaded from `app_settings` via
`quality_scorers.qa_cfg()` (called as `_qa_cfg()` inside this service).
Every threshold has a sensible default — see `quality_scorers.py` for
the full list. The most-touched ones:

* `qa_pass_threshold` (default `70.0`) — overall-score cut-off for
  `passing=True`.
* `qa_critical_floor` (default `50.0`) — minimum-dimension floor (if
  clarity/readability/relevance falls below this the overall is capped
  at that value).
* `qa_artifact_penalty_per` (default `5.0`) — points subtracted per
  artifact category found (photo metadata, image-gen prompt leak, etc.).
* `qa_artifact_penalty_max` (default `20.0`) — total artifact-penalty cap.
* `qa_fk_target_min` / `qa_fk_target_max` (defaults `8.0` / `12.0`) —
  Flesch-Kincaid grade-level acceptance band.

LLM-pattern detection (`_score_llm_patterns`) — the bulk of the
DB-tunable surface, all under the `qa_llm_*` prefix. Toggle the entire
detector with `qa_llm_patterns_enabled` (default `True`). Per-pattern
thresholds (selected — see `quality_service.py` lines 587-615 for the
full set):

* `qa_llm_buzzword_warn_threshold` (`3`) / `qa_llm_buzzword_fail_threshold` (`5`)
* `qa_llm_buzzword_penalty_per` (`0.5`) / `qa_llm_buzzword_max_penalty` (`5.0`)
* `qa_llm_filler_warn_threshold` (`2`) / `qa_llm_filler_fail_threshold` (`4`)
* `qa_llm_opener_penalty` (`5.0`) — cliché AI opener
* `qa_llm_transition_penalty_per` (`1.0`) / `qa_llm_transition_min_count` (`2`)
* `qa_llm_listicle_title_penalty` (`2.0`)
* `qa_llm_hedge_ratio_threshold` (`0.02`) / `qa_llm_hedge_penalty` (`2.0`)
* `qa_llm_repetitive_starter_penalty_per` (`1.0`) / `qa_llm_repetitive_min_count` (`3`)
* `qa_llm_formulaic_structure_penalty` (`2.0`) /
  `qa_llm_formulaic_min_avg_words` (`50`) / `qa_llm_formulaic_variance` (`0.2`)
* `qa_llm_exclamation_threshold` (`5`) / `qa_llm_exclamation_penalty_per` (`0.3`)

Per-dimension scoring tunables (clarity word-per-sentence bands,
accuracy citation bonuses, completeness word-count step function,
relevance keyword-density gates, SEO/engagement baselines) all live
under the `qa_*` prefix in `quality_scorers.qa_cfg()`.

## Dependencies

* **Reads from:**
  * `services.quality_scorers` — every per-dimension scorer plus the
    `qa_cfg()` config loader.
  * `services.quality_models` — `EvaluationMethod`, `QualityAssessment`,
    `QualityDimensions`, `QualityScore`, `RefinementType` types.
  * `services.site_config` — indirectly via `quality_scorers.qa_cfg()`
    and directly inside `_score_llm_patterns()` for the `qa_llm_*` keys.
  * Injected `llm_client` (when `method != PATTERN_BASED`).
* **Writes to:**
  * `quality_evaluations` table — only when `database_service` is
    injected and `store_result=True`. Persistence is best-effort; any
    exception is logged at error and swallowed (the assessment still
    returns to the caller).
* **External APIs:** none directly. The injected `llm_client` is what
  talks to Ollama/cloud.
* **Sister-service callers:**
  * `modules.content.stages.quality_evaluation` — the `quality_evaluation`
    graph\_def node (moved from `services/stages/` to `modules/content/stages/`
    during the 2026-06-04 content-module migration).
  * `main.py` — constructed at startup as `UnifiedQualityService()` via the
    `modules.content.api` thin-adapter boundary.

## Failure modes

* **`evaluate()` raises** — outer try/except catches anything from the
  per-method branches, logs `[_evaluate] Evaluation failed: <e>` at
  ERROR with traceback, returns a stub assessment (all 5.0/10, passing
  False, evaluator `UnifiedQualityService-Error`). Pipeline keeps going.
* **LLM client returns malformed JSON** — `_evaluate_llm_based`
  catches `JSONDecodeError`/`KeyError`/`TypeError`/`ValueError` and
  falls back to `PATTERN_BASED`. The LLM call itself can throw — also
  caught, also fallback.
* **No `llm_client` and `LLM_BASED` requested** — logs warning,
  returns a `PATTERN_BASED` assessment.
* **`HYBRID` with LLM fallback** — if `_evaluate_llm_based` returned
  a `PATTERN_BASED` result (because of one of the failures above),
  hybrid returns the pattern result alone (no double-weighting).
* **Truncated content** — `_evaluate_pattern_based` always sets
  `passing=False` regardless of score, and inserts an explicit
  truncation suggestion at the top of the suggestions list. The score
  itself is NOT zeroed — the dimensions reflect what's there.
* **Persistence failure** — `_store_evaluation` catches all exceptions,
  logs at error. Caller sees a successful assessment; the row just
  isn't there. No retry.
* **Missing `task_id` / `content_id` in context** — `_store_evaluation`
  logs at debug and returns without writing. (Without an ID the row
  has nothing to FK against; silent skip is the right call.)
* **`qa_llm_patterns_enabled = false`** — entire LLM-pattern detector
  short-circuits to `(0.0, [])`. Score is unaffected by buzzwords,
  filler, etc. Useful when validating a deliberately-stylized post.

## Common ops

* **Lower the pass bar for a genre that scores low for legitimate
  reasons** (e.g. very short news posts):
  `poindexter settings set qa_pass_threshold 60`
* **Disable buzzword penalties temporarily:**
  `poindexter settings set qa_llm_buzzword_penalty_per 0`
  (or the nuclear option: `qa_llm_patterns_enabled false`).
* **Inspect recent quality evaluations:**
  `SELECT created_at, overall_score, passing, evaluation_method
  FROM quality_evaluations
  ORDER BY created_at DESC LIMIT 50;`
* **Find LLM-pattern-heavy posts** — search the suggestions JSON
  column on `quality_evaluations` for `"AI writing pattern"` to see
  how often the writer falls back on slop patterns by category.
* **Run a one-off evaluation in the REPL:**
  ```python theme={null}
  import asyncio
  from modules.content.quality_service import UnifiedQualityService, EvaluationMethod
  qs = UnifiedQualityService()
  result = asyncio.run(qs.evaluate("# Post\n\nBody...", context={"topic": "fastapi"}))
  print(result.overall_score, result.passing, result.suggestions)
  ```
* **Debug "why did this pass with a 50?"** — check the truncation flag
  AND the FK grade-level vs target band; the suggestions list usually
  spells out the mid-tier reasons.

## See also

* `docs/architecture/services/multi_model_qa.md` — companion adversarial
  reviewer; uses quality scoring as one of several inputs.
* `docs/architecture/services/content_validator.md` — companion
  programmatic hard-rule layer (no scoring; pure pass/fail).
* `docs/architecture/anti-hallucination.md` — full QA pipeline picture.
* `services.quality_scorers` — per-dimension scoring functions and
  the `qa_cfg()` settings dictionary.
* `services.quality_models` — data classes for assessments, dimensions,
  evaluation methods, refinement types.