docs-quality.yml
The GitHub Actions workflow that runs the ruleset and annotates the pull request.
Two workflows. The pull-request check lints the changed files and checks every internal link, so it stays fast and still catches a rename that breaks a link in a file nobody touched. The nightly run lints the whole corpus and checks external links, which break without anyone touching the repository.
.github/workflows/docs-quality.yml
name: Docs quality
on:
pull_request:
paths:
- 'docs/**'
- '.vale.ini'
- '.vale/**'
# A new push supersedes the previous run. Without this, a branch pushed
# three times in a minute holds three runners to produce two stale results.
concurrency:
group: docs-quality-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
jobs:
prose:
name: Prose linting
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# The diff below needs the base branch's history.
fetch-depth: 0
- name: Collect changed documentation files
id: changed
run: |
FILES=$(git diff --name-only --diff-filter=ACMR \
origin/${{ github.base_ref }}...HEAD -- 'docs/**/*.md' \
| tr '\n' ' ')
echo "files=${FILES}" >> "$GITHUB_OUTPUT"
if [ -z "${FILES}" ]; then
echo "No documentation files changed."
fi
- name: Install Vale
if: steps.changed.outputs.files != ''
run: |
curl -sfL https://github.com/vale-cli/vale/releases/download/v3.24.0/vale_3.24.0_Linux_64-bit.tar.gz \
| sudo tar -xz -C /usr/local/bin vale
vale --version
- name: Sync external style packages
if: steps.changed.outputs.files != ''
run: vale sync
# Vale rules cannot read another file, so the version rule is built
# from versions.yml here. See VersionedCommands.yml.tmpl.
- name: Generate the version rule
if: steps.changed.outputs.files != ''
run: |
CURRENT=$(yq '.current' docs/_data/versions.yml)
sed -e "s/__CURRENT_RE__/${CURRENT//./[.]}/" \
-e "s/__CURRENT__/${CURRENT}/" \
.vale/styles/Docs/VersionedCommands.yml.tmpl \
> .vale/styles/Docs/VersionedCommands.yml
# Vale's own exit code is the gate. The annotation step below is for
# the writer's benefit and must not be what decides the result.
- name: Lint changed files
if: steps.changed.outputs.files != ''
run: |
vale --output=JSON ${{ steps.changed.outputs.files }} > vale.json || true
vale --output=line --minAlertLevel=error \
${{ steps.changed.outputs.files }}
- name: Check required front matter
if: steps.changed.outputs.files != ''
run: |
status=0
for f in ${{ steps.changed.outputs.files }}; do
for key in title description last_reviewed; do
value=$(yq --front-matter=extract ".${key} // \"\"" "$f")
if [ -z "$value" ]; then
echo "::error file=${f},line=1::Front matter is missing '${key}'."
status=1
fi
done
done
exit $status
# Runs even when linting failed, so the writer sees every finding
# rather than fixing them one CI run at a time.
- name: Annotate the diff
if: always() && steps.changed.outputs.files != ''
run: |
jq -r 'to_entries[] as $f | $f.value[] |
"::\(if .Severity == "error" then "error" else "warning" end) file=\($f.key),line=\(.Line),col=\(.Span[0])::[\(.Check)] \(.Message)"' \
vale.json
# One summary comment per pull request, rewritten on every run. A new
# comment per push buries the review thread.
- name: Update the summary comment
if: always() && steps.changed.outputs.files != ''
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const marker = '<!-- docs-quality-summary -->';
const data = JSON.parse(fs.readFileSync('vale.json', 'utf8'));
const all = Object.values(data).flat();
const errors = all.filter(a => a.Severity === 'error');
const byCheck = {};
for (const a of all) byCheck[a.Check] = (byCheck[a.Check] || 0) + 1;
const rows = Object.entries(byCheck)
.sort((a, b) => b[1] - a[1])
.map(([check, n]) => `| \`${check}\` | ${n} |`);
const body = all.length === 0
? `${marker}\n### Docs quality: no findings`
: [
marker,
`### Docs quality: ${errors.length} error(s), ` +
`${all.length - errors.length} warning(s)`,
'',
'Findings are annotated inline on the diff.',
'',
'| Rule | Count |',
'|---|---|',
...rows,
'',
errors.length
? '**Errors block the merge.** Each rule explains the ' +
'defect it prevents; the reasoning is in ' +
'`docs/contributing/style.md`.'
: 'No blocking errors. Warnings are advisory.',
].join('\n');
const { owner, repo } = context.repo;
const issue_number = context.issue.number;
const comments = await github.paginate(
github.rest.issues.listComments, { owner, repo, issue_number });
const existing = comments.find(c => c.body?.startsWith(marker));
if (existing) {
await github.rest.issues.updateComment({
owner, repo, comment_id: existing.id, body });
} else if (all.length > 0) {
await github.rest.issues.createComment({
owner, repo, issue_number, body });
}
links:
name: Internal links
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# --offline checks relative links and anchors only. External links are
# checked nightly: a third-party outage should not block a merge.
- name: Check internal links across the whole corpus
uses: lycheeverse/lychee-action@v2
with:
args: >-
--offline
--include-fragments
--no-progress
'docs/**/*.md'
fail: true
The link job checks the whole corpus, not only the changed files. A renamed page breaks the links that point at it, and those links are in files the pull request did not touch.
.github/workflows/docs-nightly.yml
name: Docs quality (full corpus)
on:
schedule:
- cron: '0 2 * * *'
workflow_dispatch:
permissions:
contents: read
issues: write
jobs:
full-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Vale
run: |
curl -sfL https://github.com/vale-cli/vale/releases/download/v3.24.0/vale_3.24.0_Linux_64-bit.tar.gz \
| sudo tar -xz -C /usr/local/bin vale
vale sync
- name: Generate the version rule
run: |
CURRENT=$(yq '.current' docs/_data/versions.yml)
sed -e "s/__CURRENT_RE__/${CURRENT//./[.]}/" \
-e "s/__CURRENT__/${CURRENT}/" \
.vale/styles/Docs/VersionedCommands.yml.tmpl \
> .vale/styles/Docs/VersionedCommands.yml
- name: Lint the whole corpus
run: vale --glob='*.md' --output=JSON docs > vale.json || true
# Findings here were not introduced by the change that surfaced them,
# so they go to one rolling issue rather than blocking anyone's merge.
- name: Update the rolling issue
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const data = JSON.parse(fs.readFileSync('vale.json', 'utf8'));
const errors = Object.entries(data).flatMap(([file, alerts]) =>
alerts
.filter(a => a.Severity === 'error')
.map(a => `- \`${file}:${a.Line}\` [${a.Check}] ${a.Message}`));
const { owner, repo } = context.repo;
const open = await github.rest.issues.listForRepo({
owner, repo, labels: 'docs-quality', state: 'open' });
const existing = open.data[0];
// Nothing to report: close the issue if one is open, so it never
// describes errors that have already been fixed.
if (errors.length === 0) {
if (existing) {
await github.rest.issues.update({
owner, repo, issue_number: existing.number, state: 'closed' });
}
return;
}
const shown = errors.slice(0, 50);
const body = [
'Full-corpus lint found errors not introduced by any single ' +
'pull request. Most come from a rule that was tightened ' +
'after the page was last edited.',
'',
...shown,
errors.length > shown.length
? `\n…and ${errors.length - shown.length} more. Run Vale locally for the full list.`
: '',
].join('\n');
const title = `Docs corpus: ${errors.length} error(s) on ` +
new Date().toISOString().slice(0, 10);
// One rolling issue, updated. A new issue every night is a
// notification channel people mute.
if (existing) {
await github.rest.issues.update({
owner, repo, issue_number: existing.number, title, body });
} else {
await github.rest.issues.create({
owner, repo, title, body, labels: ['docs-quality'] });
}
external-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check every link, including external ones
uses: lycheeverse/lychee-action@v2
with:
args: >-
--no-progress
--max-concurrency 8
--accept 200,206,429,999
'docs/**/*.md'
fail: true
--accept 999 admits LinkedIn, which returns that status to automated
clients, and 429 admits a site that rate-limited the checker rather than
reporting a dead link.
Running it locally
The same ruleset, before you open the pull request:
vale sync
vale docs/ # everything
vale docs/guides/install.md # one file
vale --minAlertLevel=error docs/ # only what blocks the merge
Generate VersionedCommands.yml with the sed command from the workflow
first. Without it, Vale skips the version check silently, and a stale version
passes locally only to fail in CI.
A pre-commit hook on errors only:
# .pre-commit-config.yaml
repos:
- repo: local
hooks:
- id: vale
name: Vale (errors only)
entry: vale --minAlertLevel=error
language: system
files: ^docs/.*\.md$
Errors only, deliberately. A hook that blocks a commit over a warning gets
bypassed with --no-verify, and a bypassed hook enforces nothing.