generative-ai, githubactions
📝 This article is a translation of the original Japanese post.
View originalAutomating Blog Post Reviews with Generative AI
I’ve been writing blog posts for about 8 years now, but since I write them on my own, nobody reviews them.
So I developed go-zen-chu/aictl to have articles reviewed via GitHub Actions when I open a PR.
There are products and tools like Copilot that do code review with generative AI, but there don’t seem to be many tools for having blog posts reviewed, and it looked easy to build using the OpenAI client, so I made one — also as a learning exercise.
How to Set It Up
The setup is described in the README of go-zen-chu/aictl, but in short:
- Place the yaml below at
.github/workflows/check-pr.yml - Set AICTL_OPENAI_API_KEY in the repository secrets
- For the API key, see API Reference - OpenAI API
and setup is done (secrets.GITHUB_TOKEN appears in the workflow, but it exists by default when GitHub Actions runs, so no configuration is needed).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
| name: check-pr
on:
pull_request_target:
types: [opened, synchronize]
jobs:
aictl-review:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Get number of commits for fetch depth
run: echo "fetch_depth=$(( commits + 1 ))" >> $GITHUB_ENV
env:
commits: ${{ github.event.pull_request.commits }}
- name: Git checkout until fetch-depth
uses: actions/checkout@v4
with:
fetch-depth: ${{ env.fetch_depth }}
# need to specify ref to fetch PR head, otherwise you get main branch with no HEAD
ref: ${{ github.event.pull_request.head.sha }}
- name: Fetch base branch as origin
run: git fetch origin ${{ github.event.pull_request.base.ref }}
- name: Get the diff between PR and origin branch and get commit message too
id: git-diff
run: |
# If fetch-depth is not specified in checkout, HEAD cannot be found
# Make sure to filter out deleted file in `git diff` since you cannot review deleted files
diff_files=$(git diff --name-only --diff-filter=ACMR origin/${{ github.event.pull_request.base.ref }}..HEAD | tr '\n' ',' | sed 's/,$/\n/')
echo "diff_files: ${diff_files}"
echo "diff-files=${diff_files}" >> "$GITHUB_OUTPUT"
# In PR, you cannot get `github.event.head_commit.message` so you need to get commit message via git log
commit_msg=$(git log --format=%B -n 1 ${{ github.event.pull_request.head.sha }})
echo "commit_msg: ${commit_msg}"
echo "commit-msg=${commit_msg}" >> "$GITHUB_OUTPUT"
- name: Run aictl for code review
id: aictl-review
uses: go-zen-chu/aictl@main
if: ${{ steps.git-diff.outputs.diff-files != '' && !contains(steps.git-diff.outputs.commit-msg, '[skip ai]') }}
with:
query: |
The Markdown files contain technical articles about programming, life hack articles, book reviews, and so on.
Please review them from the perspective of making them easier to read and reaching more readers.
The other files are the source code of the blog system itself.
Please review them from the perspective of code quality and maintainability so that a better blog system can be built.
language: 日本語
text-files: ${{ steps.git-diff.outputs.diff-files }}
env:
AICTL_OPENAI_API_KEY: ${{ secrets.AICTL_OPENAI_API_KEY }}
- name: Post aictl review result to PR comment
if: steps.aictl-review.outcome != 'skipped'
run: |
# make sure to checkout to pr branch and resolve detached HEAD state
git checkout ${{ github.ref_name }}
# TIPS: surrounding with single quote, you can ignore `code` string in outputs
cat <<'AICTL_REVIEW_EOF' > response.md
${{ steps.aictl-review.outputs.response }}
AICTL_REVIEW_EOF
gh pr comment --body-file response.md "${URL}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
URL: ${{ github.event.pull_request.html_url }}
|
Results
To explain the above, the flow is:
- It runs when a PR is created or updated
- It fetches all commits in the PR and produces the file diff (git-diff)
- It sends the diff from step 2 to the generative AI via aictl. The contents of
query are up to you- Putting
[skip ai] in the commit message skips execution
- It posts aictl’s output to the PR
Here’s an image of this very article being reviewed in a PR.
I’d say the reviews it gives are pretty good. If you want more detailed reviews, you can give more specific instructions about what to review in the query of aictl-review.
Limitations and Future Plans
It’s reasonably practical at this point, but here’s what I’d like to implement in aictl going forward.
- Right now it’s forced to the cheap GPT-4o-Mini model, so let the model be chosen freely
- Support other generative AI providers as well (this one looks time-consuming)
- Let it remember what was submitted in past PRs so it can understand context
I intend to keep developing it as I find the time.