Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Show useful data about token URLs #192

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions .github/workflows/check_links.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
name: Check image links
runs-on: ubuntu-latest
env:
lychee_args: '--no-progress --require-https --cache --max-cache-age 1h --exclude http://www.w3.org'
lychee_args: '--no-progress --require-https --cache --max-cache-age 8h --exclude http://www.w3.org'

steps:
- name: Checkout
Expand All @@ -31,14 +31,42 @@ jobs:
key: cache-lychee-${{ github.sha }}
restore-keys: cache-lychee-

# Extract and check URLs directly from token files
# Extract and check URLs directly from token files (checking dumped file did not work)
- name: Check token art URLs
uses: lycheeverse/lychee-action@v1
with:
args: '${{env.lychee_args}} -- tokens.xml challenge_tokens.xml'
args: '${{env.lychee_args}} -- tokens.xml'
fail: true
jobSummary: true

# Extract picture URLs from tokens.xml
- name: Extract URLs
id: tokens_pic_urls
uses: lycheeverse/lychee-action@v1
with:
args: '--dump --exclude www.w3.org -- tokens.xml'
output: lychee/out.md
fail: true
jobSummary: false

# Analyse extracted links (1/2)
- name: List duplicated image links
if: steps.tokens_pic_urls.outcome == 'success'
shell: bash
# Remove blank lines | trim trailing integers (Scryfall) | sort | count and list duplicates
run: |
echo "🪞 **Duplicated Image Links**" >> $GITHUB_STEP_SUMMARY
grep . lychee/out.md | sed 's/\.jpg?.*/.jpg/' | sort | uniq -cd >> $GITHUB_STEP_SUMMARY

# Analyse extracted links (2/2)
- name: List image hosting sources
if: steps.tokens_pic_urls.outcome == 'success'
shell: bash
# Extract domains from URLs | remove blank lines | sort | count and list | sort descending
run: |
echo "📊 **Image Hosting Statistics**" >> $GITHUB_STEP_SUMMARY
awk -F/ '{print $3}' lychee/out.md | grep . | sort | uniq -c | sort -nr >> $GITHUB_STEP_SUMMARY

# Always save cache
- name: Save lychee cache
uses: actions/cache/save@v3
Expand Down