Skip to content

Commit a3dca82

Browse files
authored
ci(csubst): add ci-test workflow and scripts (kfuku52#79)
* Create tag_from_version.yml * Update tag_from_version.yml * Update tag_from_version.yml * Update tag_from_version.yml * Create bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Update bioconda_pr.yaml * Create bioconda_PR_automation * Rename bioconda_PR_automation to bioconda_PR_automation.yml * Rename bioconda_pr.yaml to bioconda_pr.yml * Update bioconda_pr.yml * Update bioconda_pr.yml * Update bioconda_pr.yml * Update bioconda_pr.yml * Create auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Delete .github/workflows/bioconda_PR_automation.yml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Delete .github/workflows/bioconda_pr.yml * Update auto_bioconda_pr.yaml * Update auto_bioconda_pr.yaml * Delete .github/workflows/auto_bioconda_pr.yaml * add smoke test * add smoke test * modified: pthon version * modified: csubst-smoke.yml * modified: csubst-smoke.yml * modified: csubst-smoke.yml and csubst_smoke.sh * modified: csubst-smoke.yml and csubst_smoke.sh * modified: temporary exclusion of bioconda from installing yml jobs. * modified: csubst_smoke.sh * modified: csubst-smoke.yml and csubst_smoke.sh * added: csubst_cli.sh, modified: csubst_smoke.sh and csubst-smoke.yml * modified: * added: csubst_commands.sh and command tests * modified: csubst_commands, point out --branch_id fg in the site command * modified: --foreground of ci/csubst_commands.sh * modified: csubst-ci.yml, add biopython in the Install runtime deps * modified: csubst-ci.yml * modified: csubst_commands.sh * modified: csubst_commands.sh * ci: trigger workflow * added: roundtrip test, test for output-tsv * modifid: add conda-forge::pyvolve line in the yaml file * modifid: yaml file * modifid: csubst-ci.yml * modifid: to aviod post ENOENT / not find ... in .bash_profile in csubst-ci.yml * modifid:csubst-ci.yml * modified: expansion of python versions * modified: expansion of python versions * fix: f-string; ci: skip site without PyMOL * fix: f-string; ci: skip site without PyMOL * ci(smoke): tolerate non-zero rc from analyze; skip site when PyMOL absent; harden CLI tests * ci(commands): skip site without PyMOL; skip simulate without pyvolve; keep output-based validation * ci(smoke): robust output-based validation; always skip site/simulate; artifact collect * fix: csubst-ci.yml and csubst_commands.sh * fix: ci works for master branch and workflow_dispatch
1 parent 6157102 commit a3dca82

File tree

6 files changed

+489
-1
lines changed

6 files changed

+489
-1
lines changed

.github/workflows/csubst-ci.yml

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
name: csubst ci
2+
3+
on:
4+
push:
5+
branches: [ "master" ]
6+
pull_request:
7+
branches: [ "master" ]
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: csubst-ci-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
jobs:
15+
smoke:
16+
runs-on: ${{ matrix.os }}
17+
timeout-minutes: 30
18+
strategy:
19+
fail-fast: false
20+
matrix:
21+
os: [ubuntu-latest, macos-latest]
22+
python: ['3.9', '3.10', '3.11', '3.12']
23+
install: [source]
24+
25+
steps:
26+
- name: Checkout
27+
uses: actions/checkout@v4
28+
29+
- name: Set up micromamba
30+
uses: mamba-org/setup-micromamba@v1
31+
with:
32+
environment-name: csubst-smoke
33+
create-args: >-
34+
python=${{ matrix.python }}
35+
iqtree numpy scipy cython biopython requests pip
36+
condarc: |
37+
channels:
38+
- conda-forge
39+
- bioconda
40+
- defaults
41+
channel_priority: strict
42+
cache-environment: true
43+
cache-downloads: true
44+
45+
# post の ENOENT 回避
46+
- name: Workaround micromamba post-step quirks
47+
if: always()
48+
run: |
49+
mkdir -p "$RUNNER_TEMP/setup-micromamba"
50+
: > "$RUNNER_TEMP/setup-micromamba/micromamba-shell"
51+
touch "$HOME/.bash_profile"
52+
grep -q 'micromamba activate csubst-smoke' "$HOME/.bash_profile" \
53+
|| echo 'micromamba activate csubst-smoke' >> "$HOME/.bash_profile"
54+
55+
# pyvolve を pip から
56+
- name: Install pyvolve (pip)
57+
shell: bash -l {0}
58+
run: |
59+
python -m pip install --upgrade pip
60+
pip install --no-input pyvolve==1.1.0
61+
62+
- name: Install runtime deps
63+
shell: bash -l {0}
64+
run: |
65+
micromamba install -y \
66+
numpy=1.26.* cython=0.29.* iqtree \
67+
conda-forge::biopython conda-forge::requests
68+
69+
- name: Install csubst (from source)
70+
if: matrix.install == 'source'
71+
shell: bash -l {0}
72+
run: |
73+
python -m pip install --upgrade pip
74+
pip install --no-build-isolation .
75+
76+
# --- Smoke ---
77+
- name: Run smoke test
78+
shell: bash -l {0}
79+
run: bash ci/csubst_smoke.sh
80+
81+
# --- CLI tests ---
82+
- name: Run CLI tests
83+
shell: bash -l {0}
84+
run: bash ci/csubst_cli.sh
85+
86+
- name: Upload artifacts (always)
87+
if: always()
88+
uses: actions/upload-artifact@v4
89+
with:
90+
name: smoke-and-cli-${{ matrix.os }}-${{ matrix.install }}
91+
path: |
92+
$RUNNER_TEMP/csubst_smoke/_artifacts*
93+
if-no-files-found: ignore
94+
retention-days: 7
95+
96+
# PyMOL の有無を検出(heredoc の閉じは行頭に PY)
97+
- name: Check PyMOL
98+
id: pymol
99+
shell: bash -l {0}
100+
run: |
101+
HAVE=$(python - <<'PY'
102+
try:
103+
import pymol # noqa: F401
104+
print("true")
105+
except Exception:
106+
print("false")
107+
PY
108+
)
109+
echo "have=$HAVE" >> "$GITHUB_OUTPUT"
110+
111+
- name: Run command tests
112+
shell: bash -l {0}
113+
env:
114+
MPLBACKEND: Agg
115+
HAVE_PYMOL: ${{ steps.pymol.outputs.have }}
116+
run: bash ci/csubst_commands.sh
117+
118+
- name: Validate outputs (schema-lite)
119+
shell: bash -l {0}
120+
run: python ci/check_csubst_outputs.py

ci/check_csubst_outputs.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/usr/bin/env python3
2+
import glob, sys, os
3+
4+
def check_tsv(path):
5+
with open(path, "r", encoding="utf-8", errors="ignore") as f:
6+
lines = [l.rstrip("\n") for l in f]
7+
assert len(lines) >= 2, f"{path}: 行数が少なすぎます"
8+
header = lines[0].split("\t")
9+
assert len(header) >= 2, f"{path}: タブ区切りのヘッダがありません"
10+
# 代表的なメトリクス名の痕跡(将来名前が変わっても緩めに)
11+
if os.path.basename(path).startswith("csubst_cb_stats"):
12+
assert any(("omegaC" in h) or ("OCN" in h) or ("OCS" in h) for h in header), \
13+
f"{path}: 代表メトリクス列が見当たりません"
14+
15+
def main():
16+
tsvs = sorted(glob.glob("csubst_*.tsv"))
17+
if not tsvs:
18+
print("WARN: csubst_*.tsv がありません(前段で生成できていればOKのはず)")
19+
sys.exit(0)
20+
for t in tsvs:
21+
check_tsv(t)
22+
print(f"OK: {t}")
23+
# IQ-TREE中間の存在も最終確認
24+
for req in ["alignment.fa.state", "alignment.fa.rate"]:
25+
assert os.path.exists(req) and os.path.getsize(req) > 0, f"{req}: 見つからない/空です"
26+
print("OK: IQ-TREE intermediates present")
27+
28+
if __name__ == "__main__":
29+
main()

ci/csubst_cli.sh

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
echo "== csubst CLI tests =="
5+
6+
which csubst
7+
csubst --version || true
8+
9+
# --- ヘルプ(主要サブコマンド/引数の存在) ---
10+
csubst -h | head -n 20
11+
csubst -h | grep -E '\banalyze\b' >/dev/null
12+
csubst -h | grep -E '\bdataset\b' >/dev/null
13+
csubst analyze -h | grep -- '--alignment_file' >/dev/null
14+
csubst analyze -h | grep -- '--rooted_tree_file' >/dev/null
15+
csubst analyze -h | grep -- '--foreground' >/dev/null
16+
17+
# 作業ディレクトリ(smoke と共用)
18+
WORKDIR="${RUNNER_TEMP:-$(mktemp -d)}/csubst_smoke"
19+
mkdir -p "$WORKDIR"
20+
cd "$WORKDIR"
21+
22+
# 最小データが無ければ生成
23+
if [ ! -s alignment.fa ]; then
24+
csubst dataset --name PGK
25+
fi
26+
test -s alignment.fa && test -s tree.nwk && test -s foreground.txt
27+
28+
# --- 異常系:欠損入力で失敗すること ---
29+
set +e
30+
csubst analyze --alignment_file __NO_FILE__.fa \
31+
--rooted_tree_file tree.nwk --foreground foreground.txt --threads 1 >/dev/null 2>&1
32+
rc=$?
33+
set -e
34+
[ $rc -ne 0 ] || { echo "ERROR: 欠損入力で成功してしまった"; exit 1; }
35+
echo "OK: 異常系で非0終了を確認"
36+
37+
# --- 正常系(ECM系:デフォルト) ---
38+
rm -f alignment.fa.{iqtree,log,rate,state,treefile} || true
39+
env PYTHONOPTIMIZE=1 OMP_NUM_THREADS=1 csubst analyze \
40+
--alignment_file alignment.fa \
41+
--rooted_tree_file tree.nwk \
42+
--foreground foreground.txt \
43+
--threads 1
44+
45+
shopt -s nullglob
46+
CB1=(csubst_cb_*.tsv)
47+
[ ${#CB1[@]} -ge 1 ] || { echo "ERROR: ECM 実行後に cb TSV が無い"; exit 1; }
48+
[ $(wc -l < "${CB1[0]}") -ge 2 ] || { echo "ERROR: cb TSV が空"; exit 1; }
49+
grep $'\t' "${CB1[0]}" >/dev/null || { echo "ERROR: TSV がタブ区切りでない可能性"; exit 1; }
50+
51+
# --- 正常系(GY系:別分岐) ---
52+
rm -f alignment.fa.{iqtree,log,rate,state,treefile} || true
53+
env PYTHONOPTIMIZE=1 OMP_NUM_THREADS=1 csubst analyze \
54+
--alignment_file alignment.fa \
55+
--rooted_tree_file tree.nwk \
56+
--foreground foreground.txt \
57+
--iqtree_model GY+F3x4+R2 \
58+
--threads 1
59+
CB2=(csubst_cb_*.tsv)
60+
[ ${#CB2[@]} -ge 1 ] || { echo "ERROR: GY 実行後に cb TSV が無い"; exit 1; }
61+
62+
# アーティファクト収集(任意)
63+
ART="$WORKDIR/_artifacts_cli"
64+
mkdir -p "$ART"
65+
cp -v csubst_cb_*.tsv "$ART" 2>/dev/null || true
66+
cp -v alignment.fa.{iqtree,log,rate,state,treefile} "$ART" 2>/dev/null || true
67+
68+
echo "CLI tests OK"

0 commit comments

Comments
 (0)