software/README.md

1# Japanese Bible Translator23A machine-assisted Bible translation system that generates modern Japanese translations from original Greek (New Testament) and Hebrew (Old Testament) source texts using Claude AI.45> **Disclaimer**: This project produces AI-generated translations for experimental and educational purposes. These translations are not endorsed by any religious organization and should not be used as authoritative scripture. Always consult established translations and qualified scholars for serious biblical study.67## Features89- **Full Bible Support**: Translate all 66 books (39 OT + 27 NT)10- **Source Text Fidelity**: Works directly from MorphGNT (Greek) and MorphHB (Hebrew) morphologically-tagged texts11- **Parallel Processing**: Fault-tolerant orchestration with multiprocessing for efficient batch translation12- **Translation Memory**: Maintains consistency across translations with segment reuse13- **Pericope-Based Translation**: Translates coherent narrative units for better contextual accuracy14- **Configurable Style**: Adjust equivalence approach, formality, gender language, and theological tradition15- **Automatic Evaluation**: Quality scoring based on configurable criteria16- **Progress Tracking**: SQLite-based job state with crash recovery and resume capability17- **Rich CLI**: Progress bars, status displays, and interactive TUI viewer1819## Requirements2021- Python 3.11+22- [uv](https://github.com/astral-sh/uv) (recommended) or pip23- [Claude CLI](https://claude.ai/claude-code) with active Anthropic subscription2425> **Note**: This project requires the Claude CLI binary (`claude`) which requires an Anthropic subscription. API costs will apply for translations.2627## Installation2829```bash30# Clone the repository31git clone https://github.com/yourusername/j-t-bible-translator.git32cd j-t-bible-translator3334# Install dependencies35uv sync3637# Install dev dependencies (for testing)38uv sync --extra dev39```4041## Quick Start4243### 1. Download Source Texts4445```bash46# Download all Bible source texts (Greek NT, Hebrew OT, reference translations)47uv run python scripts/download_sources.py --all48```4950### 2. Translate a Single Verse5152```bash53uv run python main.py verse john:1:154```5556### 3. Translate a Chapter5758```bash59uv run python main.py chapter john 160```6162### 4. Translate an Entire Book6364```bash65uv run python main.py book john66```6768### 5. Full Bible Translation (Parallel)6970```bash71# Translate all 66 books with 4 parallel workers72uv run python main.py bible --parallel 47374# NT only75uv run python main.py bible --nt-only7677# OT only78uv run python main.py bible --ot-only7980# Specific books81uv run python main.py bible --books john matthew genesis psalms82```8384### 6. Resume Interrupted Translation8586```bash87# Resume the most recent incomplete job88uv run python main.py resume8990# Resume a specific job91uv run python main.py resume --job-id <job-id>92```9394## CLI Commands9596| Command | Description |97|---------|-------------|98| `verse <ref>` | Translate a single verse (e.g., `john:1:1`) |99| `chapter <book> <num>` | Translate a chapter |100| `book <name>` | Translate an entire book |101| `bible` | Full Bible translation with parallel processing |102| `resume` | Resume an interrupted translation job |103| `job-status <id>` | Check status of a translation job |104| `jobs list` | List all translation jobs |105| `jobs clean` | Clean old jobs from database |106| `jobs stats` | Show database statistics |107| `status <book>` | Show translation progress for a book |108| `export <book> <formats>` | Export translations (json, text, html) |109| `style show` | Show current translation style settings |110| `style edit` | Modify translation style settings |111| `evaluate <book> --chapter <n>` | Evaluate translation quality |112| `pericope list <book>` | List pericopes for a book |113| `pericope translate <id>` | Translate a specific pericope |114| `tm status` | Show translation memory statistics |115| `tui` | Launch interactive TUI viewer |116117## Translation Style Configuration118119Configure translation approach via CLI or `data/config/style.json`:120121```bash122# Show current style123uv run python main.py style show124125# Edit style settings126uv run python main.py style edit --equivalence dynamic --formality standard127```128129### Available Options130131| Setting | Options |132|---------|---------|133| Equivalence | `formal`, `dynamic`, `optimal` |134| Gender Language | `traditional`, `moderate_inclusive`, `fully_inclusive` |135| Readability | `scholarly`, `general_public`, `children` |136| Theological Tradition | `evangelical`, `catholic`, `orthodox`, `ecumenical` |137| Term Consistency | `strict`, `balanced`, `contextual` |138| Formality | `casual`, `standard`, `formal` |139140## Project Structure141142```143j-t-bible-translator/144├── main.py                 # CLI entry point145├── src/146│   ├── context/           # Context building (glossary, pericopes, TM)147│   ├── data/              # Data loading and storage148│   ├── evaluation/        # Translation quality evaluation149│   ├── models/            # Data models150│   ├── orchestration/     # Parallel job orchestration151│   ├── output/            # Export formatters152│   ├── pipeline/          # Translation pipeline153│   ├── tui/               # Terminal UI154│   └── validation/        # Translation validation155├── data/156│   ├── sources/           # Downloaded source texts (gitignored)157│   ├── output/            # Generated translations (gitignored)158│   ├── config/            # Style configuration159│   ├── memory/            # Translation memory160│   └── metadata/          # Glossary, pericopes, book context161├── scripts/               # Utility scripts162└── tests/                 # Test suite (98%+ coverage)163```164165## Data Sources and Attribution166167This project uses the following open-licensed source texts:168169| Source | Description | License |170|--------|-------------|---------|171| [MorphGNT](https://github.com/morphgnt/sblgnt) | Greek NT (SBLGNT) with morphological tagging | CC-BY-SA 4.0 |172| [MorphHB](https://github.com/openscriptures/morphhb) | Hebrew OT (Westminster Leningrad Codex) with morphology | CC-BY 4.0 |173| [OpenBible](https://github.com/churchstudio-org/openbible) | Reference translations (KJV, Kougo, etc.) | MIT |174175### Reference Translations176177- **KJV (King James Version)** - Public domain178- **ASV (American Standard Version)** - Public domain179- **Kougo-yaku (口語訳)** - 1955 Japanese translation180- **Meiji-yaku (明治訳)** - 1887 Japanese translation181- **Taisho-yaku (大正訳)** - 1917 Japanese translation182183## Translation Memory184185The system maintains a translation memory (`data/memory/segments.json`) that stores previously translated segments for consistency. This file contains AI-generated translations and is included in the repository for development purposes.186187> **Note**: All translations in this repository are machine-generated and should be treated as experimental output, not authoritative translations.188189## Testing190191```bash192# Run all tests with coverage193uv run pytest tests/ --cov=src --cov-report=term-missing194195# Run specific test file196uv run pytest tests/test_pipeline.py -v197```198199The project maintains 98%+ test coverage.200201## Job Management202203Translation jobs are tracked in a SQLite database (`data/jobs/state.db`) for fault tolerance:204205```bash206# List recent jobs207uv run python main.py jobs list208209# Show database statistics210uv run python main.py jobs stats211212# Clean completed jobs (keep latest 2)213uv run python main.py jobs clean --keep-latest 2214215# Delete jobs older than 30 days216uv run python main.py jobs clean --older-than-days 30217218# Delete a specific job219uv run python main.py jobs clean --job-id <job-id>220```221222## License223224This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.225226## Disclaimer227228This software is provided for experimental and educational purposes only.229230**Important notices:**2312321. **Not Authoritative Scripture**: Translations produced by this software are AI-generated and have not been reviewed by qualified biblical scholars or endorsed by any religious organization.2332342. **Use at Your Own Risk**: These translations may contain errors, inaccuracies, or theological interpretations that differ from established translations.2352363. **For Study and Research**: This tool is intended for exploring machine translation capabilities and should not replace established Bible translations for worship, teaching, or personal devotion.2372384. **API Costs**: Using this software incurs costs through the Anthropic API. Users are responsible for their own API usage and associated charges.2392405. **No Warranty**: The software and its output are provided "as is" without warranty of any kind.241242For serious biblical study, please consult established translations and qualified scholars.243244## Contributing245246Contributions are welcome! Please ensure:2472481. All tests pass (`uv run pytest`)2492. Coverage remains at 98%+2503. Code follows existing patterns251252## Acknowledgments253254- [Anthropic](https://anthropic.com) for Claude AI255- [MorphGNT](https://github.com/morphgnt/sblgnt) project for Greek NT texts256- [OpenScriptures](https://github.com/openscriptures/morphhb) for Hebrew OT texts257- [OpenBible](https://github.com/churchstudio-org/openbible) for reference translations258259## Verify 日常訳第1版 before migration260261Run these read-only, standard-library scripts from the repository root with262Python 3.11+. Defaults resolve relative to the scripts, so another working263directory also works. Neither script edits the translations or site assets.264265```sh266python3 cli/translate-bible/scripts/verify_output.py --report /tmp/nichijo-v1-verification.json267python3 cli/translate-bible/scripts/review_output.py --report /tmp/nichijo-v1-review.csv268python3 -m unittest discover -s cli/translate-bible/tests -p test_output_audit.py269```270271Both scripts accept `--source /path/to/output` and `--baseline /path/to/JapKougo.csv`.272The default source is `cli/translate-bible/data/output/`; the baseline is273`web/japanbible-net/web/public/JapKougo.csv`.274275- `verify_output.py` checks JSON syntax and duplicate keys, canonical paths,276  chapter metadata, verse keys, nonempty text, CSV-incompatible control characters,277  markup/code fences/replacement characters, and furigana base-text agreement.278  It compares every reference against the main site's 66-book, 1,189-chapter corpus,279  including baseline rows with empty text. Missing/extra references require280  versification review; matching references do **not** prove matching verse content.281  Empty/absent furigana uses the existing exporter's plain-text fallback.282  The JSON report contains all findings and SHA-256 hashes of the scanned files.283- `review_output.py` writes a CSV queue with the translation, corresponding Kougo284  text, and heuristic flags for source scripts, Latin text, commentary/placeholders,285  repetition, identical verses within a chapter, and extreme length differences.286  Repeated verses and length differences may be legitimate. This is a review aid,287  not a semantic or theological accuracy check. Length comparisons skip empty288  baseline text. It scans valid text records even when structural findings exist.289290Exit codes for both scripts: `0` means no structural/reference findings, `1` means291findings need resolution, and `2` means an input/report could not be processed.292Heuristic flags alone do not cause a nonzero exit. The structural check is deliberately293strict about raw newlines even though the experimental exporter flattens them:294review them before deciding whether to normalize or correct the source.295296The initial working-copy findings are recorded in297[reports/nichijo-v1-initial-audit.md](reports/nichijo-v1-initial-audit.md).298Run against a stable output directory (pause translation writes or use a snapshot)299when preparing a release; file hashes identify the data read, but the scan is not300an atomic snapshot of a live translation job.301302The subsequent [correction review](reports/nichijo-v1-correction-review.md)303records 147 applied fixes, all Opus 5 review decisions, reported usage, validation,304and the remaining source-numbering alignment work. Exact original/corrected text305and hashes are preserved in its linked correction manifest.306307### Reviewed numbering and original-language parallels308309Run these offline commands from `cli/translate-bible`:310311```sh312uv run python main.py audit-alignment313uv run python main.py build-originals314uv run pytest tests/test_alignment.py tests/test_versification.py --cov=src.output.alignment --cov=src.output.originals --cov=src.data.versification --cov-fail-under=100315```316317`audit-alignment` applies the reviewed routes to a candidate report, leaving the318harness's source-numbered JSON intact. It rejects changes to the reviewed input319snapshot or Kougo text. Sentence splits must retain every character exactly once;320joins and duplicate selections require explicit, snapshot-bound decisions.321The report includes aligned changes, all 206 extra references, rejected variants,322conflicting candidates, and missing passages. Exit `1` means numbering work remains;323it is not permission to publish a partial translation. See the324[numbering review](reports/nichijo-v1-numbering-alignment.md).325326`build-originals` writes 66 lazy-loaded book assets to the main site's327`web/public/originals`. The source map, reviewed word boundaries, language tags,328Psalm headings, and source omissions are checked independently of the Japanese329translation. Its manifest records source and asset hashes; `NOTICE.txt` records330attribution. The chapter and passage readers offer a saved original-language331toggle, with Hebrew/Aramaic rendered right-to-left and Koine Greek left-to-right.332Run the website's normal build after regenerating these assets.333334### Importing verified passage repairs and exporting 日常訳第1版335336The completed [repair round](reports/numbering-repairs/README.md) resolves the337128 missing passages, 249 duplicate choices and six ambiguous records. Its 379338repairs include Opus 5 subscription CLI proposals, local comparisons and exact339final-text hashes. Before-review inputs and rejected candidates remain in the340audit. Reproduce the offline import, audit and export from this directory:341342```sh343uv run python main.py import-alignment-repairs --review reports/numbering-repairs/before-review.json --bundle reports/numbering-repairs/repair-bundle.json --out /tmp/nichijo-reviewed.json344uv run python main.py audit-alignment --review /tmp/nichijo-reviewed.json --out /tmp/nichijo-aligned.json345uv run python main.py export-aligned --audit /tmp/nichijo-aligned.json --out /tmp/nichijo-release346uv run python scripts/verify_aligned_release.py --audit /tmp/nichijo-aligned.json --release /tmp/nichijo-release347```348349Use an empty export destination. Import rejects incomplete verification, changed350candidate text, mismatched source selections and changed input review hashes.351Audit checks the raw translation, reference and original-source snapshots. Export352requires zero unresolved queues and leaves raw translation files intact. The353verification script checks every exported reference and text against the audit,354all repair texts, JSON keys, asset hashes and invalid text characters. These355commands make no model calls. The release is plain Japanese text without furigana;356website publication is a separate integration step.357358## Exact request archive359360Every synchronous and asynchronous `Translator` CLI attempt now writes an361exclusive input record before invoking Claude and a separate output record362with untrimmed stdout/stderr, return code and UTC timestamps. Retries get new363identities; an input without an output indicates an interrupted attempt.364Both verse and chunk translation, including evaluation through this boundary,365use this archive. Files are local, mode 0600, under `data/request-archive/`366(gitignored). Archive write failures stop the call rather than silently losing367the record. Timeouts and missing CLI failures are recorded; timed-out async368children are killed and awaited. Provider-internal context is not observable.369370These are future records, not reconstructed historical evidence. Prompts may371include independently licensed references: inspect and clear individual fields372before any public export. Never copy this directory wholesale into a release.373