Coverage Documentation

BELLHOP supports code coverage analysis for both Fortran and Python components. The reports are generated separately (one for each language).

Prerequisites

Coverage analysis requires:

For Fortran Coverage: - gfortran (GNU Fortran compiler) with GCOV support - gcc to provide gcov binary

For Python Coverage: - Python 3.12+ - coverage package (included in dev dependencies)

These are typically available on most Linux systems. On Ubuntu/Debian:

sudo apt install gfortran
pip install coverage

Generating Coverage Reports Locally

Complete Coverage Analysis (Recommended)

To generate coverage reports for both Fortran and Python:

make cov

This single command performs the complete workflow: - Builds Fortran code with coverage instrumentation and generates Fortran HTML reports - Runs Python tests with coverage and generates Python HTML reports

Individual Coverage Components

You can also run coverage analysis for individual components:

Fortran Coverage Only:

make coverage-clean
make coverage-build
make coverage-install
make coverage-test
make coverage-report
make coverage-html

Python Coverage Only:

make covp

Understanding Coverage Reports

The coverage analysis generates several types of files:

  • .gcno files: Coverage note files created during compilation
  • .gcda files: Coverage data files created when running instrumented executables
  • .gcov files: Human-readable coverage reports showing line-by-line execution counts

Coverage reports show: - Lines executed: Percentage of executable lines that were run - Branches executed: Percentage of conditional branches that were taken - Calls executed: Percentage of function/procedure calls that were made

Example coverage output:

File 'monotonicMod.f90'
Lines executed:100.00% of 8
Branches executed:100.00% of 16
Taken at least once:62.50% of 16

Viewing Coverage Reports

Coverage reports are available in multiple formats:

1. Individual Report Access

Fortran Coverage (GCOV): - Raw text reports: Check .gcov files in fortran/ directory - HTML reports: Open _coverage/coverage-index.html

Python Coverage: - HTML reports: Open _coverage_python/index.html

2. Coverage Report Features

Fortran Reports: - Line-by-line execution counts - Branch coverage analysis - Call coverage statistics - Color-coded coverage visualization

Python Reports: - Statement coverage percentages - Missing line identification - Function and class coverage - Interactive source code browsing Coverage reports are created as .gcov files in the fortran/ directory. Each report shows the original source code with execution counts:

        -: 1:!! Monotonicity testing utilities
        1: 2:MODULE monotonicMod
        -: 3:  IMPLICIT NONE

Where: - Numbers indicate how many times each line was executed - - indicates non-executable lines (comments, declarations) - ##### indicates executable lines that were never run

3. Interactive HTML Reports (FORD Integration)

For enhanced browsability, coverage reports are generated as interactive HTML reports:

make coverage-html  # Generate HTML reports in _coverage/

The HTML reports provide: - Interactive Coverage Dashboard - Overview of all source files with coverage statistics - Color-Coded Source Views - Line-by-line coverage visualization with execution counts - Coverage Metrics - Detailed line, branch, and call coverage percentages - Browsable Navigation - Easy switching between different source files

Accessing HTML Coverage Reports: - Locally: Generate with make coverage-html then open generated HTML files in _coverage/ - The coverage reports are standalone HTML files

3. Coverage Report Features

The HTML coverage reports include: - Green highlighting: Lines executed during testing - Red highlighting: Lines that were never executed - Gray highlighting: Non-executable lines (comments, declarations) - Execution counts: Number of times each line was executed - Summary statistics: Overall coverage percentages for lines, branches, and calls

GitHub Actions Integration

Code coverage analysis runs automatically in GitHub Actions:

Coverage Workflow

  • Triggered on: Pull requests and pushes to the main branch
  • Generates: Coverage analysis for both Fortran (GCOV) and Python (coverage.py)
  • Uploads: Coverage HTML reports to GitHub Pages (as part of the docs build)

Workflow Steps

  1. Build Phase: Compiles Fortran code with coverage instrumentation
  2. Test Phase: Runs full test suite with coverage collection for both languages
  3. Report Generation: Creates HTML reports for Fortran and Python
  4. Deployment: Uploads coverage reports to GitHub Pages for easy access

Accessing Reports

Coverage reports are automatically published to GitHub Pages and linked from the main documentation at: - Fortran Coverage: /static/coverage/_coverage/coverage-index.html - Python Coverage: /static/coverage/_coverage_python/index.html

Cleaning Coverage Files

To remove all coverage-related files:

make coverage-clean

This removes: - Fortran coverage data (.gcda, .gcno, .gcov files) - Python coverage data (.coverage file) - Generated Python HTML reports (_coverage_python/)

Note: _coverage/ (Fortran HTML reports) is not removed by make coverage-clean.