• jsmith-bdai's avatar
    Adds a runner script to execute all tests in the `source` directory (#327) · 51ccd99f
    jsmith-bdai authored
    # Description
    
    This PR adds a script to run all the unit tests (any Python script named
    `test_*.py`) and report on the success rate and timing. It also enables
    the skipping of known breaking tests via tests_to_skip.py. The tests can
    be run via `orbit -t`, and the test output will be printed to both a log
    (`test_results.log`) and to the console.
    
    This can be used for CI/CD but also can be temporarily used to run each
    test without invoking them manually one by one until we have a better
    solution.
    
    A few tests are currently broken; for now, these are added to
    tests_to_skip - I will put follow-up issues + PRs to fix these as soon
    as possible. I recommend we add running tests as a requirement before
    merging PRs until the CI/CD project is complete, too. One way to help
    facilitate this is to have PR authors copy/paste the results at the end
    of the test run to the PR description.
    
    Fixes #309 
    
    ## Type of change
    
    - New feature (non-breaking change which adds functionality)
    
    ## Screenshots
    
    An example test result summary with timeout set to 60 seconds running on
    my workstation:
    
    ```
    ===================
    Test Result Summary
    ===================
    Total: 39
    Passing: 29
    Failing: 0
    Skipped: 8
    Timing Out: 2
    Passing Percentage: 94.87%
    Total Time Elapsed: 0.0h31.0m52.38s
    ```
    
    ## Checklist
    
    - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
    `./orbit.sh --format`
    - [ ] I have made corresponding changes to the documentation
    - [x] My changes generate no new warnings
    - [x] I have added tests that prove my fix is effective or that my
    feature works
    - [ ] I have updated the changelog and the corresponding version in the
    extension's `config/extension.toml` file
    - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
    exists there
    51ccd99f
Name
Last commit
Last update
.github Loading commit data...
.vscode Loading commit data...
docker Loading commit data...
docs Loading commit data...
source Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.flake8 Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.pre-commit-config.yaml Loading commit data...
CONTRIBUTING.md Loading commit data...
CONTRIBUTORS.md Loading commit data...
LICENSE Loading commit data...
README.md Loading commit data...
VERSION Loading commit data...
orbit.sh Loading commit data...
pyproject.toml Loading commit data...