Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI for volunteers page #42

Merged
merged 1 commit into from
Jan 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[run]
branch = True
source = volunteers/

[report]
exclude_lines =
if self.debug:
pragma: no cover
raise NotImplementedError
if __name__ == .__main__.:
ignore_errors = True
omit =
volunteers/tests.py

25 changes: 25 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
on: [push]

jobs:
test_job:
runs-on: ubuntu-latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not make a big difference, but we tend to use debian rather than ubuntu.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get "Can't find any online and idle self-hosted runner in the current repository, account/organization that matches the required labels: 'debian-latest'" when I try that.

It seems like GitHub has decided to stick with Ubuntu for all of their Linux CI hosts: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners

So, unfortunately, we will be limited to the Docker container for Debian.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SourceHut offers Debian VM images and I'd be glad to integrate it with this repo if it is desired.

Copy link
Member

@johanvdw johanvdw Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The os doesn't really matter as we are using a virtualenv anyway. But thanks for the suggestion anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed @johanvdw , but @McSinyx if you have a plan how to do that integration, I would be fine with it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing it would require either trusting me for owner privilege or creating a paying account for FOSDEM ($20/year; builds.sr.ht got hit by cryptocurrency miners and decided to require payment earlier than initially planned). Personally I'm OK with either but it's up for the maintainers to decide if it's worth it.

env:
OS: ubuntu-latest
PYTHON: '3.7'
name: Linting and Testing
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Test action step
uses: ./
id: test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
files: ./coverage.xml
flags: unittests
env_vars: OS,PYTHON
name: codecov-umbrella
fail_ci_if_error: true
path_to_write_report: ./coverage/codecov_report.txt

6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,9 @@ secret.txt

logs
static/*

public/
penta.db
volunteers.db
.coverage
coverage.xml
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
fosdem-volunteers
=================

![build](https://github.com/Logout22/volunteers/actions/workflows/main.yml/badge.svg)
![codecov](https://codecov.io/gh/Logout22/volunteers/graph/badge.svg)

Volunteers management system for conferences, originally written for FOSDEM.

Development setup
Expand Down
6 changes: 6 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
name: 'Test'
description: 'Linting and Testing'
runs:
using: 'docker'
image: 'ci/Dockerfile'

8 changes: 8 additions & 0 deletions ci/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM python:3.7-bullseye

RUN apt-get update -y && \
DEBIAN_FRONTEND="noninteractive" apt-get install -y sqlite3

COPY ./entrypoint.sh ./

ENTRYPOINT ["/entrypoint.sh"]
11 changes: 11 additions & 0 deletions ci/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/sh
set -o errexit
set -o nounset

utils/create-venv.sh
utils/install-dependencies.sh
utils/setup-file-system.sh
utils/setup-server.sh
utils/run-linter.sh
utils/run-tests.sh

34 changes: 34 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
arabic-reshaper==2.1.1
astroid==2.5.6
coverage==5.5
dj-static==0.0.6
Django==2.2.18
django-guardian==2.3.0
django-userena-ce==6.0.1
easy-thumbnails==2.5
future==0.18.2
gunicorn==20.0.4
html2text==2014.12.29
html5lib==1.1
isort==5.8.0
lazy-object-proxy==1.6.0
mccabe==0.6.1
Pillow==8.1.0
psycopg2-binary==2.8.6
pylint==2.8.2
pylint-django==2.4.4
pylint-plugin-utils==0.6
PyPDF2==1.26.0
python-bidi==0.4.2
python-dateutil==2.8.1
pytz==2021.1
reportlab==3.5.59
six==1.15.0
sqlparse==0.4.1
static3==0.7.0
toml==0.10.2
typed-ast==1.4.3
vobject==0.9.6.1
webencodings==0.5.1
wrapt==1.12.1
xhtml2pdf==0.2.5
6 changes: 6 additions & 0 deletions utils/create-venv.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh
set -o errexit
set -o nounset

python3 -m venv venv

7 changes: 7 additions & 0 deletions utils/install-dependencies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/sh
set -o errexit
set -o nounset

. venv/bin/activate
pip install -r requirements-dev.txt

79 changes: 79 additions & 0 deletions utils/run-dockerized.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail

show_help() {
cat <<EOM
Invocation:

$0 [-h|-?] [-s] [-u USER] [-e EXECUTABLE] [--] [ARGS...]

Options:

-h|-? : Display this help text
-s : Elevate this script to run Docker as root
(default: false)
-u USER : User parameter to pass on to docker run
(default: current UID / GID)
-e EXECUTABLE : Entrypoint to pass on to docker run
(default: none)
ARGS : Arguments for the dockerized process
EOM
}

##### Option parsing #####
AS_ROOT=false
USER=$(id -u):$(id -g)
ENTRYPOINT=

while getopts "h?su:e:" opt; do
case "$opt" in
h|\?)
show_help
exit 0
;;
s) AS_ROOT=true
;;
u) USER=$OPTARG
;;
e) ENTRYPOINT=$OPTARG
;;
esac
done

shift $((OPTIND-1))

if [[ "${1:-}" = "--" ]]; then
shift
fi

##### Elevate script #####
if [[ $AS_ROOT = true && $UID -ne 0 ]]; then
sudo "$0" -u "$USER" -e "$ENTRYPOINT" "$@"
exit
fi

##### Build container #####
CONTAINER_NAME=volunteers-test
docker build --pull -t "$CONTAINER_NAME" ci

##### Run container #####
ADDITIONAL_ARGS=
EXEC_ARGS=
if [[ -n "$ENTRYPOINT" ]]; then
ADDITIONAL_ARGS="--entrypoint $ENTRYPOINT"
fi
if [[ $# -gt 0 ]]; then
EXEC_ARGS="$*"
fi
docker run \
-v "$PWD:$PWD" \
--rm \
--name "$CONTAINER_NAME" \
--workdir "$PWD" \
--user "$USER" \
$ADDITIONAL_ARGS \
"$CONTAINER_NAME" \
$EXEC_ARGS

8 changes: 8 additions & 0 deletions utils/run-linter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
set -o errexit
set -o nounset

. venv/bin/activate
echo " ##### Run linter #####"
DJANGO_SETTINGS_MODULE=volunteer_mgmt.settings pylint --load-plugins pylint_django --fail-under=7 volunteers/

9 changes: 9 additions & 0 deletions utils/run-tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh
set -o errexit
set -o nounset

. venv/bin/activate
echo " ##### Run test #####"
coverage run manage.py test
coverage xml

8 changes: 8 additions & 0 deletions utils/setup-file-system.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
set -o errexit
set -o nounset

mkdir -p logs tool/media/mugshots
sqlite3 volunteers.db
sqlite3 penta.db

16 changes: 16 additions & 0 deletions utils/setup-server.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/sh
set -o errexit
set -o nounset

sed "\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using sed, I would provide a specific settings_sqlite.py and a settings_postgres.py . I'm okay doing this now, but in any case, I think we can improve how this is done:
#44

s/backends.postgresql_psycopg2/backends.sqlite3/; \
s/'NAME': '\([a-z]\+\)'/'NAME': '\1.db'/; \
" \
volunteer_mgmt/localsettings_example.py \
> volunteer_mgmt/localsettings.py

. venv/bin/activate
./manage.py migrate
./manage.py collectstatic
./manage.py createsuperuser --noinput --username test --email [email protected]

1 change: 0 additions & 1 deletion volunteer_mgmt/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@
USERENA_MUGSHOT_DEFAULT="/static/img/octicon-person.svg"
USERENA_DISABLE_PROFILE_LIST = False
AUTH_PROFILE_MODULE = 'volunteers.Volunteer'
TEST_RUNNER = 'django.test.simple.DjangoTestSuiteRunner'
USERENA_SIGNIN_REDIRECT_URL = '/volunteers/%(username)s/'
USERENA_REGISTER_PROFILE = False

Expand Down
20 changes: 10 additions & 10 deletions volunteers/tests.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
"""
This file demonstrates writing tests using the unittest module. These will pass
when you run "manage.py test".

Replace this with more appropriate tests for your application.
Tests to be run via "manage.py test"
"""

from unittest.mock import patch
from django.test import TestCase
from volunteers.views import promo


class SimpleTest(TestCase):
def test_basic_addition(self):
"""
Tests that 1 + 1 always equals 2.
"""
self.assertEqual(1 + 1, 2)
class PromoTestCase(TestCase):
@patch('volunteers.views.render')
def test_promo_view_renders_static_page(self, render_mock):
static_promo_page_content = 'Some promo page'
render_mock.return_value = static_promo_page_content
ret = promo(None)
self.assertEqual(ret, static_promo_page_content)