[Automated] Merged develop into target main

CHANGELOG.md

@@ -0,0 +1,38 @@
+# Changelog
+
+## 1.0.16 - 2024-07-19
+
+_Download of collected data donations is only possible through the API not through the admin interface. 
+Fix and comprehensive documentation will be released in a subsequent version._
+
+### Changed
+
+- Improve robustness of extraction operators: ([`5b3a0af`](https://github.com/uzh/ddm/commit/5b3a0af), [`855796f`](https://github.com/uzh/ddm/commit/855796f)) 
+
+    - Detect and convert date-strings (in ISO, RFC2822, or HTTP format) to date objects for greater/smaller-comparisons.
+    - Detect and convert numeric-strings to numbers for 'greater/smaller'-comparisons.
+    - Exclude strings from 'greater/smaller'-comparisons.
+    - Convert all variable types to string before regex comparison functions.
+- Update appearance of donation instructions. ([`8cb4d79`](https://github.com/uzh/ddm/commit/8cb4d79), [`0b90ea2`](https://github.com/uzh/ddm/commit/0b90ea2))
+- Limit the number of times an error with the same code is posted in an upload attempt from the participation interface to the server to 5. ([`56d30ff`](https://github.com/uzh/ddm/commit/56d30ff))
+- Display extraction rules ordered according to the execution order (instead of their IDs). ([`776b384`](https://github.com/uzh/ddm/commit/776b384))
+- Improve clarity of administration interface by adding collapsibles for additional information. ([`6a851d2`](https://github.com/uzh/ddm/commit/6a851d2), [`0745f6c`](https://github.com/uzh/ddm/commit/0745f6c))
+- Change `DonationProject.date_created` to readonly in admin view. ([`a851042`](https://github.com/uzh/ddm/commit/a851042))
+
+
+### Added
+
+- Add option for "all-in-one consent" to File Uploader. ([`e0b78c7`](https://github.com/uzh/ddm/commit/e0b78c7))
+- Add description of extraction operators to documentation. ([`a4810c1`](https://github.com/uzh/ddm/commit/a4810c1))
+- Add three new types of extraction errors related to regex extraction operators. ([`4b6152e`](https://github.com/uzh/ddm/commit/4b6152e))
+- Add tests for vue extraction functions. ([`4bad2c7`](https://github.com/uzh/ddm/commit/4bad2c7),[`eb0de44`](https://github.com/uzh/ddm/commit/eb0de44))
+
+### Removed
+
+- Remove the `get_simple_bar_plot` template tag and bokeh dependency. ([`57696d6`](https://github.com/uzh/ddm/commit/57696d6))
+
+### Fixed
+
+- Change type of `EventLogEntry.description` from CharField to TextField to allow the posting of longer event descriptions. ([`d0d1c7d`](https://github.com/uzh/ddm/commit/d0d1c7d))
+- Start data entry count from 1 instead of 0 in the data donation feedback navigation. ([`88e9295`](https://github.com/uzh/ddm/commit/88e9295)) 
+- Fix bug that has limited the number of extraction rules to 10 and remove this limitation. ([`d9ef58c`](https://github.com/uzh/ddm/commit/d9ef58c))

ddm/admin.py

@@ -10,6 +10,7 @@ class DonationProjectAdmin(admin.ModelAdmin):
     Provides an overview of all active Donation Projects.
     """
     list_display = ['name', 'owner', 'date_created', 'edit_link']
+    readonly_fields = ['date_created']
 
     def has_add_permission(self, request, obj=None):
         return False

ddm/migrations/0047_fileuploader_combined_consent.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.2.13 on 2024-03-13 15:12
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('ddm', '0046_donationblueprint_expected_fields_regex_matching'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='fileuploader',
+            name='combined_consent',
+            field=models.BooleanField(default=False, help_text='If enabled, participants will be asked to consent to submit all uploaded data at once. Otherwise, participant will be asked to consent to the submission of the data separately for each blueprint.', verbose_name='All-in-one consent'),
+        ),
+    ]

ddm/migrations/0048_alter_eventlogentry_description.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.2.13 on 2024-07-09 11:47
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('ddm', '0047_fileuploader_combined_consent'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='eventlogentry',
+            name='description',
+            field=models.TextField(),
+        ),
+    ]

ddm/models/core.py

@@ -47,6 +47,9 @@ class DonationProject(models.Model):
         max_length=50,
         help_text='Project Name - for internal organisation only (can still be changed later).'
     )
+
+    # Note: Value of "date_created" attribute must not be changed after
+    # instance creation as it is used for en-/decryption.
     date_created = models.DateTimeField(default=timezone.now)
     owner = models.ForeignKey(
         'ResearchProfile',
@@ -326,6 +329,8 @@ def get_donation_info(self):
             'n_success': donations.filter(status='success').count(),
             'n_pending': donations.filter(status='pending').count(),
             'n_failed': donations.filter(status='failed').count(),
+            'n_consent': donations.filter(status='success', consent=True).count(),
+            'n_no_consent': donations.filter(status='success', consent=False).count(),
             'n_no_data_extracted': donations.filter(status='nothing extracted').count()
         }
         return donation_info
@@ -372,6 +377,14 @@ class UploadTypes(models.TextChoices):
         verbose_name='Upload type',
     )
 
+    combined_consent = models.BooleanField(
+        default=False,
+        verbose_name='All-in-one consent',
+        help_text='If enabled, participants will be asked to consent to submit '
+                  'all uploaded data at once. Otherwise, participant will be asked to '
+                  'consent to the submission of the data separately for each blueprint.'
+    )
+
     def __str__(self):
         return self.name
 
@@ -385,6 +398,7 @@ def get_configs(self, participant_data=None):
         configs = {
             'upload_type': self.upload_type,
             'name': self.name,
+            'combined_consent': self.combined_consent,
             'blueprints': [bp.get_config() for bp in blueprints],
             'instructions': [{
                 'index': i.index,

ddm/models/logs.py

@@ -40,5 +40,5 @@ class EventLogEntry(models.Model):
         'DonationProject',
         on_delete=models.CASCADE
     )
-    description = models.CharField(max_length=100)
+    description = models.TextField()
     message = models.TextField()

ddm/static/ddm/css/public-base.css

@@ -1,3 +1,45 @@
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-Regular.ttf');
+    font-style: normal;
+    font-weight: 400;
+}
+
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-Italic.ttf');
+    font-style: italic;
+    font-weight: 400;
+}
+
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-Bold.ttf');
+    font-style: normal;
+    font-weight: 700;
+}
+
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-BoldItalic.ttf');
+    font-style: italic;
+    font-weight: 700;
+}
+
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-Light.ttf');
+    font-style: normal;
+    font-weight: 300;
+}
+
+@font-face {
+    font-family: 'Nunito Sans';
+    src: url('../fonts/NunitoSans/NunitoSans-LightItalic.ttf');
+    font-style: italic;
+    font-weight: 300;
+}
+
 .public-header {
     height: 100%;
     padding-top: 10px;
@@ -86,4 +128,8 @@ a:hover {
 }
 #briefing-consent-answer label {
     cursor: pointer;
-}
+}
+
+.carousel-item p {
+  margin-bottom: 0.5rem !important;
+}

ddm/static/ddm/css/research-interface-base.css

@@ -21,6 +21,10 @@
     align-items: center;
 }
 
+.fs-08 {
+    font-size: 0.8rem !important;
+}
+
 .ddm-nav-icon {
     color: #1a1a1a !important;
     font-size: 1.5rem;
@@ -168,7 +172,6 @@
 .ddm-admin-section-body {
     padding: 20px;
     border-radius: 0 0 5px 5px;
-    background: #fbfbfb;
  }
 .ddm-admin-section-body tbody {
     border-top: none !important;
@@ -391,4 +394,52 @@
 }
 .version-info {
     color: lightgray;
+}
+
+.ddm-info-accordion {
+    border-left: 2px solid black;
+    border-right: 2px solid black;
+    margin: 15px 10PX 25px 10px;
+}
+
+.ddm-info-accordion-header {
+    background: #f2f2f2;
+}
+
+.ddm-info-accordion-button {
+    padding: 5px 15px;
+    background: none;
+    border: none;
+    width: 100%;
+    text-align: left;
+}
+
+.ddm-info-accordion-button:after {
+    content: "\F229";
+    font-family: "bootstrap-icons";
+    display: inline-block;
+    float: right;
+    transition-duration: .35s;
+}
+
+.ddm-info-accordion-button:not(.collapsed):after {
+    content: "\F229";
+    font-family: "bootstrap-icons";
+    display: inline-block;
+    transform: rotate(-180deg);
+    transition-duration: .35s;
+    float: right;
+}
+
+.ddm-info-accordion-body {
+    padding: 15px 21px;
+    border-bottom: 1px solid lightgrey;
+}
+
+.ddm-info-accordion-body ol {
+    line-height: 1.5rem;
+}
+
+.ddm-info-accordion-body li {
+    padding-left: 5px;
 }

ddm/static/ddm/js/blueprint-edit-ui-helpers.js

@@ -83,7 +83,7 @@ closeModal = function(id) {
  * On OK-click in modal, update filter settings overview.
  */
 $( "body" ).on("click", "button[class*='ddm-modal-ok']", function() {
-  const current_id = $(this).attr("id").match(/\d/)[0];
+  const current_id = $(this).attr("id").match(/\d+/)[0];
   hideErrorMessages(current_id);
   if (checkNoFieldsMissing(current_id)) {
     updateRuleDescription(current_id);
@@ -93,7 +93,7 @@ $( "body" ).on("click", "button[class*='ddm-modal-ok']", function() {
 });
 
 $( "body" ).on("click", "button[class*='ddm-modal-cancel']", function() {
-  const current_id = $(this).attr("id").match(/\d/)[0];
+  const current_id = $(this).attr("id").match(/\d+/)[0];
   let e = $("#execution_order-" + current_id);
 
   if ($.trim(e.text()) === 'None') {
@@ -111,8 +111,8 @@ $( "body" ).on("click", "button[class*='ddm-modal-cancel']", function() {
 $(document).ready(function() {
   let IDs = new Set();
   $("[id^=id_processingrule_set-]").each(function() {
-    if( /\d/.test($( this ).attr("id")) ) {
-      IDs.add($( this ).attr("id").match(/\d/)[0]);
+    if( /\d+/.test($( this ).attr("id")) ) {
+      IDs.add($( this ).attr("id").match(/\d+/)[0]);
     }
   });
   for ( const id of IDs ) {
@@ -127,8 +127,8 @@ $(document).ready(function() {
 $("#add-inline-form").on("click", function() {
   let IDs = [];
   $("[id^=id_processingrule_set-]").each(function() {
-    if( /\d/.test($(this).attr("id"))) {
-      IDs.push($(this).attr("id").match(/\d/)[0]);
+    if( /\d+/.test($(this).attr("id"))) {
+      IDs.push($(this).attr("id").match(/\d+/)[0]);
     }
   });
   let formIdx;