Initial import of exported SVN content (r83)

refs #1

TODO.txt

@@ -0,0 +1,161 @@
+
+The goal is to move the event entries into a database and refactor the code back to a single script. Any user-customizable behavior will be placed into a configuration file, at least for the time being. At some point we could look at moving those settings into the database itself.
+
+#####################################################################################################
+  Scratch notes
+#####################################################################################################
+
+
+Future milestones
+==============================================
+
+Tables:
+
+* UNKNOWN (not sure what this would be called)
+** wiki page name
+** message subject
+** message body
+*** perhaps not if we build the templae some other way
+** message header
+** message footer
+**
+
+* contacts
+** would be needed to allow for multiple email notifcations per event
+** would also be needed in cases where recipients are CC'd instead of BCC'd
+
+* events
+** bulk of the content would go here
+** columns
+*** priority
+**** 1:1 correlation to Redmine ticket priority values
+*** description
+**** meant for humans looking at db entry
+*** title
+*** date_occurs
+*** time_occurs
+**** UTC time and then convert to localtime (so that Python can handle the
+     daylight savings conversion)
+*** contact_id?
+**** how to link the tables? Should I?
+***** one-to-many relationship here?
+**** foreign key to contacts table
+*** template_id
+
+
+* templates
+** vs flat-files, though in our case we use a Redmine include statement to pull in a wiki page
+
+* notifications
+** ties together contacts, events, ... ?
+
+* meta
+** meta.db_last_modified
+*** logging when any other table was last updated
+** meta.db_revision
+*** auto-increment for every table update
+** meta.db_schema_version
+*** would be used in order to help migrate from one schema version to another (add columns, rename older one, etc.)
+** ?
+
+Configuration file:
+
+* Queries
+** currently read-only queries should be sufficient as I would use another
+   process to get data into the database
+
+* Flags
+** to disable/enable specific classes of event notifications
+*** e.g., disable student worker tickets for times when they're not there
+*** this should probably be set directly in the database (a column in the events table)
+** to control feedback (display debug, warning, info, errors, ...)
+
+* Database credentials
+
+
+
+
+#####################################################################################################
+  TODO
+#####################################################################################################
+
+    * Create 'meta' table for logging when any other table was last updated
+      (presumably we won't be wholesale nuking the existing tables on
+      each run by this point)
+
+      - meta.db_last_modified
+      - meta.db_revision (auto-increment for every update to database)
+
+    * Create TRIGGER to handle updating meta.db_last_modified column when
+      any other table is modified
+
+      - possibly have this same TRIGGER handle updating the meta.db_revision
+        column too
+
+
+#####################################################################################################
+  Google Keep scratch notes
+#####################################################################################################
+
+Purpose:
+
+Send reminders for events. The goal is to support upcoming, current and recently passed dates and times.
+
+The primary purpose is to replace an older hard-coded script that is used to generate automated tickets, but it could be used as a reporting tool as well.
+
+For example, you could have an event reminder type of EndOfLife (eol, end-of-life, end_of_life, etc.) and query based on that type. You would then see that MariaDB 10.0 expires in 2019-03 and that 10.1 expires in 2020-10. This is assuming that we had one or more MariaDB installations we wanted to be aware of expiring soon.
+
+Tags would be more flexible though, so that would likely be implemented over fixed categories.
+
+Early builds would just have a DB manually updated and a script (likely Python) that performs the notification work. We would implement email notifications first, eventually adding support for XMPP notifications.
+
+Re DB schema, I am unsure about the tables and columns per table, but I see the need to support (or be able to calculate) these values:
+
+* event notification schedule: before or after event occurs?
+* event date
+* event time (default to a sane value if not specified?
+* reminder sent (would need some way to allow for multiple entries per event: email, xmpp, text, allowing for a mix of them
+* event description
+* event documentation reference
+* notification template support, but likely outside the database. Perhaps in the database to allow for modification via future web UI
+** should be able to create templates separately and associate same template with multiple events (hence a true template)
+* event type (multiple selection possible): tags support
+* contact name
+* contact email
+* contact phone
+* contact xmpp
+* notification type: email
+* notification type: xmpp
+* notification type: text
+* notification type: web API submission (e.g., Redmine WS API for submitting tickets)
+* notification type: redmine_email
+* redmine category
+* redmine project
+* redmine status
+* redmine due date
+
+The Redmine project, status, category and other details could be contained in a custom footer (for redmine email functionality)
+
+Early version would run as an hourly cron job, so any functionality to send reminder "0 minutes before start" would not act as expected. That functionality would need to be implemented as either an early notification or a post-event notification.
+
+
+Some of the things that the scripts do now:
+
+Daily, skip weekends
+Monday, Friday
+Friday
+2x a month, specific dates
+Once month
+
+I could set the frequency type in events table and use script call to request specific types of events. For example, the existing daily entry would call script with argument for daily tasks. Then the weekly task would do the same and so on.
+
+A standard template could be crafted since we are using an include reference in the body of the email.
+
+The data for the table could be pretty much what the tasks table has now. In a lot of ways all I would be doing is combining the separate scripts into one and getting input data out of script where it does not belong.
+
+
+Ignoring earlier comments, but probably the 0.2 revision (the 0.1 being nearly a 1:1 conversion of the scripts to one script and a db backend, but with pulling directly from db page vs using Redmine include macro) will include support for multiple secondary wiki pages. The first one pulls in the main content, additional pages would be pulled in as content to go in the place of placeholders that would be inserted into the primary Automated Ticket include pages.
+
+For example, where the main page might offer a minimum of summary content and directions and instead rely on the include macro, the 0.2 revision (including the wiki page updates to support it) would pull content from the additional (likely referenced as secondary later on) pages and insert for each placeholder. The primary Automated Ticket pages would no longer (or as seldom as necessary) use the include macro to pull in other content.
+
+My initial impression is that this would require at least one more table, this one to hold wiki page references. The entries would tie back to a specific event and would note whether the table entry was for a primary page or a secondary inclusion page.

automated_tickets.ini

@@ -0,0 +1,129 @@
+# Purpose:
+#
+#  Expose general configuration options so values do not have to be
+#  hardcoded in the main script.
+
+
+
+############################################################
+# "toggle" options to broadly enable/disable behavior. Primarily used to
+# control the types of output messages used to assist when first configuring
+# or testing the script. For example, you might wish to see the output to
+# know all is well before enabling the script as part of a cron job and
+# then turning off all output
+[flags]
+############################################################
+
+# Disables sending notifications and any other possibly destructive action
+testing_mode = true
+
+# Set to true for verbose output as the script executes. Useful for troubleshooting.
+display_debug_messages = true
+
+# Less verbose output. Enabling this provides status details.
+display_info_messages = true
+
+# Output warning messages which indicate that a desired step was not
+# completed as expected, but the issue does not appear to be severe
+# enough to stop the script from completing in an "acceptable" manner.
+# See the 'fail_on_warnings' option in case you would rather not
+# continue operation when warning conditions are encountered
+# TODO: Add support for this
+display_warning_messages = true
+
+# Output error messages which block proper operation of the script
+# TODO: Add support for this
+display_error_messages = true
+
+# If enabled, this treats all warnings as failures and exits immediately with
+# one or more messages indicating that this option is enabled and what warning
+# condition occured
+# TODO: Add support for this
+fail_on_warnings = false
+
+# If enabled, events/tasks that are flagged as an intern_task are processed
+# like all other matching events. If false, then those events are filtered out
+process_intern_events = true
+
+# If enabled, any '{{include(PAGE_NAME_HERE)}}' macro calls found when pulling
+# the wiki page contents from the first page will be expanded (macro calls
+# replaced by the include pages) so that the final content is self-contained
+# with no further direct dependencies on external pages. "Related pages"
+# sections are prefectly acceptable, but not commonplace among the include
+# pages intended for ticket generation.
+expand_include_macros_in_wiki_pages = true
+
+
+##############################################################################
+# Used by the MySQL database connector module
+[mysqldb_config]
+##############################################################################
+
+# Read-only access to Redmine wiki and event_reminders database tables is sufficient
+user = events_ro
+
+# Password for the MySQL database account. Reminder: read-only access
+# is sufficient
+password = ChangeM3
+
+# MySQL database host. If you use a stunnel or other forwarded connection you
+# will likely wish to set this to 127.0.0.1 and override the default port
+# for the 'port' setting
+host = 127.0.0.1
+
+# If this script runs on the same box as the MySQL/MariaDB server then you
+# probably want to enter 3306 here, otherwise if using stunnel or another
+# port forwarding setup you will likely need to enter an alternate port.
+port = 3306
+
+# The database which holds event reminder entries.
+events_database = event_reminders
+
+# The Redmine database holding the wiki pages which this script will pull
+# from in order to generate "automated" tickets with pre-filled content
+redmine_database = help_redmine_db
+
+# Specific to the upstream MySQL Connector module > 2.0.5. This forces all
+# warning conditions to trigger exceptions, which if uncaught will terminate
+# the script. In normal operation you shouldn't receive any exceptions, so
+# this is enabled by default to help catch configuration or script errors
+raise_on_warnings = true
+
+
+[email]
+
+# These values are used if an event_reminders entry doesn't have a value recorded
+# Note: These are helper values that are not yet directly referenced by Python
+# code, but instead referenced by the enabled_event_table_entries query in
+# this file.
+default_from_address = [email protected]
+default_to_address = automated-tickets@localhost
+
+
+##############################################################################
+# Used by the MySQL database connector module
+[queries]
+
+# Pull wiki page contents from Redmine database
+#   The wiki_pages.title value is the name of the page as shown in the URL
+#   The project.identifier value is the project "shortname", shown in the URL
+wiki_page_contents = SELECT wiki_contents.text FROM wiki_contents INNER JOIN wiki_pages ON wiki_pages.id = wiki_contents.id INNER JOIN wikis ON wikis.id = wiki_pages.wiki_id INNER JOIN projects ON projects.id = wikis.project_id WHERE wiki_pages.title = '{}' AND projects.identifier = '{}'
+
+# The query needed to pull event table entries. As is, this query does not limit
+# the returned results by event schedule or whether the flag is set for
+# processing "intern" tasks. That is handled programatically by the script
+event_table_entries = SELECT IF(email_to_address IS NOT NULL, email_to_address, '${email:default_to_address}') AS email_to_address, IF(email_from_address IS NOT NULL, email_from_address, '${email:default_from_address}') AS email_from_address, email_subject_prefix, redmine_wiki_page_name, redmine_wiki_page_project_shortname, redmine_new_issue_project, redmine_new_issue_category, redmine_new_issue_status, IF(redmine_new_issue_due_after_days IS NOT NULL, CURRENT_DATE() + INTERVAL redmine_new_issue_due_after_days DAY, NULL) AS redmine_new_issue_due_after_days, redmine_new_issue_priority, event_schedule FROM events WHERE enabled = 1
+
+##############################################################################
+
+
+# FIXME: Come up with a better name. Perhaps have a separate section per
+# notification type? If so, then 'email', 'xmpp', ... and this section
+# can go away.
+[notification_servers]
+
+email_server_ip_or_fqdn = localhost
+
+#email_server_username
+
+#email_server_password