Skip to content

Commit

Permalink
Add an option to reset failed cronjobs from the UI (thewca#10509)
Browse files Browse the repository at this point in the history
* Provide a way to reset CAD in UI

* Add debug information in 'running but errored' cronjob state
  • Loading branch information
gregorbg authored Jan 2, 2025
1 parent fc1ff1e commit 5b00470
Show file tree
Hide file tree
Showing 5 changed files with 34 additions and 4 deletions.
5 changes: 5 additions & 0 deletions app/controllers/admin_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,11 @@ def do_compute_auxiliary_data
redirect_to admin_compute_auxiliary_data_path
end

def reset_compute_auxiliary_data
ComputeAuxiliaryData.reset_error_state!
redirect_to admin_compute_auxiliary_data_path
end

def generate_exports
end

Expand Down
13 changes: 11 additions & 2 deletions app/jobs/wca_cronjob.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ class WcaCronjob < ApplicationJob
before_enqueue do |job|
statistics = job.class.cronjob_statistics

if statistics.scheduled? || statistics.in_progress? || statistics.recently_errored > 0
if statistics.scheduled? || statistics.in_progress? || statistics.recently_errored?
statistics.increment! :recently_rejected

# Make ActiveJob abort and do NOT enqueue the job
Expand Down Expand Up @@ -90,7 +90,7 @@ class WcaCronjob < ApplicationJob
end

class << self
delegate :in_progress?, :scheduled?, :enqueued_at, :finished?, :last_run_successful?, :last_error_message, to: :cronjob_statistics
delegate :in_progress?, :scheduled?, :enqueued_at, :finished?, :last_run_successful?, :last_error_message, :recently_errored?, to: :cronjob_statistics

def cronjob_statistics
CronjobStatistic.find_or_create_by!(name: self.name)
Expand All @@ -107,5 +107,14 @@ def end_date
def successful_start_date
self.cronjob_statistics.successful_run_start
end

def reset_error_state!
self.cronjob_statistics.update!(
run_start: nil,
run_end: nil,
recently_errored: 0,
last_error_message: nil,
)
end
end
end
17 changes: 16 additions & 1 deletion app/views/admin/_cronjob_status_panel.html.erb
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<% cronjob ||= nil %>
<% trigger_link ||= nil %>
<% reset_link ||= nil %>

<h2><%= "Controls for #{cronjob.name}" %></h2>

Expand All @@ -11,7 +12,21 @@
<% if cronjob.scheduled? %>
<%= alert :info, "The job has been scheduled #{time_ago_in_words cronjob.enqueued_at} ago, but it hasn't been picked up by the job handler yet. Please come back later." %>
<% elsif cronjob.in_progress? %>
<%= alert :info, "The job is running. Thanks for checking =)" %>
<% if cronjob.recently_errored? %>
<%= alert :danger do %>
The job started to run but crashed :O The error message was: "<%= cronjob.last_error_message %>"
<% end %>

<div>
If the source of the error has been fixed, you can <%= link_to "reset", reset_link %> the status for this cronjob.
<em>Use at your own risk! If you push this button without consulting WST, you risk incurring their wrath!</em>
</div>

<%# Debug information for WST %>
<%= tag.pre(JSON.pretty_generate cronjob.cronjob_statistics.attributes) %>
<% else %>
<%= alert :info, "The job is running. Thanks for checking =)" %>
<% end %>
<% elsif cronjob.finished? %>
<% if cronjob.last_run_successful? %>
<%= alert :success do %>
Expand Down
2 changes: 1 addition & 1 deletion app/views/admin/compute_auxiliary_data.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<div class="container">
<%= render layout: "nav" do %>
<%= render 'cronjob_status_panel', cronjob: ComputeAuxiliaryData, trigger_link: admin_do_compute_auxiliary_data_path do %>
<%= render 'cronjob_status_panel', cronjob: ComputeAuxiliaryData, trigger_link: admin_do_compute_auxiliary_data_path, reset_link: admin_reset_compute_auxiliary_data_path do %>
<div>
<h4>This script:</h4>
<ul class="list-group">
Expand Down
1 change: 1 addition & 0 deletions config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,7 @@
get '/admin/person_data' => 'admin#person_data'
get '/admin/compute_auxiliary_data' => 'admin#compute_auxiliary_data'
get '/admin/do_compute_auxiliary_data' => 'admin#do_compute_auxiliary_data'
get '/admin/reset_compute_auxiliary_data' => 'admin#reset_compute_auxiliary_data'
get '/admin/generate_exports' => 'admin#generate_exports'
get '/admin/generate_db_token' => 'admin#generate_db_token'
get '/admin/do_generate_dev_export' => 'admin#do_generate_dev_export'
Expand Down

0 comments on commit 5b00470

Please sign in to comment.