WCA Live Integration Pre Work #1: Job and Websockets

[FinnIckler]

I will do the database changes in a follow up PR so currently I am just directly broadcasting from the job (instead of in an after_save hook).

I have a test page, should I commit that too?

[maxidragon]

Just curious, what do you mean by "WCA Live integration"? Are there any documents describing this available?

[gregorbg]

We're slowly, gradually absorbing WCA Live into the main website. This is a month(s)-long project, and as such we don't have detailed documentation available. But the gist of it is: Get rid of the separate URL and "Sync Now"

[jfly]

I'd also be interested or hear more about this. One of the primary motivators for WCIF was to make it possible for people to build a pieces of competition infrastructure without having to build everything, and without having to yeet code at the WST to host (leaving the WST responsible for more than it can/wants to handle).

Concretely, (and meaning no disrespect to WCA Live here): after this integration is complete, would it be possible for a community member to build and use an alternative to WCA Live?

[gregorbg]

Before we dive any deeper, I'd like to clarify that we're not getting rid of WCIF at all. The existing endpoints for custom pieces of competition infrastructure will continue to exist and be maintained.

For WCA Live specifically, it was started as a custom piece of software, but quickly became so popular that (outside of a few. select regions) it is the de-facto standard for running WCA competitions. And it has already been "yeeted at the WST", so we are already responsible for maintaining it.

Right now, that responsibility rests solely on Jonatan's shoulders. We want to change this and reduce the truck factor, so we have to come up with an integration that is maintainable for a broader audience. There were lengthy discussions last year involving the Board, paid contributors, and Jonatan, and we figured out together that this "absorption" into the main website would be the best way forward. Here are the core arguments at a glance:

• Elixir is super-cool for high-load, distributed live applications, but it's even less widely spread than Ruby. As a team that relies on broad volunteer contributions, we cannot just tell people to "learn Elixir or forget it". Viable alternatives such as https://github.com/anycable/anycable-rails exist.
• For large competitions, the WCIF payload can exceed several MB in filesize. For pretty much all larger championships (Continentals, Worlds) during the past two years we had to manually increase server timeouts, to allow them to "Sync now" in WCA Live. The fact that WCA Live relies on the main website as a "glorified backup" storage for in-progress results makes this very inconvenient for users
• WCA Live duplicates a lot of data that already lives in the monolith. Custom WCIF integrations are cool if you want to use the data to generate certificates, print nametags, etc. But if you look at the table schema for WCA Live, you will realize that it's almost a 1:1 copy of our core results infrastructure, and this creates a lot of pain points. For example, the average user is interested in seeing whether a certain live result is a PB, so WCA Live has to store everybody's PBs in what is essentially a shadow-copy of our main Results table. When the Delegate ports the competition into WCA Live 2 weeks ahead of time, the PBs for frequent competition goers might be out of date and a solve that was faster than your two-week old PB doesn't consider the fact that you already beat this PB at another competition last weekend. Of course this can be fixed with another sync, but if you're at Worlds and three kiddos are complaining to have their PBs fixed you generally have better things to do than shove around ~300.000 lines of JSON from a WCA server to another WCA server just so that a gray badge icon in ~0.5% of your competitor's live results disappears.
• Lastly, not quite as important but still worth considering: WCA Live uses Material UI which is not consistent with the design language and brand of our main website. We consider it "official" in terms of using it for result entry at comps, but it does not look official in terms of design language.

Furthermore, I can also already hint at the fact that this "merge refactor" will have another cool side effect: We will open up the API to post a live result, so that other things like TimeBase can submit their results and have them appear directly, just as manual results entry would.
Essentially, we're building a "live results API" and at the same time we're porting over the existing WCA Live frontend to become a first class citizen of that API. But (after figuring out how to "authenticate" that TimeBase is allowed to post results and some random script kiddie's Python three-liner isn't) other programs will be able to use that exact same API and to the average website user it will look the same.

Lastly, results submission will be greatly simplified by this and it will allow us to easily implement some quality of life changes and data integrity checks that WRT has been yearning for for years.

[maxidragon]

Thanks Gregor for detailed explanation, I agree with you in almost all aspects. One thing I'd argue about is MUI, I think it's super cool for Live and much better than SemUI for admin things, like data entry. But at the same time I understand your point about consistency.

Furthermore, I can also already hint at the fact that this "merge refactor" will have another cool side effect: We will open up the API to post a live result, so that other things like TimeBase can submit their results and have them appear directly, just as manual results entry would.

Just wanted to note that WCA Live API already exists and is doing great ;)

[jfly]

That largely makes sense! 100% agree that if the WST is effectively already maintaining WCA Live, then it'll be easier to maintain if it lives inside of the rails monolith.

I'm not sure you answered my last question, though:

after this integration is complete, would it be possible for a community member to build and use an alternative to WCA Live?

[FinnIckler]

I would say that it would be even easier, as the community could just use the same APIs that we are building for this new WCA Live integration. We have talked about the possibility of working with cubing china to see if they can set up their live results system that way (honestly I don't even know if they are even using WCIF or not just doing everything by themselves and then dumping the JSON into the WCA in the end!)

[gregorbg]

Can you move the if check into a globally accessible WcaLive.enabled? variable like we did with paypal?

[gregorbg]

In line with my other comment: reject unless WcaLive.enabled?

[gregorbg]

Also, please share this broadcast key (perhaps in the same module as the "WCA Live enabled?" feature above)

[gregorbg]

Shared constant yay