Open-Source, Convenient Judging in Competitive Programming

Though I'm learning to enjoy working on larger projects, I still much prefer gaining experience through smaller programming challenges. They often incorporate problems that are typically difficult to solve, either through difficulty of comprehension or algorithmic complexity. Such challenges force the participants to think critically about how they implement their solutions to the problem. It is possible (and even sometimes a valid solution) to brute force the problem; other times, optimizations must be made to produce a more efficient algorithm. I highly value these exercises; I have learned many cool concepts that I wouldn't have known otherwise.

Just as a reference, here are some of the places that I frequent:

The Problem

These are some of the most popular places to find exercises for programming and mathematics. Though they are very high-quality in terms of their problems, there are some issues with the quality of the services. For example, with the automated judges that need to compile and run the code, there is limited diversity with the languages they provide. Depending on the judging framework, the problem author is responsible for writing unit tests specific to each language, which limits the diversity even further.

In addition to the problems outlined above, there is a common issue with judging frameworks: there are very few open source judges. This adds to the problem of language support; since the pool of contributors is much more limited on closed- source projects.

The Solution

What I'd like to do is start an open-source project for online judging (written primarily in Rust of course). Here are the goals for this project:

  • Simple interface for publication. This involves the publisher being able to support as many languages as they want with minimal intervention/manual work. I am considering the format that Open Kattis uses where all test cases are evaluated through standard input/output.
  • Separation of concerns with different programs for the web frontend and judging backend. This would mean that the infrastructure can be scaled as needed to meet the needs of the deployer. In addition, the workers will not need to be tied to the same machine as the web services.
  • Common, reusable interface for compiling and running code in any language. To add support for another language, simply write a worker configuration file with the environment and scripts needed to compile and run tests. This would mean that third-party support for any language would be possible if it can compile and run on the command line.

But wait, what about OpenBlocks?

Yes, I already have a software project right now! However, that project is quite a difficult one and I often spend time unable to do anything simply because I lack the experience necessary. OpenBlocks is not going to go away; I still have every intention to finish it. However, right now I feel much more confident in my skills outside of game development. I'm considering putting the OpenBlocks project on hold, maybe working on bits and pieces for it when I need a break from this new project.