Next features for the results system

[StefanPochmann]

We've been working on a few new features for the results system and this message is a quick outlook. Sadly I tried to do too much at once and then also got sidetracked for a while, but now I'm determined to finish things and Clement might continue to help as well. So here are my intentions. The first three are decided already and are a work in varying degrees of process, the latter two aren't decided yet but I hope we'll do them. The plan is to do these one by one in the given order:

1) Competitions manager allowing Ron to more easily add and configure competitions. I think this is almost finished.

2) Competition editor allowing competition organizers to edit their competition, i.e., enter the date/location/events/limits/etc. This is maybe half done.

3) Standard registration allowing competitors to register for competitions right on our competition pages so that competition organizers don't have to set up their own system (although the intention is to offer embedding in their sites if they have some and want it). Started, but didn't get far yet.

4) User accounts with passwords, possibly allowing people to edit their personal information and manage their competition registrations. Would also make people more aware of their IDs here. Not worked on so far.

5) Separate, somewhat less official accompanying website offering several new views/tools/services and possibly access to a copy of the database (reduced to public data, and enriched for easier working). I recently created a mockup showing Ron my vision of this site, but we haven't had time to go through it and decide whether/how we'll do this.

[BryanLogan]

For the registration, is it possible to have other questions? I know some competitions like to ask the "Will you be willing to help judge/scramble?" And I like to get ZIP codes of competitors just so I have a general ID where people are coming from.

[StefanPochmann]

My intention so far is to let organizers request additional information which competitors can then provide in a text area. So the standard form which is the same for all competitions, plus one more input area which organizers can label. If we need more than that, it should be easy to adapt the system from there.

[cada]

Are you taking requests? Something probably easy for you to do that I'd like to see is putting ranking on the regions page. I could look at a country and see the global rank of each national record. Of course, a bunch of 1s for the world region would be annoying.

Chris

[Dave Campbell]

I do have a request, and I guess it falls partly under option five, and partly under option three. Though I am not certain what both of these items encompass based on the information above, so forgive me if I am asking for what you are planning currently.

To start, I want to offer my opinion on option three. My first reaction is that I do not want to surrender control over registration for the competitions I host. I use my registration to create fairly accurate schedules based on algorithms using the number of people in each event, number of timing stations, average solve time per event, number of solves per event etc. I also create printed name tags for all competitors. When someone registers on our site, I save it in a specific format that I can port around easily to the various applications I have made to streamline the process on my end. If you did force organizers to host registration on the WCA site, I would need access to the data in a common format (XML) in an on-demand basis.

Now, onto my initial request. I have touched upon this in another thread, but for consistency I will recap it here for those that did not read it elsewhere. Basically, I want access to the WCA database using the REST architecture approach. It would function similarly to how Youtube, among many other sites, provides assess to their data. Under this scenario, I would send a request to the WCA server, for example:

http://www.worldcubeassociation.org/query.php?Key=12345&search=Person&param=Dave+Campbell

Where the Key is a unique ID given to me by the WCA to let me have access to the data. Search is the resource I am targeting, in this case a person (could be a competition, event, etc), and the param is the data to be used to perform the search.

So far this is very similar to how the WCA site functions, minus the Key. Only currently, I am served up a complex XHTML page as the result which may or may not contain the information for which I am searching. Under this new approach, I would be send back an XML response with a standard header section that tells me if the request succeeded or failed. The body would be either empty if it failed, or contained marked up information of my results.

How much data contained in the response is up to the WCA. At a minimum, I would expect all data that we require for a successful registration (full name, country, WCA ID, DOB, sex). It would be even better if I could get access to all their records, like is returned when you view someone’s WCA profile page. And like I alluded to above, it would be great if we could query other things beyond just a person, but all results from a past competition.

Now, I understand that there is a concern about giving away the information. But it is currently freely available anyway, only in an XHTML form. I have successfully managed to parse it already, and I am not the only one who can do it. If someone wanted to do this under the current set up, they could. However, using the key-based approach I mentioned, you could provide a rich and easy means to gain access to this information in a controlled environment. The WCA is deciding to whom they will give a key. The key essentially be coming a membership ID. In fact, I will go on record to say that I would pay a nominal fee for the key on an annual basis. The WCA could generate some revenue from making this freely available like this.

I would be willing to work with you to architect this approach if the WCA is interested in making this available, though I think it is pretty straight forward. I can provide examples of the XML response if required, or something else that would be useful to implement this if desired. I just feel that this information is really great, and it would be an asset to have access to it. The web seems to be going towards this approach already, so I would love to see the WCA willingly employ it.

[StefanPochmann]

Chris: I recorded your request, might do it when there's time and if I get convinced it won't clutter the display too much. Consistency is also an issue, as I'd like the different region views to behave as equal as makes sense.

Dave: We (at least I, don't know exactly whether Ron and I have talked about this) don't intend to force organizers to use the WCA standard registration, it would be an offer so they don't have to build their own registration system, but they still could if they wanted to. Although right now when Ron gets new competition results, he has to match the competitors against the existing database and the standard registration would help even more than the tool we built so far, and I could imagine if organizers use their own registration system, they might be requested to provide as much and as accurate data as they can. Oh and yes, export of the registration data as at least CSV/Excel and XML are planned, so organizers could request the information. Maybe also email notifications for all changes.

Again Dave: The requests technique you describe is pretty much what I meant with "services" in my fifth item talking about the accompanying website.

[Dave Campbell]

Stefan, thanks for getting back to me. That is great that the approach i described is what you had in mind. My offer to assist in the "how" aspect still stands. If you want to discuss it off line sometime, shoot me an email. And in regards to a standard format to share data, i think it is a fantastic idea. I know we currently use Excel files, but if we should ever wish to move away from it to a more structure format, such as a WCAML or something, i would be available to help determine the format as well.

And while i have your attention, congrats on the job, and thanks for all your work thus far.

[StefanPochmann]

Little update: I didn't get to work on it again yet, though I'm slowly getting there (read: getting familiar with the code and intentions again). But Clement is quite active right now, looks like he'll soon be done with the above 1) and 2).

[Dave Campbell]

Though, i seem to get this a lot when sending requests to the root:

Warning: gzinflate() [function.gzinflate]: data error in /services/webpages/util/s/p/speedcubing2.site.aplus.net/html/includes/bootstrap.inc on line 627

I even get it with the browser at times, both IE and FF. I first noticed it while trying to read the rss feed programmatically. Any idea why?

[StefanPochmann]

Does that error have to do with the results system (anything under /results) and the topic of this thread, or are you talking about other parts of the website?

[Dave Campbell]

It does. So i didn't want to start a new thread about it, especially if your answer was covered in one reply.

If something is preventing a programmatic request from executing successfully, like i get when i query the RSS feed, or the site has a tendency to go unavailable, it would pose a problem when i try to send the request to the new site under item 5 (relating to my first post here). Call it requirements gathering for item 5 if it helps you answer my question.

[StefanPochmann]

But the results system doesn't even have an RSS feed, so I'm kind of lost here.

My assumption is that you're talking about the front page of the WCA site, which is made with drupal, I think. Maybe that has an RSS feed. Don't know and can't check, as currently it's apparently double-zipping again like it does from time to time. Maybe that's what you mean? The results system though is independent from that and is working fine right now.

[StefanPochmann]

To be sure: Please answer my above question about whether you get these problems with anything under /results.

[Dave Campbell]

Stefan

I apologize, i guess i made an assumption. I figured the new site would be built using the same engine and run on the same server as the current results system, which appears to be located on the worldcubeassociation.org site. But perhaps you have the domain names pointing elsewhere. I didn't think there was that level of sophistication involved here. My concern was that if all of the new development would fall on the same server, and be generated using the same PHP engine, that the same issue would arise as i have experienced on the root page and RSS feed. But i think you have indicated that there will be a separate box entirely running the new site, and thus not be subjected to any perils that the current system encounters. And as such, you have answered my question.

Sorry if i was unclear or confusing.

[StefanPochmann]

Indeed I intended the new site mentioned above as 5) to be on a separate server, though that was because of worries that the new functionality could use too much resources and could hurt the main results system. I talked with Ron again about this and it seems he's now more leaning towards not outsourcing stuff but rather doing it here, though we haven't really decided yet what to do and what not.

The front page is working again so now I see the RSS you mentioned. I'd like to emphasize again that that is Drupal, and the results system doesn't have anything to do with it. I don't know why that Drupal part sometimes has problems, but the results system isnt't affected by it as far as I know. Just like in my previous post where I couldn't see the front page but the results system was working perfectly.

So as far as I know, the current server and results system engine work perfectly, only that Drupal part is buggy. They're on the same server/webspace, but they're really independent. Again, if you noticed any outage/malfunction under /results, let me know.