PlayHQ Platform Update

The AFL have provided the following response to the issues encountered by users of the PlayHQ platform last weekend (Saturday 13th April).

What went wrong?

PlayHQ encountered a site-wide system outage on Saturday 13th April between 6pm (AEST) and 11:30 pm (AEST).  Around one third of users visiting playhq.com and the PlayAFL App saw a 404 error message instead of the fixtures and match centres they sought. Those visiting the admin portal and registration flows would have found it slow. Some users would have been affected when downloading games to score on the scoring portal but scoring functioned without issue otherwise.

What caused it?

After a change to mobile apps deployed in January 2024, PlayHQ’s mobile app requests to the PlayHQ Neo4j database layer were not being cached correctly. Our mobile app requests were hitting our Neo4j database and not the cache as expected. This issue was exacerbated by the significant growth in usage of PlayHQ’s mobile apps.  Neo4j hit its database limit, and all the services relying on this database failed.

What is being done about it?

PlayHQ is focused on ensuring it doesn’t happen moving forward with the following actions implemented:

1. PlayHQ has fixed the caching issue with the mobile apps.
2. As an insurance policy, PlayHQ has increased CPU for our Neo4j database.
3. PlayHQ is improving automated forecasting, monitoring & alerts.
4. Increasing caching and specific request optimisations across different experiences to improve site speed.
5. Fixing the deployment and QA processes to ensure caching is implemented properly.

Pleasingly, there were no such ‘outage’ issues with the PlayHQ platform on the weekend. Along with other Metro Football Leagues, we will continue to monitor the platform closely and have AFL address all concerns.

Access results, ladders and player stats on PlayHQ via the VAFA website here.