Matching list and lookup no longer time out under load
The matching list and lookup routes stopped answering for a minute or two at a time on October 10 when several requests arrived together. They now answer from memory in a few milliseconds.
On October 10, between about 21:38 and 22:20 UTC, GET /v1/matching-markets/sports and GET /v1/matching-markets/sports/lookup answered 503 or an empty 502 for a minute or two at a time, every few minutes, whenever several requests arrived together. Every request was reading and decoding tens of megabytes of matching data from scratch, and a handful at once took longer than the nine seconds the API allows a request. The other routes were not affected.
That is fixed. The API now keeps the matching data it has already decoded and reads each new snapshot once, instead of once per request. In our measurements a default page of events costs about 3 milliseconds of work instead of about 170, a lookup about 3 instead of 17, and ten requests arriving together about 30 instead of 1,660. Under load the API now answers in milliseconds instead of timing out.
One thing is slightly different. The archive of settled games is now read in the background about every 30 seconds, no longer on every request. A game that has just finished can reach the default list, and a cancelled game can leave it, up to about 30 seconds later than before. No field, parameter or status code changed, so nothing in the SDKs needs updating.