Reduce the number of used APIs (only api.geneontology.org)

[kltm]

Recently, we discovered that api.geneontology.xyz is still in used by Alliance pages to return whether or not there are GO-CAM models available for display with the pathway viewer. Since the work from @sierra-moxon , these endpoints are duplicated within the GO API at api.geneontology.org, which is what should be used for all such operations.

• Determine if only the Alliance is using api.genentology.xyz with logs or proxy (may be tricky) @dustine32 - yes it is. here: https://github.com/alliance-genome/agr_ui/blob/main/src/components/pathway/pathwayWidget.js
• Determine if any other users are using api.geneontology.xyz with logs or proxy.
  • use by PomBase
• Open an issue for the API endpoint change in the Alliance to api.geneontology.org (or open a PR?) @sierra-moxon
• If there are other apparent users of the API, see if we can track them down
  • update from @sierra-moxon : FlyBase is using api.geneontology.org correctly.
• Try and figure out how to get (current) api.geneontology.xyz users onto api.geneontology.org
• Shutdown the api.geneontology.xyz

[kltm]

Related: geneontology/go-site#1800

[kltm]

@dustine32 Will be looking at ways of capturing requests to api.geneontology.xyz, so we can contact users, understand use cases, and create a plan to deactivate this endpoint.

[kltm]

Known users of api.geneontology.xyz (edit to add):

• PomBase (dev?)
• Alliance https://github.com/alliance-genome/agr_ui/blob/main/src/components/pathway/pathwayWidget.js

[kltm]

Tagging @kimrutherford and @dlrice as interested parties.

[kimrutherford]

Speaking for PomBase, our ideal is access the production models from Noctua as soon as they are marked as "production".
We use and like api.geneontology.xyz because it gives us that.

We have gocam-viz configured to point to that API and we also access it programmatically a couple of times per week grab the genes and terms from the production models for storing in our database.

If we could get access the models created by PomBase as JSON files that would be fine. We would serve those from pombase.org for gocam-viz. And we'd be able to read any data we need for adding to our database.

Could a daily job pull production models for each group from the Noctua database and host them somewhere?

Something like https://github.com/geneontology/noctua-models but with a sub-directory per group (PomBase/FlyBase, etc.) and files in JSON format could replace api.geneontology.xyz for us. It would also make the Noctua data very accessible for any other downstream user.

@ValWood

[kltm]

@kimrutherford thank you for your feedback (and my apologies again for my unthoughtful words on slack).
If we are going to be looking at some kind of raw feed of the "live" models, understanding the frequency needs is very useful.

[sierra-moxon]

@kltm - I find no evidence of alliance using api.geneontology.xyz in their public codebase: https://github.com/search?q=org%3Aalliance-genome%20api.geneontology.xyz&type=code, but I do see api.geneontology.org calls: https://github.com/search?q=org%3Aalliance-genome+api.geneontology.org&type=code

I don't think Alliance is one of the users of api.geneontology.xyz -- it could be we were tracking "FlyBase" usage here in this ticket (which uses a different implementation of the ribbon (and pathway widget?), at least, than Alliance)

[sierra-moxon]

I took a look through FlyBase repos: https://github.com/search?q=org%3AFlyBase+api.geneontology&type=code&p=2. They too use api.geneontology.org and there are no instances of api.geneontology.xyz

[dustine32]

@sierra-moxon Are you including the agr_ui codebase in your Alliance assessment or just looking specifically at their API code? There is still this usage of .xyz here:
https://github.com/alliance-genome/agr_ui/blob/23eed427cea404ef97feaf17ecbbaaabe2a0a275/src/components/pathway/pathwayWidget.js#L165
I just confirmed by looking at the browser network tab that this call is made on the Alliance gene pages:

This brings up an issue with determining the website source of calls to .xyz. This agr_ui code is run client-side, meaning that the call is directly from the website user's computer to the .xyz endpoint and thus will not have the Alliance site's IP in the .xyz API call logs (that I set up on AWS CloudWatch). This results in (after filtering out all the calls from search crawlers) a bunch of different IP's for the end user's ISP or institution (e.g., UPenn, UMass Medical School) and not the nice, small list of websites we can contact to ask them to kindly redirect to a different endpoint.

[sierra-moxon]

thanks @dustine32!
fwiw, that should be a very easy swap to api.genontology.org