/mesh/sparql API endpoint failing with 502 Server Error: Proxy Error

[dhimmel]

For the last several days, I've noticed that queries to https://id.nlm.nih.gov/mesh/sparql are failing, with status code 502 and a response of:

Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /mesh/sparql.

Reason: Error reading from remote server

Here's the full URL for my query.

[dhimmel]

I wonder if this is because the query is taking too long and timing out. I thought this query used to work, but perhaps not.

[danizen]

Thank you - I will investigate. We moved from MeSH RDF 2020 to 2021, and there are some other changes associated with this.

[danizen]

You are correct that this is a timeout. Our virtuoso server is configured with the following parameter:

[SPARQL]
[SPARQL]                                                                                                                                                                       
ResultSetMaxRows = 10000                                                                                                                  
MaxQueryCostEstimationTime    = 400                                                                   
MaxQueryExecutionTime    = 60                                                                         
DeferInferenceRulesInit    = 0

The error can be reproduced against the QA server, and appears in the server's logs as a broken pipe exception. In an ideal world, the JDBC driver would be able to be more specific about the error, and I will see whether I can get some feedback from OpenLink, Inc.

At the same time, I suppose your question would be how to download the type, identifier, and label information as a CSV.

Since the list of types is fairly static, I would suggest you inquire against each of the types individually. I shall post some partial code as a github gist shortly.

[danizen]

Some time ago, I did some experimental work designed to find a way to import our authorized RDF into Bioportal. While I had some success, I met with feedback that this would remove some of the semantic richness of the model by flattening it. There was also some disagreement about who should do the work.

The general mechanism that worked for me was to craft a query to obtain a COUNT of all resources that should be obtained, and then page through the results generally.

As an example, here is how I could obtain a tree of Topical descriptors.

1. count the total descriptors I expect to see:

    SELECT (COUNT(?resource) as ?count)
    FROM <http://id.nlm.nih.gov/mesh>
    WHERE {
      ?resource a meshv:TopicalDescriptor .
      ?resource meshv:active "true"^^xsd:boolean .
    }

2. Obtain the root descriptors, which are those without a meshv:broaderDescriptor:

    SELECT ?resource ?label
    FROM <http://id.nlm.nih.gov/mesh>
    WHERE {
      ?resource a meshv:TopicalDescriptor .
      ?resource meshv:active "true"^^xsd:boolean .
      ?resource rdfs:label ?llabel .
      BIND(STR(?llabel) AS ?label) .
      MINUS {
        ?resource meshv:broaderDescriptor ?parent .
      }
    }
    ORDER BY ?resource

3. continue a BFS until done (by count) by querying on children of a parent and using offset/limit with this query:

    SELECT ?resource ?label
    FROM <http://id.nlm.nih.gov/mesh>
    WHERE {
      ?resource a meshv:TopicalDescriptor .
      ?resource meshv:active "true"^^xsd:boolean .
      ?resource rdfs:label ?llabel .
      BIND(STR(?llabel) AS ?label) .
      ?resource meshv:broaderDescriptor ?parent .
    }
    ORDER BY ?resource

I suggest that something similar to this could work for you. I also have found it helpful to assemble the SPARQL queries by part from YAML, and to have a unit test that assures that they parse successfully once assembled using Jena/rdf4j.

Although serializing our MeSH RDF as SKOS is not a supported operation, I submit this SPARQL in the hopes it would help you - https://gist.github.com/danizen/8975c279b304ff35a90e5e986a86e7c2

I will continue to follow-up about getting a better HTTP status code for queries that time out.