During the finalization of the UI for endpoints, specifically the probeset page, modifications were made to streamline the ProbeSet RDF transform. Unnecessary metadata was removed to reduce the transformed data. Most of the metadata on the probeset page is utilized for constructing resources. An example of a probeset page is:
Under the "Resource Links" section on this page, there's a specific URL:
Attempting to prefix this resource link as "genemania:mus-musculus/Atp5j2" is not valid in Turtle format.
To circumvent this issue, a workaround will be implemented:
gn:probeset1435395_s_at gnt:hasGeneManiaResource <https://genemania.org/search/mus-musculus/Atp5j2> . <https://genemania.org/search/mus-musculus/Atp5j2> rdf:type gnc:ResourceLink .
The straightforward approach would be to construct this structure in the front-end. However, the problem lies in the fact that these resource links are inferred, making it challenging to discern their connection within GN without visiting the website. Therefore, it's preferable to store this information in RDF despite the ease of constructing it in the front-end.
One link that's difficult to construct is the UCSC link:
To construct the above link, we need to have the ProbeSet.refseq_transcriptid and the GeneList.TxStart/GeneList.TxEnd. However, doing a join between these 2 tables takes too long and the generated file get's too big.
Transforming ProbeSet metadata takes long. The exact command:
time guix shell guile-dbi \
guile-hashing -m manifest.scm \
-- ./pre-inst-env \
./examples/probeset.scm --settings \
conn.scm --output \
/export/data/genenetwork-virtuoso/probeset-metadata.ttl \
--documentation ./docs/probeset-metadata.md
The aforementioned command takes:
A second try:
The file-size of the transformed metadata is 6.0G which is to be expected.
Optimisations---perhaps using guile-fibers---can be considered later.