• assigned:alem,Fred
• type: upgrade
• priority: medium
• status: closed, completed
• keywords: correlation,rust,parralel

• [X] implementation of basic pearson and spearman correlation in rust

• [X] add unittests and benchmarks

• [X] loading datasets;format

• [X] package the library as a guix package

• [x] gn2 integration

• [x] parsing input datasets

• [x] benchmark for all datasets rust v gn2 version

• [ ] Figure out how to tell cargo to use the declared dependencies, rather than dowloading the dependencies

• [*] db queries migration to gn3 or use partial correlation queries

• [x] add parralel computation

• [x] code optimization and minor fixes

• [x] lmbd parser

• [x] implement biweight mid correlation for rust

• [] fix lmdb-rust packaging

• [] production deployment

You can call the lib as an external process from any language or directly call it in a rust cargo

code can be found here:

• https://github.com/Alexanderlacuna/correlation_rust.git

added a documentation for correlation rust

• https://github.com/Alexanderlacuna/correlation_rust/blob/main/README.md

PR for integrating to genenetwork:

integration of correlation rust ongoing with @fred working also including optimizing of slow queries,code refatoring

use text files similar to gn1 see example

• https://github.com/genenetwork/genenetwork1/blob/d2381dd5bb35cb906032f7d530e9d14984e1de21/web/webqtl/correlation/CorrelationPage.py#L271

also something to be considered is lmdb

integrated codebase to store db in text files for probeset types

• https://github.com/genenetwork/genenetwork2/pull/736

• implemented Parallel computation implementation with rust completed

• implemented both lmdb and csv parser in rust

• https://github.com/Alexanderlacuna/correlation_rust/pull/3/files#diff-87f0275ed65ae8ffe36df25091c9b975defbc920da7b5ec3c2dadb7d57ad4d67