• assigned: alexm
• type: feature, improvement
• priority: high
• keywords: correlations, materialised views, optimisation, script
• status: inprogress

This issue supercedes the

• Materialised Views for Correlations

issue, but the information in the older issues is helpful here too.

A command-line script needs to be created in GN2 that can be invoked any time there is need to regenerate the text files that help speed up the computations.

The script could be triggered in any of the following ways

• manually: a human invokes the script manually to generate the text file(s)
• periodically/scheduled: An automated scheduler invokes the script at predefined times/intervals
• on-change: A "job" is run in the background to recreate the text file whenever data for a particular text dataset/trait/... is changed (created/updated/deleted).

The automated triggers can be built in later once we have a working script. My (fredm) preferred method is to have the "on-change" trigger to avoid running the script even when data has not changed.

[x] enable to compute with cache and without

[x] fetching all possible strains for specific probeset

[x] fetch all probesetfreezes for script

[] store results in lmdb

[] table tracker for change in tables then invoke the script

[] add as a job

while using innodb engine last update time is null possibly write a script/table o create track of this

relevant resources

• https://dba.stackexchange.com/questions/291291/how-to-check-the-last-time-a-table-has-been-modified

• https://github.com/genenetwork/genenetwork2/pull/764