- Using curl
document created on Dec 23 2022 by Pjotr Prins
..."2310010I16Rik"
sheepdog_run.rb -c "curl \"$URL\"|grep -i $FIND" --tag "wild-card-search-$URL" $*
```
## POST fetch
Post fetches are a bit harder. First use firefox console and click of Debugger tab.
- CI/ CD for genetwork projects
document created on Oct 12 2022 by Munyoki Kilyungi, last updated on Feb 27 2024 by Pjotr Prins
...or reuse container from staging; tag container as a release container
* Deploy container to production
* Generate documentation for tagged commit
* Generate guix declaration for re-generating...
- Backup Drops
document created on Oct 27 2022 by Pjotr Prins, last updated on Feb 18 2024 by Pjotr Prins
...-v --tag "drop-mount-dropserver" -c "sshfs -o IdentityFile=~/.ssh/id_ecdsa_backup bacchus@dropserver:/ ~/mnt/this"
sheepdog_run.rb -h rabbit --always -v --tag "drop-rsync-dropserver" -c "rsync -vrltDP...
- GeneNetwork CLI
document created on Sep 20 2021 by jgart, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
...CLI
## Tags
* assigned: ??
* type: feature-request
* priority: low
* keywords: command-line interface, CLI
## Description
It would be great for GeneNetwork to have a nice command line client for doing...
- Add button to delete an entire dataset in gn2
document created on Sep 08 2021 by BonfaceKilz, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
Add button to delete an entire dataset in gn2
## Tags
* assigned: ??
* type: bug
* keywords: ui, dangerous feature, status: unclear
## Description
An interface for deleting data existed in gn1 but...
- Using pudb
document created on Dec 21 2021 by jgart, last updated on Dec 03 2023 by Pjotr Prins
...as Python���s built-in pdb module. Just replace pdb with pudb. (One
> exception: run is called runstatement.)
=> https://docs.python.org/3/library/pdb.html?highlight=pdb#module-pdb
## Tags...
- Running pudb From an External Environment
document created on Dec 03 2023 by Pjotr Prins
...--load=guix.scm
4) Inside the container, run the debugger with something like:
* env PYTHONPATH="${PYTHONPATH}:${HOME}/pudb_env/lib/python3.8/site-packages" ... python3 -m pudb script.py
## Tags...
- Using pudb with pytest
document created on Dec 21 2021 by jgart, last updated on Dec 03 2023 by Pjotr Prins
Using pudb with pytest
```
pytest --pdbcls pudb.debugger:Debugger --pdb --capture=no
```
=> https://documen.tician.de/pudb/starting.html?highlight=pytest#usage-with-pytest
## Tags...
- Profiling Python code
document created on Dec 03 2023 by Pjotr Prins
...e.g.
* env [various-env-vars] python3 -m cProfile the-script.py 2>error.log 1>performance.log
which should help will separating errors from the performance data, for easier analysis
## Tags...
- LMM Notes
document created on Aug 13 2021 by Pjotr Prins, last updated on Jan 08 2024 by Solomon Shelby
...Tags
* assigned: ??
* keywords: LMM
* status: unclear
* priority: low
## Time series data
FlxQTL by Saunak Sen can handle time series data in an (genetic) LMM
context. They have to be measured...
- Defining a scheme sexp comment with a reader macro
document created on Feb 02 2022 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...-character #\# #\; (lambda (s c n) c n (read s t nil t) (values)))
CL-USER(2): (+ 3 (+ 2 3) 1)
9
CL-USER(3): (+ 3 #;(+ 2 3) 1)
4
```
Special thanks to Ed Langley on #common-lisp:matrix.org.
## Tags...
- Debugging Lisp
document created on Dec 22 2021 by Pjotr Prins, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...provided by Common Lisp
Some links
=> https://malisper.me/debugging-lisp-part-1-recompilation/ Debug Common Lisp (CL)
=> https://www.youtube.com/watch?v=bl8jQ2wRh6k Video on CL web development
## Tags...
- Common LISP: sly
document created on Jan 29 2022 by Pjotr Prins, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
Common LISP: sly
This document will collect some resources on sly
## Tags
* type: documentation
* keywords: common-lisp, sly, emacs
## Resources
=> https://joaotavora.github.io/sly/ SLY User Manual
- Common LISP: Notes
document created on Feb 09 2022 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...com/nonguix/nonguix/-/blob/master/nongnu/packages/emacs.scm#L26 clhs
## Getting Help / Asking Questions
### Libera Chat
```
#clschool
#commonlisp
```
### Matrix
```
#common-lisp:matrix.org
```
## Tags...
- Non-Clustered and Flippable Heatmaps
document created on Oct 01 2021 by Frederick Muriuki Muriithi, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...be possible to flip the heatmap orientation from horizontal to vertical and vice-versa
* It should be possible to generate a heatmap with no dendrograms
## Members
* fredm
* pjotr
* zsloan
## Tags...
- Ideas and References
document created on Nov 22 2021 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
Ideas and References
=> gemini://drewdevault.com/2021/11/15/Reframing-gemini.gmi Reframing the philosophy of Gemini
## Tags
* type: documentation
* keywords: documentation, gemini
- gemtext cheatsheet
document created on Dec 21 2021 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
gemtext cheatsheet
=> gemini://gemini.circumlunar.space/docs/cheatsheet.gmi
=> https://gemini.circumlunar.space/docs/cheatsheet.gmi proxy
## Tags
* type: documentation
* keywords: gemini
- GeneNetwork documentation
document created on Nov 16 2021 by Pjotr Prins, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
.../gn-docs/blob/master/general/help/facilities.md
the markdown is rendered on github, which makes it easy to edit for most users, so it should be the default.
## Tags
* assigned: pjotrp
* status: unclear...
- Data Uploads: Reversing a Diff
document created on Feb 03 2022 by BonfaceKilz, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...Reversing a Diff
## Tags
* assigned: bonfacem, zachs
* status: unclear
* type: enhancement
* priority: high
* keywords: data uploads
### Introduction
Given a diff file, we should be able to reverse...
- Put user info in the a/c name in the menu
document created on Mar 14 2022 by Frederick Muriuki Muriithi, last updated on Dec 04 2023 by Frederick Muriuki Muriithi
Put user info in the a/c name in the menu
When the user signs in to GN2, there is no info on the account name in the menu.
## Tags
* assigned: bonfacem
* priority: medium
* status: closed, completed...
- Replace Hard coded authentication with gn-proxy
document created on Mar 14 2022 by Frederick Muriuki Muriithi, last updated on Dec 04 2023 by Frederick Muriuki Muriithi
...Hard coded authentication with gn-proxy
## Tags
* assigned: bonfacem
* priority: medium
* type: enhancement
* status: closed, completed
* keywords: authentication, gn-proxy
## Description
Originally...
- COVID-19
document created on Nov 14 2021 by Pjotr Prins, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
COVID-19
We are setting up a Cloud HPC for COVID-19 using Arvados as the underlying infrastructure.
## Tasks
=> https://github.com/yatisht/usher Fast phylogeny
## Tags
* assigned: pjotrp...
- Jupyter notebooks
document created on Sep 01 2021 by Pjotr Prins, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
...notebooks
## Tags
* type: documentation
* keywords: documentation, jupyter, notebooks
* status: unclear
## Run with GNU Guix
Jupyter notebooks can be installed in a profile
```
~/opt/guix-latest/bin/...
- Adding Terminal Output
document created on Aug 07 2021 by Pjotr Prins, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
...As GN3 (REST) API handles calling external tools it kinda make sense to handle the terminal output page through GN3 and communicate through websockets. But first I'll do a Sinatra test run.
## Tags...
- Shapiro Wilk Test
document created on Nov 24 2021 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...orderNorm function to
perform Ordered Quantile normalization43 on all phenotypes. QTL
--
A quantitative trait variant in Gabra2 underlies increased methamphetamine stimulant sensitivity
```
## Tags...
- Editing Phenotypes
document created on Jul 07 2023 by Frederick Muriuki Muriithi, last updated on Jul 12 2023 by Frederick Muriuki Muriithi
...Tags
* keyword: editing phenotypes
## Introduction
There is old code for editing phenotypes in place, but it was not in use. The main aim at this point is to fix any bugs and adapt it to make use...
- NGINX Configuration Notes
document created on Jun 01 2023 by Zachary Sloan
NGINX Configuration Notes
## Tags
* type: info, documentation
* keywords: nginx, configurations
## Use of proxy_redirect (in this case for authentication, which runs on GN3 through a GN2 URI)...
- Lisp For Schemers
document created on Dec 29 2021 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...Schemers
## Tags
* type: documentation
* keywords: LISP, Scheme, iota, loop, iterate
## iota
### In Scheme
```
scheme@(guile-user)> (iota 12)
$1 = (0 1 2 3 4 5 6 7 8 9 10 11)
```
### In Common Lisp...
- BNW interface for GN2
document created on Oct 18 2021 by Pjotr Prins, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
...point.
## Tags
* type: bug
* priority: high
* assigned: pjotrp
* status: in progress
* keywords: BNW
## Todo
The current output reads:
```
<title>Opening BNW</title>
<form method="post" action="http...
- Adding Platforms
document created 4 weeks ago by Frederick Muriuki Muriithi, last updated 10 days ago by Frederick Muriuki Muriithi
Adding Platforms
## Tags
* status: open
* priority: medium
* type: documentation
* assigned: fredm, flisso, acenteno
* keywords: doc, documentation, platform, gn-uploader, uploader, adding data...
- Weechat
document created on Nov 04 2021 by Pjotr Prins, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...Tags
* type: documentation
* keywords: cli, weechat, documentation
Weechat is a very versatile terminal chat that can handle IRC, Matrix and Slack.
## Weechat install + slack
Arun has packaged...
- MariaDB
document created on Oct 26 2021 by Pjotr Prins, last updated on Feb 20 2024 by Pjotr Prins
...actions
To install Mariadb (as a container) see below and
=> topics/deploy/installation
# Tags
* type: info, documentation
* keywords: mariadb, systems
# Check the transaction logs
Start the client...
- Improving Metadata Audit
document created on Jul 25 2023 by Frederick Muriuki Muriithi, last updated on Aug 11 2023 by Frederick Muriuki Muriithi
...Tags
* assigned: fredm
* priority: high
* keywords: editing, metadata, metadata audit
* type: documentation
* status: open
## Introduction
There is an existing metadata audit system that tracks...
- Fallbacks
document created on Feb 18 2024 by Pjotr Prins, last updated 4 weeks ago by Pjotr Prins
...fallback servers using Arun's Guix machine configuration. Outside the system container there are a few dependencies, particularly MariaDB and a number of file directories.
# Tags
* assigned: pjotrp...
- BigBlueButton
document created on Mar 05 2022 by jgart, last updated on Feb 28 2023 by Pjotr Prins
...Tags
* type: documentation
* keywords: BigBlueButton
## BASH script to install BigBlueButton in 30 minutes.
https://github.com/bigbluebutton/bbb-install
```
wget -qO- https://ubuntu.bigbluebutton...
- Top 10 Things to Test
document created on Mar 28 2022 by BonfaceKilz
...10 Things to Test
## Tags
* assigned: bonfacem
* type: tests
* keywords: documentation, testing, checklist
## Introduction
Important things:
- Correctness
- Speed (fast enough)
- Reliability
Let's do...
- GN Services
document created on Sep 15 2021 by Pjotr Prins, last updated on Dec 23 2022 by Pjotr Prins
...Tags
* type: documentation
* status: in progress
* keywords: gn services, documentation
# Description
First make sure to have a stable guix checkout. E.g.
```
gn3@tux01:~/production/gn-proxy$ /home/...
- Shepherd
document created on Aug 27 2021 by Pjotr Prins, last updated on Dec 02 2023 by Pjotr Prins
...record information on running shepherd (on Debian).
# Tags
* assigned: pjotrp, efraimf
* type: documentation
* keywords: system, shepherd
* status: wip
* priority: normal
# Issues
* [ ] Currently guix...
- Adding Species
document created 10 days ago by Frederick Muriuki Muriithi, last updated 10 days ago by Frederick Muriuki Muriithi
Adding Species
## Tags
* status: open
* priority: medium
* type: documentation
* assigned: fredm, flisso, acenteno
* keywords: doc, documentation, platform, gn-uploader, uploader, adding data...
- Add Metadata To The Trait Page (RDF)
document created on Sep 28 2022 by Munyoki Kilyungi, last updated on Dec 03 2023 by Pjotr Prins
...and contributors.
* Add metadata as links, tooltips, or html <summary> tag to the relevant html section(s).
## Resources
=> https://www.linkedin.com/pulse/six-secret-sparql-ninja-tricks-kurt-cagle/
- Invoking SQLite3: CLI
document created 5 weeks ago by Frederick Muriuki Muriithi
...Tags
* status:
* assigned:
* priority:
* type: documentation
* keywords: doc, documentation, sqlite, sqlite3
## TLDR
At minimum, always invoke sqlite with "PRAGMA foreign_keys = ON;" as follows...
- Phenotype Naming Conventions
document created on Nov 23 2022 by Munyoki Kilyungi, last updated on Dec 03 2023 by Pjotr Prins
...BW_M_6m* as the body weight of males at 6 months for 6 or more publications. Then we may need to programmatically add further tags such as year of publication (last two digits).
3. We have to decide...
- Generating Key-Pairs
document created 2 weeks ago by Frederick Muriuki Muriithi, last updated 12 days ago by Frederick Muriuki Muriithi
...Key-Pairs
## Tags
* type: documentation
* keywords: doc, documentation, gn-auth, key-pair, jwt
## Generating the Key-Pair
If openssl is not present on your system, you need to get it. You can either...
- Common Lisp Condition Handling
document created on Dec 28 2021 by jgart, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
.../Condition-Handling-2001.html Condition Handling in the Lisp Language Family
=> https://gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html Conditions and Restarts
## Tags...
- Data Uploads: Inserting Data
document created on Jan 27 2022 by BonfaceKilz, last updated on Mar 16 2022 by Frederick Muriuki Muriithi
...Tags
* assigned: bonfacem, zachs
* type: bug
* priority: high
* status: in progress
* keywords: data uploads
### Introduction
The current uploader work documented in `editing-data.gmi` only caters...
- Logging with Flask and Gunicorn
document created 3 weeks ago by Frederick Muriuki Muriithi
...Tags
* type: documentation
* keywords: deploy, deployment, logging, flask, gunicorn
## Logging with Flask and Gunicorn
Flask has its own output logger, `flask.logger`, which is an extension...
- Using PDB to Troubleshoot Python Code
document created on Dec 03 2023 by Pjotr Prins
...Code
## Tags
* type: documentation
* keywords: debugging, python
Historically, debugging GeneNetwork code has been a bit of a pain where you would have `print' and `logging' statements to help view...
- GeneNetwork Authentication/Authorisation Architecture
document created on Dec 04 2023 by Frederick Muriuki Muriithi
.../Authorisation Architecture
## Tags
* type: docs, documentation
* status: ongoing, open
* keywords: authentication, authorisation, architecture, docs, documentation
* author: Frederick Muriuki Muriithi...
- Partial Correlation
document created on Oct 15 2021 by Frederick Muriuki Muriithi, last updated on May 16 2022 by Frederick Muriuki Muriithi
...Add in search results.
Pick 3 and hit 'Partial'
Put one each in X, Y and Z columns
And compute against database (lower half).
That gives you a list of hits.
## Members
* fredm
* pjotp
* alex
## Tags...
- Data Upload Process
document created 4 days ago by Munyoki Kilyungi, last updated 4 days ago by Munyoki Kilyungi
...Upload Process
## Tags
* assigned: priscilla
* keywords: documentation, gn-uploader
The process of uploading data to the GeneNetwork platform began with retrieving the dataset from the Gene Network...
- GeneNetwork Hacking Documentation
document created on Mar 11 2022 by Frederick Muriuki Muriithi, last updated on Jul 18 2022 by Frederick Muriuki Muriithi
...to help with clarifying some concepts and critiquing the documentation.
The goal of this document is to encourage and track the documentation that assist with the development of the system.
## Tags...
- Automated Testing
document created on Oct 12 2022 by Munyoki Kilyungi, last updated on Dec 03 2023 by Pjotr Prins
...Testing
## Tags
* keywords: testing, CI, CD
## Introduction
As part of the
=> ../systems/ci-cd.gmi CI/CD effort
there is need for automated tests to ensure that the system is working as expected.
- Fire up system container for GN
document created on Mar 02 2024 by Pjotr Prins, last updated 4 weeks ago by Pjotr Prins
Fire up system container for GN
# Tags
* assigned: pjotrp
* priority: medium
* type: system administration
* keywords: system administration, GN, tux01, tux02, tux04, balg01
# Tasks
We have the...
- Editing Data
document created on Nov 23 2021 by BonfaceKilz, last updated on Jul 25 2023 by Frederick Muriuki Muriithi
...Tags
* assigned: bonfacem
* keywords: metadata
* status: unclear
* priority: medium
### Introduction
At the moment, you can edit metadata related to a published phenotype and a probeset. When an...
- Deploying gn-auth
document created on Mar 04 2024 by Frederick Muriuki Muriithi, last updated 8 weeks ago by Frederick Muriuki Muriithi
Deploying gn-auth
## Tags
* type: doc, docs, documentation
* keywords: gn-auth, deployment
* status:
* priority:
* assigned:
## ���
���
### Run Migrations
At this point, you have...
- Binding notebooks
document created on Aug 22 2021 by jgart, last updated on Mar 14 2022 by Frederick Muriuki Muriithi
...R/shiny notebooks and Pluto notebooks. These binding tools do a lot more and do not leverage GNU Guix lightweight container infrastructure. In this project we want to create a binding-lite.
## Tags...
- Editing Case-Attributes
document created on Jul 07 2023 by Frederick Muriuki Muriithi, last updated 8 weeks ago by Frederick Muriuki Muriithi
...-Attributes
## Tags
* type: document
* keywords: case-attribute, editing
* assigned: fredm, zachs, acenteno
* status: requirements gathering
## Introduction
Case-Attributes are essentially the metadata...
- GeneNetwork Uploader Requirements
document created on Feb 22 2024 by Frederick Muriuki Muriithi, last updated on Feb 22 2024 by Frederick Muriuki Muriithi
...Uploader Requirements
*OR*: "Reattaching the Head on Frederick's Headless-Chicken Development :-)"
## Tags
* assigned: fredm, flisso
* type: doc, requirements
* priority: high
* status: open...
- Migrate GN1 Clustering
document created on Jul 18 2021 by Pjotr Prins, last updated on Mar 22 2022 by Frederick Muriuki Muriithi
...map' icons. Computation may take a while.
Next click on 'Cluster traits' and 'Redraw' map. This is the one we want in GN3/GN2!
=> ./heatmap.png
## Members
* fredm
* pjotr
## Kanban
* fredm
## Tags...
- Precompute mapping input data
document created on Mar 20 2023 by Pjotr Prins, last updated on Feb 12 2024 by Pjotr Prins
...functionality. Here we prepare for a new generation of functionality that introduces LMMs for compute and multiple significant scores for queries.
# Tags
* assigned: pjotrp
* type: precompute, gemma...