Edit this page | Blame

Setting up Local Development Database

You need to set up a quick local database for development without polluting your environment.

Method 1 (Using Guix)

Setting up mariadb in a Guix container is the preferred and easier method. But, you need root access to run the container. The genenetwork2 repo comes with a guix system container definition to run MariaDB and Redis. From the genenetwork2 repo, you can build and run the container using:

$ sudo $(./containers/db-container.sh)

You should now be able to connect to the database using

$ mysql --protocol tcp -u root

Create a database db_webqtl_s

MariaDB [mysql]> CREATE DATABASE db_webqtl_s;

Load the small database dump into the database. You may find the small database either on space or tux02 at /home/aruni/gn2.sql.lz

$ lzip -cd gn2.sql.lz | mysql --protocol tcp -u root db_webqtl_s

Since this is a develpoment server accessible only from localhost, it is ok to use the root user with no password. Configure your development instance of genenetwork2 with the following SQL_URI.

SQL_URI="mysql://root@127.0.0.1:3306/db_webqtl_s"

Method 2 (Manual method without Guix)

  • An assumption is made that the GeneNetwork2 profile is in ~/opt/gn_profiles/gn2_latest for the purposes of this documentation. Please replace as appropriate.
  • We install the database files under ~/genenetwork/mariadb. Change as appropriate.

Set up directories

mkdir -pv ~/genenetwork/mariadb/var/run
mkdir -pv ~/genenetwork/mariadb/var/lib/data
mkdir -pv ~/genenetwork/mariadb/var/lib/mysql

Set up default my.cnf

cat <<EOF > ~/genenetwork/mariadb/my.cnf
[client-server]
socket=~/genenetwork/mariadb/var/run/mysqld/mysqld.sock
port=3307

[server]
user=$(whoami)
socket=~/genenetwork/mariadb/var/run/mysqld/mysqld.sock
basedir=~/opt/gn_profiles/gn2_latest
datadir=~/genenetwork/mariadb/var/lib/data
ft_min_word_len=3
EOF

Install the database

~/opt/gn_profiles/gn2_latest/bin/mysql_install_db \
    --defaults-file=~/genenetwork/mariadb/my.cnf

Running the daemon:

~/opt/gn_profiles/gn2_latest/bin/mysqld_safe \
    --defaults-file=~/genenetwork/mariadb/my.cnf

Connect to daemon

~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf

Set up password for user

MariaDB [(none)]> USE mysql;
MariaDB [mysql]> ALTER USER '<your-username>'@'localhost' IDENTIFIED BY '<the-new-password>';
MariaDB [mysql]> FLUSH PRIVILEGES;

Now logout and login again with

$ ~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf --password mysql

enter the newly set password and voila, you are logged in and your user has the password set up.

Now, set up a new user, say webqtlout, and a default database they can connect to

MariaDB [mysql]> CREATE DATABASE webqtlout;
MariaDB [mysql]> CREATE USER 'webqtlout'@'localhost' IDENTIFIED BY '<some-password>';
MariaDB [mysql]> GRANT ALL PRIVILEGES ON webqtlout.* TO 'webqtlout'@'localhost';

Now logout, and log back in as the new webqtlout user:

~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf \
    --user=webqtlout --host=localhost --password webqtlout

and enter the password you provided.

Setting up the Small Database

Download the database from

Say you downloaded the file in ~/Downloads, you can now add the database to your server.

First stop the server:

$ ps aux | grep mysqld # get the process ids
$ kill -s SIGTERM <pid-of-mysqld> <pid-of-mysqld_safe>

Now extract the database archive in the mysql data directory:

$ cd ~/genenetwork/mariadb/var/lib/data
$ p7zip -k -d ~/Downloads/db_webqtl_s.7z

Now restart the server:

~/opt/gn_profiles/gn2_latest/bin/mysqld_safe \
    --defaults-file=~/genenetwork/mariadb/my.cnf

Then update the databases

$ ~/opt/gn_profiles/gn2_latest/bin/mysql_upgrade \
    --defaults-file=~/genenetwork/mariadb/my.cnf \
    --user=frederick --password --force

and login as the administrative user:

$ ~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf \
    --user=$(whoami) --password

and grant the privileges to your normal user:

MariaDB [mysql]> GRANT ALL PRIVILEGES ON db_webqtl_s.* TO 'webqtlout'@'localhost';

now logout as the administrative user and log back in as the normal user

~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf \
    --user=webqtlout --host=localhost --password db_webqtlout_s

MariaDB [db_webqtlout_s]> SELECT * FROM ProbeSetData LIMIT 20;

verify you see some data.

A Note on Connection to the Server

So far, we have been connecting to the server by specifying --defaults-file option, e.g.

~/opt/gn_profiles/gn2_latest/bin/mysql \
    --defaults-file=~/genenetwork/mariadb/my.cnf \
    --user=webqtlout --host=localhost --password db_webqtlout_s

which allows connection via the unix socket.

We could drop that specification and connect via the port with:

~/opt/gn_profiles/gn2_latest/bin/mysql \
    --user=webqtlout --host=127.0.0.1 --port=3307 --password db_webqtlout_s

In this version, the host specification was changed from

--host=localhost

to

--host=127.0.0.1

whereas, the **--defaults-file** file specification was dropped and a new **--port** specification was added.


(made with skribilo)