Edit this page | Blame

Update production checklist

Tasks

The following are at the system level

Install underlying Debian

For our production systems we use Debian as a base install. Once installed:

It may help to mount the old root if you have it. Now it is on

mount /dev/sdd2 /mnt/old-root/

Get Guix going

We can bootstrap with the Debian guix package. Next move the store to a large partion and hard mount it in /etc/fstab with

/export2/gnu /gnu none defaults,bind 0 0

Run guix pull

wrk@tux04:~$ guix pull -p ~/opt/guix-pull --url=https://codeberg.org/guix/guix-mirror.git

Use that to install guix in /usr/local/guix-profiles

guix package -i guix -p /usr/local/guix-profiles/guix

and update the daemon in systemd accordingly. After that I tend to remove /usr/bin/guix

The Debian installer configures guix. I tend to remove the profiles from /etc/profile so people have a minimal profile.

Check database

Basically recover the database from a backup is the best start and set permissions. We usually take the default mariadb unless production is already on a newer version - so we move to guix deployment.

On tux02 mariadb-10.5.8 is running. On Debian it is now 10.11.11-0+deb12u1, so we should be good. On Guix is 10.10 at this point.

apt-get install mariadb-server

Next unpack the database files and set permissions to the mysql user. And (don't forget) update the /etc/mysql config files.

Restart mysql until you see:

mysql -u webqtlout -p -e "show databases"
+---------------------------+
| Database                  |
+---------------------------+
| 20081110_uthsc_dbdownload |
| db_GeneOntology           |
| db_webqtl                 |
| db_webqtl_s               |
| go                        |
| information_schema        |
| kegg                      |
| mysql                     |
| performance_schema        |
| sys                       |
+---------------------------+

Recover database

We use borg for backups. First restore the backup on the PCIe. Also a test for overheating!

Check sending E-mails

The swaks package is quite useful to test for a valid receive host:

swaks --to testing-my-server@gmail.com --server smtp.network
=== Trying smtp.network:25...
=== Connected to smtp.network.
<-  220 mailrouter8.network ESMTP NO UCE
 -> EHLO tux04.network
<-  250-mailrouter8.network
<-  250-PIPELINING
<-  250-SIZE 26214400
<-  250-VRFY
<-  250-ETRN
<-  250-STARTTLS
<-  250-ENHANCEDSTATUSCODES
<-  250-8BITMIME
<-  250-DSN
<-  250 SMTPUTF8
 -> MAIL FROM:<root@tux04.network>
<-  250 2.1.0 Ok
 -> RCPT TO:<pjotr2020@thebird.nl>
<-  250 2.1.5 Ok
 -> DATA
<-  354 End data with <CR><LF>.<CR><LF>
 -> Date: Thu, 06 Mar 2025 08:34:24 +0000
 -> To: pjotr2020@thebird.nl
 -> From: root@tux04.network
 -> Subject: test Thu, 06 Mar 2025 08:34:24 +0000
 -> Message-Id: <20250306083424.624509@tux04.network>
 -> X-Mailer: swaks v20201014.0 jetmore.org/john/code/swaks/
 ->
 -> This is a test mailing
 ->
 ->
 -> .
<-  250 2.0.0 Ok: queued as 4157929DD
 -> QUIT
<-  221 2.0.0 Bye                                                                                                                             === Connection closed with remote host

An exim configuration can be

dc_eximconfig_configtype='smarthost'
dc_other_hostnames='genenetwork.org'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost='smtp.network'
CFILEMODE='644'
dc_use_split_config='false'
dc_hide_mailname='false'
dc_mailname_in_oh='true'
dc_localdelivery='maildir_home'

And this should work:

swaks --to myemailaddress --from john@network --server localhost

Backups

(made with skribilo)