Edit this page | Blame

Improving Alerting/Logging in GN2

Problem Statement

Current logging in the genenetwork ecosystem is noisy and difficult to parse programatically which makes it hard to:

  • Integrate logs into some observability pipeline (E.g. sheepdog).
  • Troubleshoot issues as they occur. We always learn of bugs from users.

Goals

  • Standardize logging format and config across GN2 flask apps and gn-guile.
  • Adopt structured logging.
  • Extend sheep-dog to be able to parse gn logs and send alerts on e-mail or matrix.

Non-goals

  • Logging in GN3.
  • Parsing logs to extract goals.
  • Getting rid of "gn.db" global object and in so doing removing "MySqlAlchemy" [that we really shouldn't be using].
  • Adding log messages to existing functions.

Actual Design

  • Get rid of "utility.logger" module and replace it with Flask's or Python's in-built logging.
  • Configure the logging system to automatically add the module name, line number, time-stamps etc.
  • Use a better format for displaying the logs.
  • Delete "with Bench..." calls.

Resources

(made with skribilo)