Collect Magento logs with Papertrail

by Cristian Balan | Dec 13, 2020 | Linux, Magento, Monitoring

I’ve used Papertrail before its acquisition from SolarWinds and was impressed by its simple interface and the logs scrapping capabilities. At the time, I have set Papertrail to monitor a few auth logs and trigger Slack notifications whenever SSH access occurred from a different IP than those whitelisted.

Well, years have passed since I’ve last played with it and now I have a new use case to monitor Magento errors recorded in its var/log/system.log and var/log/exception.log files. As I’ve forgotten how I did this at the time I thought I’ll put down my steps which might help later on.

First thing I’ve gone straight into reading the docs. That brought me into the app log files aggregate page.

The first step is to download the latest remote_syslog2 script from their GitHub repository. The only problem I have with this method is that you will struggle to keep things up-to-date without manually checking the repo for a new version and update. Hopefully one day we would be able to install via an OS package.
Ok, let’s download the latest current version and install. As I’m using Ubuntu I’ve done this:
wget https://github.com/papertrail/remote_syslog2/releases/download/v0.20/remote-syslog2_0.20_amd64.deb
sudo dpkg -i remote-syslog2_0.20_amd64.deb

The second indicated step is that of configuring and starting remote-syslog. The example only shows you how to do it with a single log so without too much fuss I’ve instead downloaded the custom config file and replaced the content of /etc/log_files.yml with:

files:
  - /home/user/myMagentoWebsite.co.uk/public/var/log/system.log
  - /home/user/myMagentoWebsite.co.uk/public/var/log/exception.log
destination:
  host: logs7.papertrailapp.com
  port: 77777
  protocol: tls
exclude_patterns:
  - main.INFO
pid_file: /var/run/remote_syslog.pid

Figured out that we can add multiple files just under the first example line. Also, the original /etc/log_files.yml had an exclude_patterns example which I’ve used to prevent main.INFO records from filling Papertrail as these are outside the scope.

The remote-syslog GitHub page has obviously more examples and explanations if we want to dig further into how it works.

Finally, the sudo remote_syslog command should get us set.