, 3 min read
Using GoAccess with Hiawatha Web-Server
Original post is here eklausmeier.goip.de/blog/2021/05-22-using-goaccess-with-hiawatha-web-server.
GoAccess is a remarkable analyzer for your log-files written by the web-server. For example, GoAccess can read and analyze the log-files from Apache web-server. In the same vein, after some configuration, it can also read and analyze the log-files from Hiawatha web-server. The Hiawatha web-server is used for this blog. I have written on Hiawatha here: Set-Up Hiawatha Web-Server, and Set-Up "Let's Encrypt" for Hiawatha Web-Server.
Below is the required configuration such that GoAccess can read Hiawatha log-files:
date-format %a %d %b %Y
time-format %T %z
log-format %h|%d %t|%s|%b|%r|%R|%u|Host: %v|%^|%^|%^|%^|%^|%^|%^|%^|%^|%^
This should be placed in file $HOME/.goaccessrc.
First go to the log-file directory: cd /var/log/hiawatha. Running goaccess is thus:
goaccess access.log
Usually you will have multiple log-files, the majority of which will be zipped. GoAccess can read from stdin, therefore you can feed the unzipped log-files to GoAccess. For example:
zcat access.*.gz | goaccess access.log access.log.1 > /srv/http/goaccess.html
The HTML output looks something like this for OS distribution:
Overall info in HTML format looks like this:
I was quite surprised how much traffic the web-server handles, even before this blog was hosted on Hiawatha. Much of this traffic seems to stem from bots or is just junk. See Filtering Bots and Crawlers from Access.log to see the percentage of bots on this website.
GoAccess can either produce HTML reports, or can show the results directly in your terminal. Below is the output from st or xterm:
Dashboard - Overall Analyzed Requests (09/May/2021 - 22/May/2021) [Active Panel: Requests]
Total Requests 10782 Unique Visitors 1031 Requested Files 1119 Referrers 199
Valid Requests 10576 Init. Proc. Time 1s Static Files 138 Log Size 3.40 MiB
Failed Requests 206 Excl. IP Hits 0 Not Found 2218 Tx. Amount 192.51 MiB
Log Source access.log; access.log.1
> 2 - Requested Files (URLs) Total: 366/1119
Hits h% Vis. v% Tx. Amount Mtd Proto Data
---- ------ ---- ------ ---------- ---- -------- ----
893 8.44% 589 57.13% 1.06 MiB GET HTTP/1.1 /
191 1.81% 155 15.03% 1.59 MiB GET HTTP/1.1 /blog/index.html
112 1.06% 16 1.55% 24.81 KiB HEAD HTTP/1.1 /
101 0.95% 16 1.55% 929.90 KiB POST HTTP/1.1 /build/search.php
68 0.64% 58 5.63% 557.15 KiB GET HTTP/1.1 /build/blog/index.html
54 0.51% 30 2.91% 186.07 KiB GET HTTP/1.1 /feed.xml
47 0.44% 6 0.58% 10.97 KiB HEAD HTTP/1.1 /blog/index.html
3 - Static Requests Total: 138/138
Hits h% Vis. v% Tx. Amount Mtd Proto Data
---- ------ ---- ------ ---------- ---- -------- ----
225 2.13% 83 8.05% 531.72 KiB GET HTTP/1.1 /favicon.ico
140 1.32% 133 12.90% 38.95 KiB GET HTTP/1.1 /robots.txt
10 0.09% 9 0.87% 168.17 KiB GET HTTP/1.1 /build/img/jpilot-plugin.png
10 0.09% 9 0.87% 593.29 KiB GET HTTP/1.1 /build/img/jpilot-search.png
8 0.08% 6 0.58% 1.67 KiB HEAD HTTP/1.0 /robots.txt
4 0.04% 2 0.19% 58.77 KiB GET HTTP/1.1 /img/PerfRyzenIntelARM1.png
4 0.04% 1 0.10% 2.03 MiB GET HTTP/1.1 /build/img/IMG_20140413_124816.jpg
4 - Not Found URLs (404s) Total: 366/2218
Hits h% Vis. v% Tx. Amount Mtd Proto Data
---- ------ ---- ------ ---------- ---- -------- ----
59 0.56% 0 0.00% 54.00 KiB GET HTTP/1.1 /wp-login.php
54 0.51% 0 0.00% 50.73 KiB GET HTTP/1.1 /phpmyadmin/
32 0.30% 0 0.00% 29.24 KiB POST HTTP/1.1 /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php
30 0.28% 0 0.00% 28.18 KiB GET HTTP/1.1 /ads.txt
29 0.27% 0 0.00% 26.48 KiB GET HTTP/1.1 /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php
27 0.26% 0 0.00% 25.26 KiB GET HTTP/1.1 /_ignition/execute-solution
23 0.22% 0 0.00% 21.50 KiB GET HTTP/1.1 /wp-content/plugins/wp-file-manager/readme.txt
5 - Visitor Hostnames and IPs Total: 366/830
[?] Help [Enter] Exp. Panel 0 - Sat May 22 18:41:56 2021 [q]uit GoAccess 1.4.6
Pressing Tab-key will navigate through the various panels.
There is a short mentioning in Wikipedia as well: GoAccess. The man-page provides a wealth of information.