Did you know you can still analyze your web traffic without Google Analytics? You can just use your NGINX access logs! As part of logrotate’s default configuration on Ubuntu, NGINX keeps an access log of the last 14 days. You can see if any access log is being rotated by checking the following directory: /etc/logrotate.d.
In here, you’ll find an nginx configuration file:
/var/log/nginx/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
endscript
postrotate
invoke-rc.d nginx rotate >/dev/null 2>&1
endscript
}
You can find these logs in /var/log/nginx/. If you want to analyze these access files with goaccess, you can run:
cd /var/log/nginx && zcat -f access.log* | goaccess --log-format=COMBINED -c
This allows you to get a quick overview of the last 14 days of visitors and requests. This way, you can find out useful information about visitors.
For example, I was able to determine I had around 1300 unique visitors, however 777 of these were crawlers, leaving around 423 visitors in 14 days, which is more than I expected.
This is what the app displays:
Dashboard - Overall Analyzed Requests (20/Oct/2019 - 03/Nov/2019) [Active Panel: Geo Location]
Total Requests 11052 Unique Visitors 1338 Requested Files 299 Referrers 0
Valid Requests 11052 Init. Proc. Time 0s Static Files 109 Log Size 0.0 B
Failed Requests 0 Excl. IP Hits 0 Not Found 1098 Tx. Amount 205.54 MiB
Log Source STDIN
> 15 - Geo Location Total: 78/78
Hits h% Vis. v% Tx. Amount Data
---- ------- ---- ------- ---------- ----
2934 50.17% 447 100.00% 82.65 MiB EU Europe
1369 23.41% 73 16.33% 31.15 MiB ├─ BE Belgium
353 6.04% 146 32.66% 9.95 MiB ├─ FR France
236 4.04% 3 0.67% 2.79 MiB ├─ SE Sweden
196 3.35% 55 12.30% 13.19 MiB ├─ CZ Czech Republic
180 3.08% 32 7.16% 3.65 MiB ├─ DE Germany
179 3.06% 54 12.08% 3.87 MiB ├─ RU Russian Federation
64 1.09% 12 2.68% 4.07 MiB ├─ GB United Kingdom
61 1.04% 7 1.57% 34.50 KiB ├─ NL Netherlands
56 0.96% 29 6.49% 131.90 KiB ├─ IE Ireland
47 0.80% 1 0.22% 11.12 KiB ├─ CH Switzerland
34 0.58% 4 0.89% 1.42 MiB ├─ UA Ukraine
31 0.53% 1 0.22% 6.67 KiB ├─ EU Europe
27 0.46% 1 0.22% 522.49 KiB ├─ LT Lithuania
24 0.41% 8 1.79% 2.27 MiB ├─ PL Poland
[?] Help [Enter] Exp. Panel 0 - Sun Nov 3 20:01:36 2019 [q]uit GoAccess 1.3
You can also filter out specific IP addresses, so you don’t track your own visits. In this example, Belgian visitors’ count is probably higher than usual because my own visits are (still) counted.
Anyway, here’s a few fun things I found out using these access logs:
- Plenty of the requests seemed to be coming from servers in China. (Asia was #1 in my regional overview.)
- Bots and such are constantly being used to find vulnerabilities in public websites. Looking at requests, I can see certain pages being requested frequently (and failing), like: /aa.php, /log.php, /2.php, /wp-login.php, /editBlackAndWhiteList, /qq.php, and more. I’m guessing some of these come from China.
- macOS Catalina (and I’m assuming any other future versions of macOS) get reported as Mac OS 10.1 Puma.
- I found that a link on my website was broken and was causing lots of 404’s. (Whoops!)
Overall, I’m really liking this. You may need to adjust your settings to retain more logs if you wish to retain your logs longer, and if you’re a company, you have to adhere to GDPR and anonymize IP addresses after a certain period of time.