January 11, 2015

grep and buffering

Recently while trying to troubleshoot a production issue I needed to tail a log and grep for some data. Problem was the data I was expecting to show up didn't. Well not exactly. It showed up about 10 minutes after I expected it to. Turns out using grep twice on a tail buffers like crazy.

tail -f /var/log/nginx/access.log | grep 1.2.3.4 | grep -i html

You can fix the delay in showing results by using the line buffered flag to the middle grep like so:

tail -f /var/log/nginx/access.log | grep --line-buffered 1.2.3.4 | grep -i html

Tags: grep