Obfuscate sensitive data in the navajo log

There are a couple of ways to prevent sensitive data from getting into the navajo log file. Note that nevisProxy itself does not know what piece of information you consider as sensitive and what not. Therefore, be able to impose some structure on your sensitive data, in the form of a regex.

The examples below use the regex hello=somesensitivevalue to detect sensitive data. Adapt the examples to your own regular expressions.

Masking sensitive data

The simplest way to prevent sensitive data from getting into the navajo log file is to mask the sensitive data. Use the following pipe command:

BC.Tracer.LogFile=pipe://(sed 's/hello=\([a-zA-Z0-9_]*\)/hello=CENSORED/g' |
/opt/nevisproxy/bin/bclogmgr size=10000000
archives=5 /var/opt/nevisproxy/default/logs/obfuscated_navajo.log)

Encrypting the sensitive data

Another possible way to prevent sensitive data from getting into the navajo log file is to encrypt (and later decrypt) the values with the following pipe command:

BC.Tracer.LogFile=pipe://(./logProcessor.sh | /opt/nevisproxy/bin/bclogmgr size=10000000
archives=5 /var/opt/nevisproxy/default/logs/obfuscated_navajo.log)

In the above command, logProcessor.sh is a custom script. See the sample code block below. (Note that this is just an example. In production, it can be a Perl script or anything else.)

#This tool encrypts sensitive data (=matching a regex) found in navajo.log lines.
It reads the standard input.
 
keyfile=/home/mykey.pem
 
function encrypt () {
         #no line breaks in base64
         echo $1 | openssl rsautl -inkey $keyfile -encrypt | base64 -w 0
}
 
regex="hello=([a-zA-Z0-9_]*)"
 
while read line
do
  if [[ "$line" =~ $regex ]]; then
           sensitivedata=${BASH_REMATCH[1]}
           encrypted=$(encrypt "$sensitivedata")
           finalencrypted="hello=$encrypted"
 
           #do not print if for some reason couldn't substitute
           #use different separator characters, cause base64 values can contain slashes
           echo $line | sed -r -n "s|$regex|$finalencrypted|gp"
  else
           echo $line
  fi
done

With the above pipe command/script, the system writes encrypted values into the log file:

2017 10 25 13:12:20.355 isi3web NavajoOp 23403.139750091425536.0a14d5f3-5b6b-140af3d5-
15f533bab7d-00000000 6-INFO : <<<<< 'GET
/somepathprefix/somefile?hello=tn7YXhn4YUvFSWMCZCYSRPlWHURGcc5dr/FXI6b9nWvv8969XgYvhKLoq
3Qynowj/kWOB21NmseTJcUVETqfn9sF+RnjLaXe8C6r9wwwJg1qZn84hz721w4xjj94qLdzrf/b6CocP0SjoJbMy
abh7ZVIQooJ1Tv38KB1E1QmO/4=' invS='>TestServlet' sC='200' bS='3877' dT='4'
rmIP='10.20.213.243' clID='<NULL>' trID='0a14d5f3-5b6b-140af3d5-15f533bab7d-00000000'
(cR=1)

You can decrypt these values individually (such as with BC.Tracer.DebugProfile.Secrets) through the following command:

echo "the_encrypted_string" | base64 --decode | openssl rsautl -inkey /home/mykey.pem -decrypt

By doing so, you have your own log obfuscator.

This solution works also off-line. So to obfuscate an existing log file, send it to this pipe command (for example with the cat command).

Caveats

If you have enabled request/response dumping, the above solutions are not appropriate. First, it is possible that your regex breaks into two lines in the dump, so the simple regex matching does not work. Additionally, and more important, the dump will still contain all sensitive data, in the form of hexadecimal numbers.