Skip to main content

IDS / IDP Troubleshooting Guide

Introduction​

This documentation shares tips on how to best configure logging, but more importantly, to suggest how to approach troubleshooting with Nevis logs.

Specific pointers regarding component troubleshooting are organised into 3 sections:

  • Logging Configuration
  • Understanding Log Entries
  • Example Troubleshooting Approaches

Kubectl Access​

Log files can be inspected/downloaded after the event via nevisAdmin4. However, it may be advantageous to stream the log files and identify entries of interest in real-time. For this, kubectl access to the cluster is required - something that is easily done with Identity Suite (IDS) deployments.

In the case of Azure (OneClick) deployments, to gain access to your cluster make sure your client has both Azure CLI and kubectl installed.

  1. Login to Azure from your client:

    az login

  2. In case you have multiple subscriptions, list/set the desired subscription:

    az account list --output table
    az account set --subscription "Your Subscription Name or ID"

  3. Discover cluster details (make sure in browser session that you have logged into the relevant directory so as to have permission):

    az aks list --output table

  4. Get the Cluster Credentials (typically obtain RG and AKS name from Azure portal):

    az aks get-credentials --resource-group <resource-group-name> --name <aks-cluster-name>

  5. Your kubeconfig file (~/.kube/config) should now have been updated with your AKS credentials. Test access e.g. get pods running in the nevis namespace:

    kubectl get pods -n nevis

Logging in Real Time​

This is like issuing a tail -f command with Linux which allows you to view log entries that appear in real time:

  1. Find the name of the pod you wish to access logs for:

    kubectl get pods -n <your-namespace>

  2. Use the --follow command to stream a pod’s logs:

    kubectl logs <pod-name> -n <your-namespace> --follow

  3. To search for specific strings in the streamed log and only output entries with a hit, use the pipe and grep approach:

    kubectl logs <pod-name> -n <your-namespace> --follow | grep "<your_search_string>"

  4. Use multiple pipe/grep to further filter the hits to entries that contain multiple values:

    kubectl logs <pod-name> -n <your-namespace> --follow | grep "<search_string1>" | grep "<search_string2>"

    note

    The order matters, i.e. if the first grep finds a match, then the 2nd string search for that entry will only search from that point to the end of the entry.

  5. Use egrep to search for and output one or other terms:

    kubectl logs <pod-name> -n <your-namespace> --follow | egrep "<search_string1>|<search_string2>"

info

If you don’t have kubectl access to be able to grep files for matches and instead rely on downloaded files, then beyond simple text-editing utilties to search for hits you can use command line tools like “findstr” (on Windows) to actally extract matching log entries, which may make it easier to focus on the entries that matter.