Stern Multi-Pod Logs

What it is

Stern is a Kubernetes multi-pod and multi-container log tailing utility that allows you to view logs from multiple pods and containers simultaneously, filter them, and highlight specific content.

Installation

Linux / macOS (using Homebrew):

brew install stern

Linux / macOS (downloading binary):

1. Visit the Stern releases page.
2. Download the appropriate binary for your OS and architecture (e.g., stern_1.11.0_linux_amd64.tar.gz).
3. Extract the archive: tar -zxvf stern_1.11.0_linux_amd64.tar.gz
4. Move the binary to your PATH: sudo mv stern /usr/local/bin/

Windows (using Scoop):

scoop install stern

Windows (downloading binary):

1. Visit the Stern releases page.
2. Download the appropriate binary for your OS and architecture (e.g., stern_1.11.0_windows_amd64.zip).
3. Extract the archive.
4. Add the directory containing stern.exe to your system’s PATH environment variable.

Core Concepts

• Pod: The smallest deployable unit in Kubernetes, representing a group of one or more containers.
• Container: A running instance of a Docker image. A pod can have multiple containers.
• Selector: A label selector used to identify which pods to tail logs from.
• Namespace: A virtual cluster within a physical cluster, used to organize resources.

Commands / Usage

Tail Logs from Specific Pods

Tail logs from all pods matching a label selector:

stern -l app=my-app

This tails logs from all pods in the current namespace that have the label app=my-app.

Tail logs from pods in a specific namespace:

stern -n staging -l app=my-app

This tails logs from all pods with the label app=my-app in the staging namespace.

Tail logs from a specific pod:

stern my-app-pod-12345

This tails logs from the pod named my-app-pod-12345 in the current namespace.

Tail logs from pods matching a regex:

stern '^my-app-.*-xyz$'

This tails logs from pods whose names match the regex ^my-app-.*-xyz$.

Tail Logs from Multiple Containers in Pods

Tail logs from all containers in pods matching a label selector:

stern -l app=my-app --all-containers

This tails logs from all containers within each pod that matches the label app=my-app.

Tail logs from specific containers by name:

stern -l app=my-app --container web --container worker

This tails logs from the web and worker containers in pods matching app=my-app.

Tail logs from a single container by name if multiple exist:

stern -l app=my-app --container main-app

This tails logs from the container named main-app in pods matching app=my-app.

Filtering and Highlighting Logs

Highlight lines containing "error" in yellow:

stern -l app=my-app --color yellow --show-loki

This tails logs from pods with app=my-app and highlights lines containing "error" in yellow. --show-loki is often used with highlighting to ensure log lines are displayed.

Color lines containing "warning" red and "error" yellow:

stern -l app=my-app --color-regex "warning:red,error:yellow"

This tails logs and applies specific colors to lines matching the provided regex patterns.

Filter out lines containing "debug":

stern -l app=my-app --exclude 'debug'

This tails logs from pods with app=my-app but omits any lines that contain the word "debug".

Show only lines containing "success":

stern -l app=my-app --include 'success'

This tails logs from pods with app=my-app and only displays lines that contain the word "success".

Controlling Output

Show timestamps:

stern -l app=my-app --timestamp

This adds a timestamp to each log line.

Tail logs and follow new entries (like tail -f):

stern -l app=my-app --follow

This tails existing logs and then continues to stream new log entries as they appear.

Limit the number of lines to show:

stern -l app=my-app --lines 100

This tails the last 100 log lines from matching pods.

Tail logs from the end of the log file:

stern -l app=my-app --tail 50

This tails the last 50 log lines from matching pods, similar to --lines but often implies starting from the end.

Disable color output:

stern -l app=my-app --no-color

This disables all color highlighting and formatting.

Advanced Usage

Tail logs from multiple namespaces:

stern -n default,staging -l app=my-app

This tails logs from pods matching app=my-app in both the default and staging namespaces.

Tail logs from all namespaces:

stern --all-namespaces -l app=my-app

This tails logs from pods matching app=my-app across all available namespaces.

Specify a context:

stern --context dev-cluster -l app=my-app

This tails logs from pods in the dev-cluster Kubernetes context.

Tail logs and output in JSON format:

stern -l app=my-app --json

This tails logs and formats each entry as a JSON object.

Use a custom template for output:

{% raw %}
stern -l app=my-app --template '{{.PodName}} {{.ContainerName}} {{.Message}}'
{% endraw %}

This allows you to define a custom Go template for how each log line is displayed.

Tail logs and only show pod names:

stern -l app=my-app -o jsonpath='{.items[*].metadata.name}'

This example uses jsonpath to extract just the pod names from the Kubernetes API response, not directly log content. For log content, --template is more appropriate.

Tail logs and only show container names:

{% raw %}
stern -l app=my-app --template '{{.ContainerName}}'
{% endraw %}

This tails logs and displays only the name of the container from which the log originated.

Show pod and container names by default:

stern -l app=my-app --show-pod-name --show-container-name

This ensures that the pod and container names are prefixed to each log line.

Tail logs from pods that do NOT match a label:

stern -l app!=my-app

This tails logs from pods that do not have the label app=my-app.

Tail logs from pods matching multiple labels:

stern -l app=my-app,env=production

This tails logs from pods that have both the app=my-app and env=production labels.

Tail logs from pods matching any of several labels:

stern -l app=my-app -l env=staging

This tails logs from pods that have either the app=my-app label OR the env=staging label.

Common Patterns

Tail logs from all pods in the current namespace and follow new entries:

stern -f '.'

The . is a shorthand for selecting all pods in the current namespace. -f is equivalent to --follow.

Tail logs from a specific deployment and follow:

stern -l app=my-deployment --follow

Assumes your deployment has a label like app=my-deployment.

Tail logs from multiple applications and highlight errors:

stern -l app=frontend -l app=backend --color-regex "error:red"

This tails logs from pods labeled app=frontend OR app=backend and highlights lines with "error" in red.

Tail logs from a specific pod and container, and limit to the last 50 lines:

stern my-specific-pod --container main-app --lines 50

Tail logs from all pods in a namespace, showing timestamps and filtering for "INFO":

stern -n my-namespace -l app=my-service --timestamp --include 'INFO'

Tail logs from all pods in all namespaces, showing pod and container names, and highlighting warnings:

stern --all-namespaces -l app=critical-service --show-pod-name --show-container-name --color-regex "WARNING:yellow"

Gotchas

• Selector Precedence: When using multiple label selectors (e.g., -l app=a -l app=b), Stern defaults to an OR logic. To achieve AND logic, use comma separation within a single -l flag (e.g., -l app=a,env=prod).
• Default Namespace: If no namespace is specified, Stern uses the namespace configured in your current Kubernetes context.
• Wildcard vs. Regex: Stern supports both simple wildcards (like *) and regular expressions for pod name matching. Be mindful of which you are using, as their behavior differs. Regex is the default for pod name patterns.
• --all-containers vs. --container: If you specify --container, it will only show logs from that specific container. If a pod has multiple containers and you don’t specify --container and don’t specify --all-containers, Stern might default to showing logs from only one container (often the first one it finds) or prompt you to choose. Use --all-containers to be explicit about seeing logs from every container in matching pods.
• --follow and Initial Output: When using --follow, Stern first tails the existing logs and then switches to following new entries. If you only want to see new entries from the moment you run the command, you might need to combine it with --tail 0 or similar, though --follow typically handles this well by default.
• Performance with Many Pods: Tailing logs from a very large number of pods or high-volume log streams can consume significant resources on your local machine. Use selectors and filters effectively to narrow down the output.
• Log Rotation: Stern tails logs as they are written by the container runtime. If logs are rotated aggressively by the container runtime or the logging agent, Stern might miss some entries during the rotation window.