yq YAML Processor

yq cheatsheet — jq-style YAML processor. yq '.key' file.yaml, yq -i '.key = value' in-place edit, yq -o json convert to JSON, yq merge. Query and transform YAML files.

8 min read

What it is

yq is a portable command-line YAML processor that works like jq but for YAML, allowing you to query, modify, and convert YAML files.

Installation

Linux

# Using snap (recommended)
sudo snap install yq

# Using Homebrew (if you have it installed)
brew install yq

# Download binary (check releases page for latest version)
wget https://github.com/mikefarah/yq/releases/download/v4.30.5/yq_linux_amd64
chmod +x yq_linux_amd64
sudo mv yq_linux_amd64 /usr/local/bin/yq

macOS

# Using Homebrew (recommended)
brew install yq

Windows

# Using Chocolatey
choco install yq

# Using Scoop
scoop install yq

# Download binary (check releases page for latest version)
# Navigate to the downloaded directory in PowerShell
Invoke-WebRequest -Uri https://github.com/mikefarah/yq/releases/download/v4.30.5/yq_windows_amd64.exe -OutFile yq.exe
# Move to PATH
Move-Item .\yq.exe "$env:ProgramFiles\yq.exe"

Core Concepts

  • YAML Path Expressions: Similar to jq’s JSON paths, yq uses path expressions to navigate and select elements within a YAML document. These paths can be simple keys, array indices, or more complex selectors.
  • In-place Editing: yq can modify files directly using the -i flag.
  • Multiple Documents: YAML files can contain multiple documents separated by ---. yq can operate on all documents or specific ones.
  • Data Types: yq preserves YAML data types (strings, numbers, booleans, nulls, arrays, maps) and handles their conversion when necessary.

Commands / Usage

Reading and Querying YAML

  • Print the entire YAML document:

    yq '.' config.yaml

    Prints the whole config.yaml file.

  • Select a top-level key:

    yq '.database.port' config.yaml

    Prints the value of the port key under database.

  • Select a nested key:

    yq '.users[0].name' users.yaml

    Prints the name of the first user in the users array.

  • Select from an array:

    yq '.servers[1]' servers.yaml

    Prints the second server object from the servers array.

  • Select all elements of an array:

    yq '.servers[].ip' servers.yaml

    Prints the ip address of each server in the servers array.

  • Select keys from array elements:

    yq '.users.[].address.city' users.yaml

    Prints the city from the address of each user.

  • Select multiple fields:

    yq '.name, .version' package.yaml

    Prints the values of name and version as a YAML array.

  • Select a map of fields:

    yq '{name: .name, version: .version}' package.yaml

    Prints a new YAML object containing name and version.

  • Filter array elements:

    yq '.users[] | select(.age > 30)' users.yaml

    Prints users whose age is greater than 30.

  • Accessing values in a map where keys have special characters:

    yq '."special-key"' config.yaml

    Accesses a key named "special-key".

  • Accessing keys that are numbers (as strings):

    yq '.["123"].value' data.yaml

    Accesses the value associated with the key "123".

Modifying YAML

  • Update a value:

    yq '.database.port = 5433' config.yaml

    Prints the config.yaml with the database port changed to 5433.

  • Update a nested value:

    yq '.users[0].active = false' users.yaml

    Prints the users.yaml with the first user’s active status set to false.

  • Add a new key-value pair:

    yq '.database.user = "admin"' config.yaml

    Prints config.yaml with a new user key and value added to database.

  • Add an element to an array:

    yq '.servers += {"ip": "192.168.1.100", "role": "backup"}' servers.yaml

    Appends a new server object to the servers array.

  • Insert an element at a specific index:

    yq '.servers[1] = {"ip": "192.168.1.50", "role": "staging"}' servers.yaml

    Replaces the element at index 1 with a new server object.

  • Delete a key:

    yq 'del(.database.password)' config.yaml

    Prints config.yaml with the password key removed from database.

  • Delete an array element by index:

    yq 'del(.servers[0])' servers.yaml

    Prints servers.yaml with the first server removed.

  • Delete array elements by condition:

    yq 'del(.users[] | select(.age < 18))' users.yaml

    Prints users.yaml with all users younger than 18 removed.

Converting Formats

  • Convert YAML to JSON:

    yq -o json '.' config.yaml

    Prints the config.yaml content as JSON.

  • Convert JSON to YAML:

    yq -p json '.' config.json

    Prints the config.json content as YAML.

  • Convert TOML to YAML:

    yq -p toml '.' config.toml

    Prints the config.toml content as YAML.

Working with Multiple Documents

  • Select a specific document (0-indexed):

    yq --doc 1 '.key' multi_doc.yaml

    Prints the value of key from the second document in multi_doc.yaml.

  • Operate on all documents:

    yq --doc '*' '.key |= . + "_suffix"' multi_doc.yaml

    Appends _suffix to the value of key in all documents.

  • Merge multiple YAML files:

    yq eval-all --inplace '. as $item ireduce ({}; . * $item)' file1.yaml file2.yaml file3.yaml

    Merges file1.yaml, file2.yaml, and file3.yaml into file1.yaml, with later files overwriting earlier ones.

In-place Editing

  • Modify a file directly:

    yq -i '.database.port = 5433' config.yaml

    Updates config.yaml in place with the new database port.

  • Modify multiple files in place:

    yq -i '.version = "1.1.0"' *.yaml

    Updates the version key in all .yaml files in the current directory.

Advanced Operations

  • Create a new YAML structure:

    yq -n '{name: "example", version: "1.0.0"}'

    Creates a new YAML structure.

  • Concatenate strings:

    yq '.message = (.greeting + " " + .name)' data.yaml

    Concatenates greeting and name fields into a new message field.

  • Iterate and transform:

    yq '.users[] |= .name = (.name | ascii_upcase)' users.yaml

    Converts all user names to uppercase.

  • Conditional assignment:

    yq '.users[] |= if .age > 18 then .status = "adult" else .status = "minor" end' users.yaml

    Assigns a status based on age.

  • Using variables:

    NEW_PORT=5433 yq '.database.port = env(NEW_PORT)' config.yaml

    Sets the database port using an environment variable.

  • Read from stdin, write to stdout:

    cat config.yaml | yq '.database.host'

    Pipes the content of config.yaml to yq to extract the database host.

  • Read from stdin, modify, write to stdout:

    cat config.yaml | yq '.database.port |= . + 1'

    Increments the database port read from stdin.

  • Read from stdin, modify in place (requires a temp file or careful piping):

    # Example: update a value and save back to the original file
yq -i '.database.port = 5433' config.yaml

    (Note: Direct in-place modification from stdin to a file isn’t a standard yq operation without redirection. The -i flag operates on the specified file path.)

Common Patterns

  • Extracting a specific value from multiple files:

    yq '.version' file1.yaml file2.yaml file3.yaml

    Prints the version from each file, outputting them sequentially.

  • Updating a common value across multiple files:

    yq -i '.environment = "production"' *.yaml

    Sets the environment to production in all YAML files in the current directory.

  • Converting a YAML file to JSON and saving:

    yq -o json '.' config.yaml > config.json

    Converts config.yaml to JSON and saves it to config.json.

  • Merging two YAML files with precedence:

    yq eval '. as $item ireduce ({}; . * $item)' base.yaml overrides.yaml

    Merges overrides.yaml into base.yaml, with values from overrides.yaml taking precedence.

  • Creating a new YAML file from scratch:

    yq -n '{
    apiVersion: "v1",
    kind: "Pod",
    metadata: {name: "my-pod"},
    spec: {containers: [{name: "nginx", image: "nginx:latest"}]}
}' > pod.yaml

    Creates a pod.yaml file with the specified content.

  • Extracting data and formatting as a string:

    yq -r '.users[] | "\(.name) is \(.age) years old"' users.yaml

    Outputs a human-readable string for each user. The -r flag outputs raw strings, not YAML.

  • Iterating over maps and modifying values:

    yq '.config.settings |= with_entries(if .key | test("timeout") then .value |= . * 1000 else . end)' config.yaml

    Multiplies any setting value whose key contains "timeout" by 1000.

Gotchas

  • YAML vs JSON Syntax: While yq can process JSON, remember that YAML has nuances like implicit typing, anchors, and aliases that jq doesn’t handle. yq aims to respect these YAML features.
  • In-place Editing (-i): Be cautious with -i. It modifies the file directly. Always have backups or use version control when performing extensive in-place modifications.
  • Multiple Documents: When operating on files with multiple YAML documents, be mindful of which document you are targeting with --doc. If omitted, operations might apply to the first document or behave unexpectedly depending on the command. eval-all is often safer for multi-document operations.
  • Path Expression Syntax: While similar to jq, yq’s path expressions can have subtle differences, especially when dealing with complex YAML structures or specific operators. Always test your expressions.
  • Operator Precedence: Complex expressions involving multiple operators might require careful consideration of precedence or the use of parentheses for clarity.
  • eval vs eval-all: eval processes the input as a single YAML document (or the first document if multiple exist). eval-all processes all YAML documents within the input. Use eval-all when you need to apply an operation consistently across all documents in a file.
  • Quoting: Shell expansion can interfere with yq’s path expressions. Always quote your yq expressions to prevent the shell from interpreting special characters. For example, use yq '.users[0].name' instead of yq .users[0].name.
  • Empty Values: Be aware of how yq handles null values or empty strings, especially during updates or comparisons.