How do I make sure a kubectl command meant for dev can never hit prod?

If you manage more than two Kubernetes clusters, you've probably lived this exact nightmare:

You open Tab 1 to debug an issue in production. Then you open Tab 2 to test a quick manifest fix in dev. You run kubectl config use-context dev-cluster in Tab 2, switch back to Tab 1, and run kubectl delete pod ... thinking you're in dev.

Except you aren't.

Why this happens

Because standard kubectl mutates a global configuration file (~/.kube/config), Tab 1 silently pointed to dev too. Or worse: you switched context in Tab 2 back to prod, and accidentally nuked a production workload from Tab 1.

The root cause is that context is global state. Every terminal on the machine reads and writes the same file, so "current context" is a single shared variable pretending to be per-terminal. Throw private GKE clusters, Google IAP (Identity-Aware Proxy) SSH bastions, and proxy ports into the mix, and managing terminal sessions quickly turns into administrative chaos.

Here's how I fixed it permanently by building a fully automated, context-isolated workflow using Ghostty, Zsh, IAP tunnels, and tmux.

The architecture

To make multi-cluster ops safe and effortless, the setup needed to hit four strict criteria:

1. Zero context leakage: switching clusters in one terminal tab must never affect any other tab.
2. Automated SSH IAP tunnels: no manually running long gcloud compute ssh commands to open background proxies.
3. Dedicated tmux sessions per cluster: typing an alias like k8s-dev or k8s-prod should automatically create (or switch to) a dedicated tmux session named after that cluster.
4. Seamless multi-tab usability: opening new tabs inside a tmux session must automatically inherit that specific cluster's proxy without typing extra commands.

The key idea: instead of a global "current context," each cluster lives in its own tmux session with its own HTTPS_PROXY scoped to that session. The state stops being global.

Step 1: the magic Zsh bastion helper

Instead of managing separate bash scripts or copy-pasting gcloud flags, I wrote a single Zsh helper function: _connect_gke_bastion.

It handles everything in one go:

• Fetches fresh GKE credentials.
• Renames long default GKE context strings (like gke_acme-dev-gcp_us-central1-a_dev-app-cluster) into clean short aliases (k8s-dev).
• Spawns a background SSH IAP tunnel mapping a unique local port on my Mac to the remote bastion's proxy port (8888 or 8889).
• Registers the HTTPS_PROXY variable strictly inside a named tmux session.
• Creates or switches to that tmux session instantly.

Add this function to your ~/.zshrc:

# -----------------------------
# Core Helper Function for GKE + IAP + Automated Tmux
# -----------------------------
_connect_gke_bastion() {
  local proj="$1"
  local cluster="$2"
  local loc_type="$3"
  local loc_val="$4"
  local bastion="$5"
  local local_port="$6"
  local remote_port="$7"
  local alias_name="$8"
  local extra_flags="$9"

  echo "Fetching GKE Credentials for ${cluster}..."
  if [ -n "${extra_flags}" ]; then
    gcloud container clusters get-credentials "${cluster}" "${loc_type}=${loc_val}" --project="${proj}" "${extra_flags}"
  else
    gcloud container clusters get-credentials "${cluster}" "${loc_type}=${loc_val}" --project="${proj}"
  fi

  # Rename long GKE context to clean short alias for prompt engines
  if [ -n "$alias_name" ]; then
    local current_ctx
    current_ctx=$(kubectl config current-context 2>/dev/null)
    if [ -n "$current_ctx" ] && [ "$current_ctx" != "$alias_name" ]; then
      kubectl config rename-context "$current_ctx" "$alias_name" 2>/dev/null
      echo "Cleaned context name to: ${alias_name}"
    fi
  fi

  # Check if SSH tunnel is already open on this local port
  if ps aux | grep -E 'gcloud compute ssh|ssh' | grep -q "${local_port}:localhost:${remote_port}"; then
    echo "⚠️ Tunnel via ${bastion} is already running on local port ${local_port}."
  else
    echo "Opening SSH IAP Tunnel: Mac port ${local_port} -> Bastion port ${remote_port}..."
    gcloud compute ssh "${bastion}" \
      --zone "us-central1-a" \
      --project "${proj}" \
      --tunnel-through-iap \
      --ssh-flag="-4 -L ${local_port}:localhost:${remote_port} -N -q -f"
  fi

  export HTTPS_PROXY="localhost:${local_port}"
  echo "✅ HTTPS_PROXY set to localhost:${local_port}"

  # Automated Tmux Session Management
  if [ -n "$alias_name" ]; then
    # 1. Create a named tmux session if it doesn't exist
    tmux has-session -t "$alias_name" 2>/dev/null || tmux new-session -d -s "$alias_name"

    # 2. Scope the proxy strictly to this tmux session
    tmux set-environment -t "$alias_name" HTTPS_PROXY "localhost:${local_port}"

    # 3. Update the currently active window in that session
    tmux send-keys -t "$alias_name" "export HTTPS_PROXY=localhost:${local_port}" C-m 2>/dev/null

    # 4. Attach or switch to the session
    if [ -n "$TMUX" ]; then
      echo "⚡ Switching tmux session to: [${alias_name}]"
      tmux switch-client -t "$alias_name"
    else
      echo "⚡ Attaching to tmux session: [${alias_name}]"
      tmux attach-session -t "$alias_name"
    fi
  fi
}

# Auto-inherit active HTTPS_PROXY when opening new tabs inside tmux
[ -n "$TMUX" ] && eval $(tmux show-environment -s HTTPS_PROXY 2>/dev/null)

Step 2: mapping clean cluster aliases

With the helper function in place, defining a new cluster connection becomes a simple one-line alias. Each environment gets its own unique local port so tunnels never collide:

# -----------------------------
# Work Project Aliases (Dummy Examples)
# -----------------------------
alias k8s-dev='_connect_gke_bastion "acme-dev-gcp" "dev-app-cluster" "--zone" "us-central1-a" "dev-bastion" "8891" "8888" "k8s-dev" "--internal-ip"'
alias k8s-staging='_connect_gke_bastion "acme-staging-gcp" "staging-app-cluster" "--region" "us-central1" "staging-bastion" "8882" "8889" "k8s-staging"'
alias k8s-prod='_connect_gke_bastion "acme-prod-gcp" "prod-app-cluster" "--region" "us-central1" "prod-bastion" "8883" "8889" "k8s-prod" "--internal-ip"'
alias k8s-integrations='_connect_gke_bastion "acme-integrations-gcp" "integrations-cluster" "--zone" "us-central1-a" "integrations-bastion" "8885" "8888" "k8s-integrations"'

Step 3: making tmux feel natural (~/.tmux.conf)

By default, tmux can feel clunky if mouse scrolling and color support aren't configured properly.

Here is my minimal ~/.tmux.conf that turns on trackpad scrolling, fixes true-color rendering for modern terminal emulators like Ghostty, renumbers tabs automatically when one is closed, and rebinds the prefix key to Ctrl-a:

# --- Prefix Configuration ---
unbind C-b
set -g prefix C-a
bind C-a send-prefix

# --- Enable Mouse & Trackpad Scrolling ---
set -g mouse on
set -g history-limit 50000

# --- Auto-Renumber Windows Sequentially ---
set -g renumber-windows on

# --- Fix True Color Support ---
set -g default-terminal "xterm-256color"
set -ga terminal-overrides ",xterm-256color:Tc"

# --- Start Window Indexing at 1 ---
set -g base-index 1
setw -g pane-base-index 1

# --- Intuitive Pane Splitting ---
bind | split-window -h -c "#{pane_current_path}"
bind - split-window -v -c "#{pane_current_path}"

Step 4: quality-of-life productivity aliases

To round out the setup, I added a few essential shortcuts at the end of my ~/.zshrc:

# 1. Quick Tab Creation inside Tmux
alias tn="tmux new-window"

# 2. Kubectl Autocomplete & Shortcut
alias k="kubectl"
if command -v kubectl &>/dev/null; then
  source <(kubectl completion zsh)
  compdef __start_kubectl k
fi

# 3. Emergency Cleanup (Kills stuck background tunnels)
alias kill-bastion="ps aux | grep -E 'gcloud compute ssh|ssh.*localhost' | grep -v grep | awk '{print \$2}' | xargs kill -9 2>/dev/null && echo '✅ All background IAP tunnels killed.'"

The daily workflow

Now, working across multiple clusters is completely frictionless:

1. Launch a cluster session: type k8s-dev. The terminal opens an IAP tunnel, fetches credentials, creates a tmux session named k8s-dev, and drops you in.
2. Open multiple tabs: type tn (or press Ctrl-a c) to open new windows for k9s, stern logs, or helm deploys. Every new tab inherits HTTPS_PROXY=localhost:8891 automatically.
3. Switch to production: type k8s-prod. You are instantly moved to a new k8s-prod tmux session running on port 8883.
4. Jump between environments: press Ctrl-a then s to open an interactive visual list of all active cluster sessions and flip between dev, staging, and production in milliseconds.

What I understood

• The bug was never kubectl — it was global state. A single ~/.kube/config shared by every terminal makes "current context" a global variable masquerading as a local one. The fix isn't discipline; it's removing the shared state.
• Isolation belongs at the session boundary. Scoping HTTPS_PROXY to a named tmux session gives each cluster its own little world — and tabs opened inside it inherit that world for free.
• Automation is a safety feature, not just convenience. Every manual gcloud flag is a chance to point at the wrong project. Collapsing it into one alias removes a whole class of human error.

No global state pollution, no manual SSH port-forwarding, and zero chance of running a dev command in production.