Install the On-Prem Worker
BigPanda's L1 Agent On-Prem Worker is a lightweight containerized agent that runs on your infrastructure to execute runbook automation steps. It connects to BigPanda using mutual TLS (mTLS) authentication and executes pre-registered workflows on demand.
This guide covers a first install: download the image, configure it to route through your corporate HTTP CONNECT proxy, run it, and confirm it reaches a healthy state. Workflow-specific configuration is added in a follow-up step coordinated with BigPanda.
For architecture, security, and FAQs, see the On-Prem Worker parent page.
Supported versions
These instructions apply to worker v13.9.0 and later. New installs use the latest version; earlier versions can no longer be installed.
Worker version | Status | Install guide | Changes |
|---|---|---|---|
v13.10.0 | Latest. Use for new installs. | Adds SSH targets ( | |
v13.9.0 | Supported | Adds the | |
v13.8.0 | Existing deployments only, closed to new installs | Not available. See Updating to a new version. | No configuration changes documented. |
v13.7.2 | Existing deployments only, closed to new installs | Not available. See Updating to a new version. | Adds the |
v13.6.3 | Existing deployments only, closed to new installs | Not available. See Updating to a new version. | Adds HTTP CONNECT proxy support ( |
To check the version you are running:
docker images | grep adr-temporal-worker
v13.9.0 and later
The worker ships as a Docker container image. BigPanda provides a presigned download URL during onboarding.
Install the worker
Before you start
Make sure you have the following ready:
Docker installed on the target host (Linux x86_64)
The mTLS certificate bundle from BigPanda (provided during onboarding): your client certificate (
tls.crt), its private key (tls.key), and BigPanda's CA certificate (ca.crt).A corporate HTTP CONNECT proxy allowlisted for CONNECT
temporal.bigpanda.io:7233, with TLS inspection (SSL decryption) disabled for that destination. TLS-inspecting proxies break the mTLS chain. Major enterprise proxies (Zscaler, Palo Alto, Squid, Forcepoint, BlueCoat, Cloudflare Gateway) support per-destination decryption exclusions.Outbound network access from the host to your proxy, and from the proxy to
temporal.bigpanda.io:7233.
Deploy the worker with Docker
Download and load the image
BigPanda shares a presigned S3 URL (valid for 24 hours) for the version tag you are installing. Download the tarball, replacing <version> with that tag (for example, v13.10.0):
curl -L '<URL>' -o adr-temporal-worker-<version>.tar.gz
If your host reaches S3 through the corporate proxy, prefix with HTTPS_PROXY:
HTTPS_PROXY=http://<host>:<port> \ curl -L '<URL>' -o adr-temporal-worker-<version>.tar.gz
Expected file size is approximately <SIZE> MB (<BYTES> bytes). Verify, then load:
ls -la adr-temporal-worker-<version>.tar.gz docker load -i adr-temporal-worker-<version>.tar.gz
Expected output: Loaded image: adr-temporal-worker:latest. Confirm:
docker images | grep adr-temporal-worker
Install certificates
Place the three files BigPanda sent into a certs/ directory next to where you will create the .env file:
certs/ ├── tls.crt # Client certificate (identifies your organization) ├── tls.key # Client private key (keep secure, do not share) └── ca.crt # BigPanda CA certificate (validates the Temporal server)
Create a configuration file
Create .env in the same directory as certs/:
# --- Connection (provided by BigPanda) --- TEMPORAL_ADDRESS=temporal.bigpanda.io:7233 TEMPORAL_TLS_CERT_PATH=/certs/tls.crt TEMPORAL_TLS_KEY_PATH=/certs/tls.key TEMPORAL_TLS_CA_PATH=/certs/ca.crt # --- Corporate egress proxy (required — provided by your network team) --- # Format: http://[user:pass@]host:port (HTTP CONNECT only) TEMPORAL_PROXY_URL=http://<host>:<port> # With basic auth (URL-encode '@' as %40, ':' as %3A in user/pass): # TEMPORAL_PROXY_URL=http://<user>:<pass>@<host>:<port> # --- HTTP integrations (provided by BigPanda during onboarding) --- # JSON array of the HTTP endpoints the worker may call. Each integration's # credentials are supplied via its own secret_env variable, set separately. ADR_HTTP_INTEGRATIONS=[ ... see Configuration reference ... ] # --- SSH targets (v13.10.0 and later; provided by BigPanda during onboarding) --- # JSON array of the hosts the worker may run commands on. Each target's private # key is supplied via its own private_key_env variable, set separately. ADR_SSH_TARGETS=[ ... see Configuration reference ... ] # --- Operational (optional) --- LOG_LEVEL=info METRICS_PORT=8080
Run the worker
docker run -d --name adr-temporal-worker \ --env-file .env \ -v $(pwd)/certs:/certs:ro \ --restart unless-stopped \ adr-temporal-worker:latest
Verify the deployment
Confirm the container is running and healthy:
docker ps | grep adr-temporal-worker docker logs --tail 50 adr-temporal-worker
GET http://localhost:8080/health → {"status":"ok","startedAt":"..."}
GET http://localhost:8080/ready → {"status":"ready"}Both endpoints return 200 once the process is up, independently of how many integrations are configured. The image is node:*-slim: it ships node but not curl or wget, so a container health check must use a node one-liner:
node -e "require('http').get('http://localhost:8080/health',r=>process.exit(r.statusCode===200?0:1)).on('error',()=>process.exit(1))"Deploy the worker with Kubernetes
If you are deploying to Kubernetes, use the following manifests as a starting point.
Create a Secret for the certificates:
apiVersion: v1 kind: Secret metadata: name: adr-temporal-worker-certs namespace: bigpanda type: Opaque data: tls.crt: <base64-encoded-cert> tls.key: <base64-encoded-key> ca.crt: <base64-encoded-ca>
Create a Secret for the monitoring API key, for each
secret_envorclient_secret_envnamed inADR_HTTP_INTEGRATIONS, and for eachprivate_key_envnamed inADR_SSH_TARGETS:apiVersion: v1 kind: Secret metadata: name: adr-temporal-worker-secrets namespace: bigpanda type: Opaque stringData: DEVICE_API_KEY: "<your-monitoring-platform-api-key>" EXAMPLE_API_CLIENT_SECRET: "<oauth client secret>" # client_secret_env SSH_KEY_APP_SERVERS: "<private key, base64 on one line>" # private_key_env (v13.10.0 and later)
Create the deployment:
apiVersion: apps/v1 kind: Deployment metadata: name: adr-temporal-worker namespace: bigpanda spec: replicas: 1 selector: matchLabels: app: adr-temporal-worker template: metadata: labels: app: adr-temporal-worker spec: containers: - name: worker image: adr-temporal-worker:latest ports: - containerPort: 8080 envFrom: - secretRef: name: adr-temporal-worker-secrets env: - name: TEMPORAL_ADDRESS value: "temporal.bigpanda.io:7233" - name: TEMPORAL_TLS_CERT_PATH value: "/certs/tls.crt" - name: TEMPORAL_TLS_KEY_PATH value: "/certs/tls.key" - name: TEMPORAL_TLS_CA_PATH value: "/certs/ca.crt" - name: TEMPORAL_PROXY_URL value: "http://<host>:<port>" - name: ADR_HTTP_INTEGRATIONS value: '[ ... see Configuration reference ... ]' - name: ADR_SSH_TARGETS value: '[ ... see Configuration reference ... ]' volumeMounts: - name: certs mountPath: /certs readOnly: true resources: requests: memory: "256Mi" cpu: "100m" limits: memory: "512Mi" cpu: "500m" livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 30 readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 15 volumes: - name: certs secret: secretName: adr-temporal-worker-certs
Configuration reference
Connection
Variable | Description | Value |
|---|---|---|
| BigPanda Temporal Server address | |
| Path in the container to the client certificate | |
| Path in the container to the client private key | |
| Path in the container to the BigPanda CA certificate | |
| JSON array of HTTP integrations the worker may call | See below |
| JSON array of SSH targets the worker may run commands on (v13.10.0 and later) | See below |
Proxy
Proxy data should be provided by your network team.
Variable | Description | Example |
|---|---|---|
| HTTP CONNECT proxy URL. Format | |
Proxy scope
Jira, Confluence, api.bigpanda.io, Itential, and ADR_HTTP_INTEGRATIONS traffic, including OAuth token endpoints, is not routed through TEMPORAL_PROXY_URL. Contact BigPanda support if that traffic must also use a proxy.
Optional
Variable | Default | Description |
|---|---|---|
| | Log level (debug, info, warn, error) |
| | Port for the health and ready endpoints |
| — | CA bundle for internally signed APIs or token endpoints |
| — |
|
ADR_HTTP_INTEGRATIONS
Variable | Required | Description |
|---|---|---|
ADR_HTTP_INTEGRATIONS | No | JSON array of HTTP integrations. See the fields and authentication types below. |
Targets of | Conditional | One variable per credential named in the array |
ADR_HTTP_INTEGRATIONS is a JSON array of up to 50 entries, one per API the worker may call on behalf of runbooks. Adding an integration is configuration only; no software update is needed.
Credentials never go in the JSON
Every auth block names an environment variable (secret_env, or client_secret_env / private_key_env for oauth2) that holds the credential. Set those variables through your runtime's secret mechanism. Query-string secrets are masked in worker debug output.
A bad entry costs only that entry
Each element is validated individually at startup. Invalid entries are dropped and logged at error as http integration rejected with the reason; the rest keep working. A runbook step that names a dropped integration fails with unknown integration '<name>'.
Fields
Field | Required | Description |
|---|---|---|
| Yes | Unique lowercase ID, |
| Yes | Tool kind, free text. For example |
| Yes | Absolute |
| No | Free text that helps runbooks target the right API. For example, Jira |
| No | One line describing this instance. |
| Yes | Credential definition. See Authentication types. |
| No | Restricts callable paths, for example |
| No | Non-empty subset of |
| No |
|
| No | Constant, non-secret headers sent on every request. Reserved names are refused: |
| No | Per-integration request timeout. Default and maximum: |
Authentication types
Choose by how the target API expects credentials.
Your API expects | auth.type | Fields | Example |
|---|---|---|---|
| | | Grafana |
|
|
| Dynatrace |
| |
| Jira Cloud, ServiceNow |
One or more custom secret headers | |
| Datadog |
| | Single | Linear |
Token in the query string | |
| Itential static token |
Session cookie | |
| — |
Client ID + secret or key, and a token URL | | See oauth2 | Any client-credentials provider |
Nothing | | — | Public APIs |
oauth2
Only the client_credentials grant is supported. Exactly one of client_secret_env (with client_auth post or basic) or private_key_env (with private_key_jwt) is required.
Field | Required | Description |
|---|---|---|
| Yes | |
| Yes | Absolute |
| Yes | Not a secret |
| Yes, unless private_key_jwt | Variable holding the client secret |
| No |
|
| No | As the provider requires |
| private_key_jwt only | Variable holding an unencrypted PEM, or its base64 on one line |
| No |
|
| No | JWT |
Token lifecycle
Tokens are cached in memory per process and renewed 30 seconds before expiry (halfway through for tokens shorter than 60 seconds). A missing expires_in is treated as 3600 seconds. The token request may take up to 10 seconds of the call's budget. A 401 on a token older than 60 seconds results in the call failing without retry. Credentials are read at startup: to rotate, update the variable and restart the worker.
Example
ADR_HTTP_INTEGRATIONS=[
{
"name": "grafana-eu-1",
"type": "grafana",
"base_url": "https://grafana-eu.example.internal",
"api_version": "10.x",
"auth": { "type": "bearer", "secret_env": "GRAFANA_EU_1_TOKEN" },
"allowed_path_prefixes": ["/api/"]
},
{
"name": "servicenow-prod",
"type": "servicenow",
"base_url": "https://acme.service-now.com",
"auth": { "type": "basic", "secret_env": "SNOW_BASIC" },
"allowed_methods": ["GET"],
"extra_headers": { "X-Tenant-Id": "acme-prod" }
},
{
"name": "example-api",
"type": "http",
"base_url": "https://api.example.internal",
"auth": {
"type": "oauth2",
"grant_type": "client_credentials",
"token_url": "https://auth.example.internal/oauth/token",
"client_id": "svc-bigpanda",
"client_secret_env": "EXAMPLE_API_CLIENT_SECRET"
}
}
]One line under --env-file
docker run --env-file does not support multi-line values. Collapse the array to one line, or use docker compose or Kubernetes, where multi-line values work.
ADR_SSH_TARGETS
v13.10.0 and later.
A JSON array with one object per group of hosts the worker may run commands on. Per-entry fields: name (unique identifier), hosts (the machines this target may reach), optional description and port, username (the account the worker authenticates as), private_key_env, host_key_sha256 (the SHA256 fingerprint of each host's SSH host key), allowed_commands (the executables that may run), and allowed_path_prefixes (the directories their arguments may address). Optional max_output_bytes and timeout_ms default to 262144 and 30000.
private_key_env names an environment variable holding the actual key: an OpenSSH private key, or that key base64-encoded onto one line, defined alongside the worker's other environment variables or injected from your secret store. Never put a raw key in ADR_SSH_TARGETS itself.
Every host must have a host_key_sha256 entry. Collect each one with ssh-keyscan -t ed25519 <host> | ssh-keygen -lf -. A command is checked against allowed_commands and allowed_path_prefixes before any connection is opened, so anything outside them never reaches the host.
ADR_SSH_TARGETS=[
{
"name": "app-servers",
"description": "Application servers - log reads",
"hosts": ["host01.example.com", "host02.example.com"],
"port": 22,
"username": "svc_bigpanda",
"private_key_env": "SSH_KEY_APP_SERVERS",
"host_key_sha256": {
"host01.example.com": "SHA256:6oPWAdTwUzje1PKJ21yzq/09O+CT71Gm/1ze8GzXx/k",
"host02.example.com": "SHA256:kJ2hQ1vZ8nR4xT7wY0mB5cF3dG6sL9pA2eN1uI8oK4M"
},
"allowed_commands": ["id", "ls", "cat", "tail", "head"],
"allowed_path_prefixes": ["/var/log/app/"]
}
]Itential
Variable | Required | Description |
|---|---|---|
ITENTIAL_URL | Conditional | Operations Manager base URL |
ITENTIAL_API_CLIENT_ID, ITENTIAL_API_CLIENT_SECRET | Conditional | OAuth service-account credentials. The worker refreshes the token itself. |
ITENTIAL_API_TOKEN | No | Legacy static token. Use only where OAuth is unavailable. |
ITENTIAL_SKIP_TLS_VERIFY | No |
|
Fetch and push
Variable | Required | Description |
|---|---|---|
JIRA_AUTH | For Jira projects | Jira API token (Cloud) or PAT (Data Center) with read access to fetched projects |
CONFLUENCE_AUTH | For Confluence spaces | Confluence API token or PAT with read access to target spaces |
UDC_API_KEY | For UDC connections | BigPanda API key (Integrations > API > Create API Key) for pushing to |
Troubleshooting
Symptom | Cause | Fix |
|---|---|---|
Error message: TEMPORAL_PROXY_URL must be a valid URL | Malformed value | Format is http://[user:pass@]host:port — port required |
Error message: TEMPORAL_PROXY_URL scheme must be 'http' | Used https:// | Use http://. The proxy hop is HTTP CONNECT; tunneled traffic is still TLS end-to-end |
Error message: TEMPORAL_PROXY_URL must include a port | No port specified | Specify the port (for example |
Error message: username but no password (or vice versa) | Half the credentials | Provide both or neither |
Error message: username may not contain ':' or '@' | Reserved character in credential | URL-encode it |
Error message: TLS cert at /certs/tls.crt has no CN in subject | Wrong cert file at that path | Verify the file BigPanda sent is at the path the env var points to |
Error message: ENOENT: no such file or directory ... /certs/tls.crt | Cert files not mounted | Verify |
Routing… proxy but never reaches RUNNING; eventually DEADLINE_EXCEEDED | Proxy not allowlisted for | Confirm the destination is allowed CONNECT pass-through with no TLS decryption |
TLS handshake failed / unable to verify the first certificate / self-signed certificate in chain | Missing or incorrect TEMPORAL_TLS_CA_PATH, or proxy intercepting TLS | Verify |
Failed to connect to proxy | Proxy host/port unreachable from the worker host | Verify the proxy URL and that the host can reach it ( |
Container restarts repeatedly | Persistent startup or connection failure | Check |
Error message: secret env var <VAR> is not set on this worker | A credential named by an integration or SSH target (v13.10.0 and later) is missing | Add the variable to .env (or your Secret) and restart |
http integration rejected … | One entry in ADR_HTTP_INTEGRATIONS failed validation | That entry is dropped; the rest keep working. The message names the reason. |
Error message: | Itential credentials are missing | Set |
Error message: |
| Set |
Error message: |
| Set |
HTTP integration validation errors
Logged as http integration rejected with the reason. The entry is dropped; other entries keep working.
Reason | Fix |
|---|---|
| Remove it. Authentication belongs in |
| Use the plain |
| Add |
| Keep only the one your |
| Move the secret to a variable referenced by |
Runtime errors
Message | Meaning |
|---|---|
| A runbook step names a dropped or unconfigured integration |
| The path is outside |
| The API refused the credential or token. Check |
| The token request failed. See OAuth 2.0 token errors ( |
Writes are never retried blind
For every auth type, a POST or PATCH that fails in transport with no response is not retried: it may already have been applied.
OAuth 2.0 token errors
An OAuth2TokenError message names the token URL and client_id, then the provider's error code and description, capped at 300 characters with the secret redacted. The raw response body and HTTP status are omitted because providers can echo credentials.
Provider error | Retried | Fix |
|---|---|---|
| No | Wrong ID or secret, or try |
| No | Fix the client's grants or scopes in the provider console |
| Yes |
|
| — | The private key is not a valid unencrypted PEM, or its base64 is broken |
Network error, timeout, 5xx | Yes | — |
TLS errors
Message | Fix |
|---|---|
| Set |
| Reissue with ≥2048-bit RSA or ≥224-bit ECC. Stopgap: |
| Wrong |
Useful commands
docker logs -f adr-temporal-worker # Stream live logs docker logs --tail 100 adr-temporal-worker # Last 100 lines docker restart adr-temporal-worker # Restart worker docker stop adr-temporal-worker # Stop worker docker rm adr-temporal-worker # Remove stopped worker docker images | grep adr-temporal-worker # Confirm image version
Updating to a new version
When BigPanda releases a new tag, you receive a fresh presigned S3 URL. Your .env and certificates do not need to change:
docker stop adr-temporal-worker && docker rm adr-temporal-worker # then re-run Download and load the image, and Run the worker
Follow the install guide for the target version; configuration may differ between versions.
Support
Contact BigPanda support for certificate renewal or reissue, changes to what data is fetched, and connectivity or authentication issues.