> ## Documentation Index
> Fetch the complete documentation index at: https://opensre.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenSearch / Elasticsearch

> Connect OpenSearch or Elasticsearch so OpenSRE can search application logs and analytics indices

## Overview

OpenSRE queries OpenSearch (or Elasticsearch) to retrieve application logs, error events, and analytics records — pulling concrete log lines into its answers alongside metrics and traces from your other observability tools.

For Amazon OpenSearch Service (AWS ES) domain specifics, see also [AWS Elasticsearch](/docs/integrations/databases/elasticsearch).

## Prerequisites

* OpenSearch 1.x/2.x or Elasticsearch 7.x/8.x reachable from the machine running OpenSRE
* The cluster URL (e.g. `https://my-cluster.us-east-1.es.amazonaws.com`)
* Credentials for one of:
  * HTTP Basic Auth (username and password) — typical for self-hosted OpenSearch
  * API key — typical for Elastic Cloud
  * No auth — only when the cluster has the security plugin disabled

OpenSearch authenticates clients via Basic Auth by default; the security plugin does not natively issue API keys ([opensearch-project/security#4009](https://github.com/opensearch-project/security/issues/4009)). Most self-hosted clusters use Basic Auth.

## Setup

### Option 1: Interactive CLI

```bash theme={null}
opensre integrations setup opensearch
```

Pick **OpenSearch / Elasticsearch**. Choose auth mode: **Username + Password**, **API key**, or **None**.

### Option 2: Environment variables

```bash theme={null}
OPENSEARCH_URL=https://my-cluster.example.com

# Basic Auth (typical for self-hosted OpenSearch)
OPENSEARCH_USERNAME=admin
OPENSEARCH_PASSWORD=secret

# API key (typical for Elastic Cloud — use instead of username/password)
OPENSEARCH_API_KEY=your-api-key

# Optional
OPENSEARCH_INDEX_PATTERN=*
OPENSEARCH_MAX_RESULTS=
```

| Variable                   | Default | Description                            |
| -------------------------- | ------- | -------------------------------------- |
| `OPENSEARCH_URL`           | —       | **Required.** Base URL of your cluster |
| `OPENSEARCH_USERNAME`      | —       | Basic auth username                    |
| `OPENSEARCH_PASSWORD`      | —       | Basic auth password                    |
| `OPENSEARCH_API_KEY`       | —       | API key (`Authorization: ApiKey …`)    |
| `OPENSEARCH_INDEX_PATTERN` | `*`     | Default index pattern                  |
| `OPENSEARCH_MAX_RESULTS`   | —       | Optional result cap                    |

Use API key **or** basic auth, not both — verify rejects both set together. If only one of username/password is set, no `Authorization` header is emitted.

### Option 3: Persistent store

```json theme={null}
{
  "version": 1,
  "integrations": [
    {
      "id": "opensearch-prod",
      "service": "opensearch",
      "status": "active",
      "credentials": {
        "url": "https://my-cluster.example.com",
        "username": "admin",
        "password": "secret"
      }
    }
  ]
}
```

## Credentials

Prefer a dedicated read-only user (or API key) scoped to the indices OpenSRE should search. For AWS ES, use fine-grained access control with an internal user — see [AWS Elasticsearch](/docs/integrations/databases/elasticsearch).

### Quick local test with Docker

```bash theme={null}
docker run -d --name opensearch-dev -p 127.0.0.1:9201:9200 \
  -e "discovery.type=single-node" \
  -e "DISABLE_SECURITY_PLUGIN=true" \
  -e "OPENSEARCH_JAVA_OPTS=-Xms256m -Xmx256m" \
  opensearchproject/opensearch:2

i=0
until curl -sf -o /dev/null http://localhost:9201; do
  i=$((i + 1))
  [ "$i" -ge 30 ] && { echo "ERROR: OpenSearch never became ready" >&2; exit 1; }
  sleep 2
done
```

Seed the index with an error **and** a non-error log line — both are needed to match the two-record output shown below. The tools sort on `@timestamp`, so use that field name, not `timestamp`:

```bash theme={null}
curl -sf -X POST "http://localhost:9201/logs-app/_doc" -H "Content-Type: application/json" -d '{
  "@timestamp": "'"$(date -u +%Y-%m-%dT%H:%M:%S)"'Z",
  "level": "ERROR",
  "message": "payment-service: connection timeout while calling billing-api",
  "service": "payment-service"
}' > /dev/null &&
curl -sf -X POST "http://localhost:9201/logs-app/_doc" -H "Content-Type: application/json" -d '{
  "@timestamp": "'"$(date -u +%Y-%m-%dT%H:%M:%S)"'Z",
  "level": "INFO",
  "message": "payment-service: request completed",
  "service": "payment-service"
}' > /dev/null &&
curl -sf -X POST "http://localhost:9201/logs-app/_refresh" > /dev/null || { echo "ERROR: seeding failed" >&2; exit 1; }
```

```bash theme={null}
export OPENSEARCH_URL="http://localhost:9201"
```

Verify:

```bash theme={null}
opensre integrations verify opensearch
```

```
SERVICE    │ SOURCE    │ STATUS   │ DETAIL
opensearch │ local env │ ✓ passed │ Configured for OpenSearch at
           │           │          │ http://localhost:9201.
```

Register the local instance through the supported setup flow:

```bash theme={null}
opensre integrations setup opensearch
```

Now ask the agent about the seeded index:

```bash theme={null}
opensre
```

Ask: *Is payment-service throwing connection timeout errors calling billing-api? Check the logs.*

Against this exact seeded index the agent finds the seeded error via
`query_elasticsearch_logs` (one payment-service timeout, zero billing-api logs
across the window) and concludes billing-api is not serving traffic. It didn't
need `query_opensearch_analytics` here — both tools share the same credentials
and index, so either is reachable from the same setup.

Teardown:

```bash theme={null}
docker rm -f opensearch-dev
```

## Tools

| Tool                         | What it does                                                      |
| ---------------------------- | ----------------------------------------------------------------- |
| `query_opensearch_analytics` | Search analytics / log indices                                    |
| `query_elasticsearch_logs`   | Log lines matching a query in a time range, plus error-like lines |

Both become available once the `opensearch` integration is configured.

## Verify

```bash theme={null}
opensre integrations verify opensearch
```

CLI verify is a **configuration-coherence** check (URL + XOR auth). Onboard may also call `GET /_cluster/health` for live feedback.

## Troubleshooting

| Symptom                             | Fix                                                                            |
| ----------------------------------- | ------------------------------------------------------------------------------ |
| **Both API key and basic auth set** | Clear one method — verify rejects both                                         |
| **403 Forbidden on AWS ES**         | Enable fine-grained access control; use internal user/password (not IAM SigV4) |
| **No tools in plan**                | Confirm verify passes and sources include opensearch                           |
| **Empty results**                   | Check `OPENSEARCH_INDEX_PATTERN` and index permissions                         |

## Security

* Prefer least-privilege read users / API keys.
* Do not use the master user for OpenSRE.
* Store credentials out of source control.