> ## Documentation Index > Fetch the complete documentation index at: https://opensre.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Adding tools and integrations # Adding Tools & Integrations — Definition of Done Use this checklist whenever you add or materially change: * a tool — under `integrations//tools/` for a single-vendor tool, or `tools/system/` / `tools/cross_vendor/` for a cross-cutting one (see [tool-placement-policy.md](tool-placement-policy.md)) * an integration under `integrations//` — its config, client, verifier, and tools * investigation source wiring for an existing tool or integration This is the detailed definition of done; use it with [AGENTS.md](../AGENTS.md) and [CI.md](../CI.md). ## 1. Tool checklist ### Files usually involved * `integrations//tools/_tool/__init__.py` — the tool package (most common path: the tool belongs to a vendor integration) * `tools/system//` or `tools/cross_vendor//` — only when the tool is not vendor-specific (e.g. `tools/system/sre_guidance_tool/`) * `integrations//client.py` — reuse a dedicated integration API client instead of inlining requests * `core/tool_framework/utils/` — shared helper code reused across vendors * `docs/.mdx` — user-facing usage, parameters, examples * `tests/tools/test_.py` — behavior and regression coverage Tools are registry-discovered from **both** `tools/` and `integrations//tools/`, so placement is about ownership, not discovery — see [tool-placement-policy.md](tool-placement-policy.md). Wherever a tool lives, it calls integration-local clients/helpers rather than inlining transport, and never lives in a top-level `vendors/` or `services/` package. Tool packages must be substantive production modules — no empty or discovery-only `__init__.py`, no thin wrapper that only satisfies registry import. Any tool with validation, credential/parameter resolution, transport/client calls, output normalization, or error handling should split those concerns into focused sibling files (`tool.py`, `models.py`, `validation.py`, `delivery.py`/`client.py`, `results.py`), leaving `__init__.py` as a small registry entrypoint that imports the public tool object. ### Contract and implementation * [ ] Pick the simplest shape that fits (`@tool(...)` for lightweight tools, a richer class only when needed) * [ ] `__init__.py` is a small registry entrypoint; non-trivial tools use sibling modules for implementation concerns * [ ] Metadata is complete and accurate: `name`, `description`, `source`, `surfaces`, `requires`, and any `use_cases` / `outputs` / `retrieval_controls` * [ ] `input_schema` matches the actual runtime arguments and required fields * [ ] `is_available` returns `True` only when the tool can genuinely run * [ ] `extract_params` maps resolved integration state into tool args correctly * [ ] Validation, credential/parameter resolution, transport/client calls, and result formatting are separated so each can be tested independently * [ ] Reusable transport or integration-specific parsing lives in `integrations//` or `core/tool_framework/utils/`, not copied into the tool body * [ ] Failure responses have a stable, investigation-friendly shape; expected external failures (missing config, auth, rate limit, upstream 4xx/5xx) return structured errors rather than raising — unexpected exceptions use the global `BaseTool` wrapper intentionally or are migrated with telemetry coverage * [ ] Output is normalized enough for the planner/LLM to consume reliably * [ ] Secrets never leak through `extract_params`, return values, logs, or traceable tool-call kwargs; secret/PII output is run through `platform/masking/` before return * [ ] External side effects declare `side_effect_level`, `requires_approval`, and `approval_reason` where appropriate * [ ] To appear in both investigation and chat, set `surfaces=("investigation", "chat")` ### Live payload parsing If the tool parses API, MCP, log, or webhook payloads: * [ ] Validate against the real or documented upstream response shape, not only idealized mocks * [ ] Handle alternate field names used in live payloads * [ ] Handle missing or partial fields without returning unusable output * [ ] Preserve important context when truncating, tailing, paginating, or flattening data * [ ] Upstream 429 / 5xx responses return a clear, investigation-friendly error rather than raising * [ ] Add at least one regression test using a realistic fixture payload Common failure modes to consider: grouped + ungrouped log content; nested/foldered resources; paginated responses; `hasMore` / cursor mismatches; content-vs-pointer shapes (`logs_content` vs `logs_url`-style payloads). ## 2. Integration checklist ### Files usually involved * `integrations//__init__.py` — config builders, validators, selectors, normalization helpers * `integrations//client.py` — a dedicated API client, when the integration makes direct remote calls * `integrations//verifier.py` — local verification logic * `integrations//tools/_tool/` — the vendor's agent-callable tools (see §1) * `integrations/catalog.py` — resolve the integration into the shared runtime config * `integrations/verify.py` — wire the local verification path * `docs/.mdx` — user-facing setup, usage, verification * `tests/integrations/test_.py`, plus `tests/tools/`, `tests/e2e/`, or `tests/synthetic/` where tools or scenarios exercise it `integrations//` owns everything about one vendor — config, resolution, clients, verifiers, helpers, **and its tools**. Only vendor-less (`tools/system/`) and cross-vendor (`tools/cross_vendor/`) tools live under top-level `tools/`. ### Examples from the repo * Datadog: `integrations/datadog/` (with `integrations/datadog/tools/`), `integrations/catalog.py`, tests under `tests/integrations/datadog/` and `tests/tools/test_datadog_*.py`. * Grafana: `integrations/grafana/` (with `integrations/grafana/tools/`), `integrations/catalog.py`, `surfaces/cli/wizard/local_grafana_stack/`, tests under `tests/integrations/grafana/` and `tests/tools/test_grafana_*.py`. * Hermes: `integrations/hermes/` (with `integrations/hermes/tools/hermes_logs_tool/` and `.../hermes_session_evidence_tool/`), `surfaces/cli/commands/hermes.py`, `tests/hermes/`, `tests/synthetic/hermes/`. ### Core completeness * [ ] Config, normalization, and validators are in place under `integrations//__init__.py` * [ ] Catalog resolution / env loading is wired correctly * [ ] Verification path is wired in `integrations/verify.py` and adapters/registry as needed * [ ] Integration-local client added under `integrations//client.py` (only if it makes direct remote calls) * [ ] Tool layer is wired and stable * [ ] CLI setup flow is updated if the integration is user-configurable locally * [ ] `opensre onboard` parity is added, or intentionally documented as out of scope * [ ] New required env vars / credentials are added to `.env.example` (never `.env`) * [ ] `make verify-integrations` passes ## 3. Investigation wiring If the tool/integration is relevant to investigations: * [ ] Review alert-source seeding in `core/domain/alerts/alert_source.py` * [ ] Review source-priority/prompt mapping in `tools/investigation/stages/gather_evidence/prompt.py` * [ ] Review evidence/source registration in `core/domain/types/` or related state models * [ ] Add scenario coverage proving the tool surfaces useful RCA evidence If the integration is first-class for an `alert_source`, review the source-to-tool maps explicitly. ## 4. Discovery and edge cases For tools that list, search, or inspect resources: * [ ] Folder/nested resource layouts are considered where the upstream supports them * [ ] Large result sets are capped or paginated intentionally * [ ] Partial fetches are surfaced clearly (`truncated`, `fetch_error`, etc.) * [ ] Time/order-sensitive results preserve causal ordering where it matters ## 5. Docs and tests ### Docs * [ ] Ship or update a `docs/` page/section in the same PR (new tool, CLI command, pipeline behavior, or integration; and whenever a tool's API/schema or an integration's setup changes) * [ ] Any new `docs/` page is registered in `docs/docs.json` (without the `.mdx` suffix) so Mintlify navigation shows it * [ ] Investigation LLM tool-calling changes follow [investigation-tool-calling.md](investigation-tool-calling.md) ### Tests * [ ] Unit tests for config/normalization * [ ] Tool contract tests, or equivalent schema/metadata coverage * [ ] A registry/discovery test proves the tool is visible on the expected surface(s) * [ ] Runtime behavior tests for success and failure paths * [ ] At least one realistic fixture for live-payload parsing when external payloads are involved * [ ] If investigation-relevant, a test proves the planner/agent can discover or invoke the tool through the normal runtime path (plus synthetic/scenario coverage when the loop depends on it) * [ ] `tests/integrations/` updated when integration wiring changes Green tests are not enough if they only cover idealized mocks. ### Final gate (new integrations) Everything above is complete, **and**: * [ ] Screenshot or demo GIF showing the integration working end-to-end * [ ] E2E or synthetic test added * [ ] CI checks pass (see [CI.md](../CI.md)) ## 6. Reviewer focus Before opening or approving the PR, confirm the items most often missed are handled **explicitly**: tool placement (§1), live-payload robustness (§1), alert-source maps (§3), onboarding/setup/docs parity (§2 and §5), pagination/truncation/partial-response behavior (§4), and tests that cover realistic payloads and investigation usefulness — not only happy-path mocks (§5). Follow [CI.md](../CI.md) for the mandatory pre-push commands.