Datadog Setup for Ledgerline

Ledgerline uses Datadog for logs, traces, errors, and metrics. When the Datadog Agent is deployed (e.g. as a Render private service), the app uses dd-trace for APM and DogStatsD for metrics. Without the agent, it falls back to HTTP APIs.

Modes

Mode	When	Logs	Traces/APM	Errors	Metrics
Agent	DD_AGENT_HOST set	Render Log Stream	dd-trace → Agent	dd-trace	DogStatsD → Agent
Agentless	DD_API_KEY set, no agent	Render Log Stream + HTTP Logs API	—	HTTP Logs API	HTTP Metrics API

What Gets Sent to Datadog

• Logs: Pino JSON to stdout → Render Log Stream → Datadog (configure in Render Dashboard)
• Traces/APM: When agent is deployed, dd-trace sends spans for Express, MongoDB, Redis
• Errors: dd-trace (agent) or HTTP Logs API (agentless)
• Metrics: DogStatsD to agent, or HTTP Metrics API when agentless

Environment Variables

Variable	Required	Description	Default
DD_API_KEY	Yes (agent or agentless)	Datadog API key; agent needs it to forward data	—
DD_AGENT_HOST	When using agent	Agent hostname (from Render fromService)	—
DD_SITE	No	Datadog site (e.g. us5.datadoghq.com for US5)	datadoghq.com
DD_SERVICE	No	Service name	ledgerline-api
DD_ENV	No	Environment tag	CONFIG_ENV
DD_LOGS_INJECTION	No	Inject trace_id/span_id into logs for correlation	true when agent
DD_TRACE_AGENT_PORT	No	Agent APM port	8126
DD_TRACE_SAMPLE_RATE	No	Trace sampling (0–1)	1
DD_PROFILING_ENABLED	No	Continuous Profiler	false
DD_DATA_STREAMS_ENABLED	No	Data Streams Monitoring	false
DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED	No	Cleaner service names	false
DD_APPSEC_ENABLED	No	Application Security (threat detection)	false
DD_ENABLED	No	Disable when key/agent present	true

Local Docker Setup

For local development with Datadog APM:

1. Set DD_API_KEY in .env (from Datadog → Organization Settings → API Keys).

2. Run with the datadog profile:

   pnpm dev:all:datadog

   Or: docker compose -f conf/docker-compose.yml --profile datadog up --build

3. The datadog-agent service joins the ledgerline network. The api and worker use DD_AGENT_HOST=datadog-agent, DD_TRACE_AGENT_PORT=8126, DD_LOGS_INJECTION=true.

4. Without the profile, pnpm dev:all runs without the agent; traces won't reach Datadog but the app runs normally.

Render Setup

1. Datadog Agent (Recommended)

The blueprint includes a datadog-agent private service. Set DD_API_KEY on the datadog-agent service (Render Dashboard → datadog-agent → Environment). The ledgerline-api receives DD_AGENT_HOST via fromService.

• Agent exposes APM (8126) and DogStatsD (8125)
• Plan: starter (or free if available for pserv)
• Same region as ledgerline-api (ohio)

2. API Key on ledgerline-api

For agentless mode (no agent), set DD_API_KEY on ledgerline-api. For agent mode, the API does not need DD_API_KEY; the agent has it.

3. Log Stream

To send stdout logs to Datadog:

1. Render Dashboard → Integrations → Log Streams
2. Add Datadog, enter API key and site
3. Attach to ledgerline-api

4. (Optional) Metrics Stream

Render can stream infrastructure metrics. See Render Metrics Streams.

Custom Metrics

The application emits the following custom metrics (all tagged service:ledgerline-api and env:<env>):

Metric	Type	Tags	Source
discord.interaction.rejected	count	reason, env, service	discord.endpoint.ts
discord.interaction.duplicate	count	tenantId, env, service	discord.endpoint.ts
discord.interaction.success	count	command, tenantId, env, service	discord.endpoint.ts
http.request.duration	gauge (ms)	method, route, status, env, service	api.init.ts
http.request.count	count	method, route, status, env, service	api.init.ts
order_session.active	gauge (count)	tenantId, env, service	order-session-gauge.job.ts (1m interval)

order_session.active reflects non-terminal sessions (states not in SUBMITTED, CANCELLED, EXPIRED, ABANDONED) per tenant at query time.

Datadog Dashboard

• Logs: Logs Explorer — service:ledgerline-api
• APM: APM Traces — when agent is used
• Metrics: Metrics Explorer — http.request.count, http.request.duration, order_session.active

Architecture

• Agent mode: instrument.ts loads dd-trace when DD_AGENT_HOST is set; dd-trace patches Express, MongoDB, Redis. Metrics go to DogStatsD (agent:8125).
• Agentless mode: HTTP Logs API for errors, HTTP Metrics API for counters/timings.
• Logs: Always via Render Log Stream (Pino JSON stdout); dd.service and dd.env in base bindings for correlation.