Request Logging & Monitoring

ProductReady includes a centralized request logging system powered by OpenTelemetry and OpenObserve. Track every HTTP request (API calls, page views, tRPC, OpenAPI) for audit trails, analytics, and performance monitoring.

Why Request Logging?

Use Cases

• Audit Trails: Track who accessed what and when (GDPR/HIPAA compliance)
• Performance Monitoring: Find slow endpoints and optimize response times
• User Analytics: Understand how users interact with your app
• Security: Detect suspicious activity (failed login attempts, rate limiting)
• Debugging: Trace errors back to specific requests

What Gets Logged

Each request logs:

• User ID (if authenticated) or "anonymous"
• Session ID (from Better Auth)
• IP Address (extracted from Cloudflare/Vercel headers)
• User Agent (browser/device info)
• HTTP Method (GET, POST, PUT, DELETE, etc.)
• URL Path (/api/users/123)
• Status Code (200, 404, 500, etc.)
• Duration (response time in milliseconds)
• Timestamp (ISO 8601 format)

What's Excluded

Auto-filtered paths (no logging):

• Static assets: /_next/, *.ico, *.png, *.css, *.js
• Health checks: /health, /ping
• Auth callbacks: /api/auth/callback/*

Architecture

┌─────────────┐
│   Request   │
└──────┬──────┘
       │
       ▼
┌─────────────────────┐
│  Next.js Middleware │  (proxy.ts → withRequestLogging)
└──────┬──────────────┘
       │
   ┌───┴───┐
   │       │
   ▼       ▼
Traces    Logs
(OTel)  (HTTP POST)
   │       │
   ▼       ▼
┌─────────────────────┐
│    OpenObserve      │  http://localhost:5080
│   (OLAP Database)   │  Traces + Logs + Dashboards
└─────────────────────┘

Dual pipeline:

• Traces (OTel SDK → OTLP): Distributed tracing, performance spans
• Logs (Direct HTTP → /_json): Structured request audit logs, searchable

Key benefits:

• ✅ Non-blocking: Logs are buffered and sent asynchronously (<1ms impact)
• ✅ Scalable: Handles 100k+ requests/day
• ✅ Flexible: Switch to Grafana Tempo, Jaeger, or any OTLP endpoint
• ✅ Cost-effective: OLAP storage is cheaper than PostgreSQL for logs

Quick Start

# Start all services including OpenObserve
make db
# Or: docker-compose up -d

# Verify it's running
curl http://localhost:5080/healthz
# {"status":"ok"}

# Enable request logging (disabled by default)
ENABLE_REQUEST_LOGGING="true"

# OpenObserve endpoint
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:5080/api/default"

# serviceName from appConfig.name, version from package.json — no extra env vars

# OpenObserve authentication
OPENOBSERVE_EMAIL="admin@productready.dev"
OPENOBSERVE_PASSWORD="productready"

pnpm dev

# You should see:
# [OTelLogger] ✅ Initialized (serviceName=productready)

Viewing Logs in OpenObserve

Basic Queries

http_response_status_code > 0

http_response_status_code >= 500

http_duration_ms > 1000

user_id = 'user_abc123'

Advanced Queries

Requests to specific endpoint:

url_path LIKE '/api/users/%'

Anonymous vs Authenticated:

user_id = 'anonymous'  -- Anonymous users
user_id != 'anonymous' -- Authenticated users

Requests by IP address:

client_address = '203.0.113.42'

Time-based filtering:

@timestamp > now() - 1h  -- Last hour
@timestamp > now() - 24h -- Last 24 hours

OpenObserve Dashboard

Creating Dashboards

1. Go to Dashboards → Create Dashboard
2. Add panels with queries:
   • Request Count: count() by http_request_method
   • Error Rate: count(http_response_status_code >= 500) / count() * 100
   • Avg Response Time: avg(http_duration_ms)
   • Top Endpoints: count() by url_path

Setting Up Alerts

1. Go to Alerts → Create Alert
2. Example: Alert on high error rate

   count(http_response_status_code >= 500) > 10

3. Configure notification (Slack, Email, Webhook)

Configuration

Environment Variables

Note: serviceName from appConfig.name, serviceVersion from package.json — no extra env vars needed.

Custom Filtering

To exclude additional paths, edit packages/share-domains/observability/logic/otel-logger.ts:

export function shouldLogPath(path: string): boolean {
  // Add your custom exclusions
  if (path.startsWith("/internal/")) return false;
  if (path.startsWith("/webhooks/")) return false;
  
  // Default exclusions...
  return true;
}

Production Deployment

Option 1: Self-Hosted OpenObserve

Deploy OpenObserve on your own infrastructure:

# docker-compose.prod.yml
services:
  openobserve:
    image: openobserve/openobserve:v0.14.0
    restart: always
    ports:
      - '5080:5080'
    volumes:
      - openobserve_data:/data
    environment:
      ZO_DATA_DIR: /data
      ZO_ROOT_USER_EMAIL: ${OPENOBSERVE_EMAIL}
      ZO_ROOT_USER_PASSWORD: ${OPENOBSERVE_PASSWORD}
    mem_limit: 4g

Option 2: OpenObserve Cloud

Use the managed service:

1. Sign up at openobserve.ai
2. Get your endpoint: https://api.openobserve.ai/api/default
3. Update .env:

   OTEL_EXPORTER_OTLP_ENDPOINT="https://api.openobserve.ai/api/default"
   OPENOBSERVE_EMAIL="your-email@example.com"
   OPENOBSERVE_PASSWORD="your-password"

Option 3: Switch to Grafana Tempo

OpenTelemetry supports any OTLP backend:

# Grafana Cloud
OTEL_EXPORTER_OTLP_ENDPOINT="https://tempo-prod-04-prod-us-east-0.grafana.net/tempo"

# Self-hosted Grafana Tempo
OTEL_EXPORTER_OTLP_ENDPOINT="http://tempo:4318/v1/traces"

Performance Impact

Benchmarks

Optimization Tips

1. Batch size: Default is 512 logs per 5 seconds. Adjust in otel-logger.ts:

   maxQueueSize: 1024,
   maxExportBatchSize: 256,
   scheduledDelayMillis: 3000,

2. Sampling: For very high traffic, sample requests:

   if (Math.random() > 0.1) return handler(req, ev); // 10% sampling

3. Field reduction: Remove unnecessary fields to reduce payload size.

Troubleshooting

echo $ENABLE_REQUEST_LOGGING
# Should output: true

curl http://localhost:5080/healthz
# Should return: {"status":"ok"}

pnpm dev
# Should see: [OTelLogger] Initialized with endpoint: ...

curl http://localhost:3000/
# Then check OpenObserve UI

maxQueueSize: 512,        // Reduce from 2048
maxExportBatchSize: 128,  // Reduce from 512

// Log only 10% of requests
if (Math.random() > 0.1) return;

OpenObserve is not running. Start it with:
make db  # or: docker-compose up -d

Check OPENOBSERVE_EMAIL and OPENOBSERVE_PASSWORD in .env

OpenObserve flattens dotted keys to underscores.
Use user_id instead of "user.id" in queries.
Check actual field names in OpenObserve UI → stream schema.

Privacy & Compliance

GDPR Compliance

Request logging is opt-in (disabled by default). To comply with GDPR:

1. Inform users: Add to your privacy policy
2. Retention policy: Delete logs after 30-90 days
3. Right to deletion: Provide endpoint to delete user logs
4. Anonymize IPs: Hash IP addresses before logging (optional)

Data Minimization

Only essential metadata is logged:

• ❌ No request bodies
• ❌ No response bodies
• ❌ No cookies (except session ID)
• ❌ No passwords or API keys
• ✅ Only: method, path, status, duration, user ID

IP Anonymization

To anonymize IPs, edit ip-utils.ts:

export function extractClientIP(headers: Headers): string | null {
  const ip = headers.get("cf-connecting-ip") || headers.get("x-real-ip");
  if (!ip) return null;
  
  // Anonymize: Replace last octet with 0
  return ip.replace(/\.\d+$/, ".0");  // 203.0.113.42 → 203.0.113.0
}

OpenObserve Field Name Mapping

Next Steps

Related Documentation

• OpenTelemetry Documentation
• OpenObserve Documentation
• OTLP Protocol Specification
• Better Auth Session Management