Advanced18 min

Logging, configuration, and secrets

Replace print with structured logs, read config from the environment, and keep API keys out of your repository.

Why this matters in AI / ML / GenAI

When an LLM endpoint misbehaves at 2am, logs are all you have. Leaked API keys in a public notebook are a real and expensive incident. Twelve-factor config is what lets the same image run in dev, staging, and prod.

logging instead of print

print has no levels, no timestamps, and no way to turn off in production. The logging module gives you all three.

Levels: DEBUG (developer detail), INFO (normal operation), WARNING (something odd), ERROR (operation failed), CRITICAL (service is down).

Get a module-level logger: logger = logging.getLogger(__name__). Configure handlers once at the entry point, never inside library code.

Log the event and its context — request id, model, token count, latency — not a vague "something failed".

Structured logs

Log aggregators (CloudWatch, Datadog, Loki) parse JSON far better than prose. Emitting one JSON object per line makes latency_ms > 2000 a query instead of a regex.

Include a correlation id on every line for a request so you can reconstruct one user's journey through retrieval, generation, and post-processing.

Never log full prompts or responses containing personal data. Log a hash, a length, and a truncated preview.

Config and secrets

Read configuration from environment variables: os.environ["OPENAI_API_KEY"] when required, os.getenv("LOG_LEVEL", "INFO") when optional.

Keep a .env file locally, load it with python-dotenv, and add .env to .gitignore. Commit a .env.example with key names only so teammates know what to set.

In production, use a secret manager (AWS Secrets Manager, GCP Secret Manager, Kubernetes secrets). Never hardcode keys, never put them in notebooks, never paste them into a prompt.

If a key does leak: rotate it immediately. Deleting the commit does not remove it from git history or from anyone who already cloned.

Copy-paste examples

Copy into your own editor, or load one into the compiler below and press Run.

Basic logging setup

Configure once at the entry point; use getLogger(__name__) everywhere else.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-8s %(name)s | %(message)s",
    datefmt="%H:%M:%S",
)
logger = logging.getLogger("rag.pipeline")

logger.debug("this is hidden at INFO level")
logger.info("retrieved %d chunks in %d ms", 4, 87)
logger.warning("low similarity: %.2f", 0.31)
try:
    1 / 0
except ZeroDivisionError:
    logger.exception("scoring failed")

Structured JSON logs

One JSON object per line — queryable in any log platform.

import json
import logging

class JsonFormatter(logging.Formatter):
    def format(self, record):
        payload = {
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        extra = getattr(record, "context", None)
        if extra:
            payload.update(extra)
        return json.dumps(payload)

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger = logging.getLogger("llm")
logger.handlers = [handler]
logger.setLevel(logging.INFO)
logger.propagate = False

logger.info("llm_call", extra={"context": {
    "request_id": "req-8891",
    "model": "gpt-4.1-mini",
    "prompt_chars": 812,
    "latency_ms": 743,
}})

Environment-based config

Required keys fail fast; optional ones have defaults.

import os

os.environ["LOG_LEVEL"] = "DEBUG"          # normally set outside the app
os.environ["MODEL_NAME"] = "gpt-4.1-mini"

def require(name):
    value = os.getenv(name)
    if not value:
        raise RuntimeError(f"missing required environment variable: {name}")
    return value

log_level = os.getenv("LOG_LEVEL", "INFO")
model_name = require("MODEL_NAME")
print("log_level:", log_level)
print("model_name:", model_name)

try:
    require("OPENAI_API_KEY")
except RuntimeError as err:
    print("startup check caught it:", err)

Redact before logging

Never write raw keys, prompts with personal data, or full documents to logs.

import hashlib

def redact(secret):
    if not secret:
        return "<unset>"
    digest = hashlib.sha256(secret.encode()).hexdigest()[:8]
    return f"{secret[:3]}***{digest}"

def preview(text, limit=40):
    return text[:limit] + ("..." if len(text) > limit else "")

api_key = "sk-live-abc123def456"
prompt = "My name is Priya and my account number is 998877. Summarize my bill."

print("key:", redact(api_key))
print("prompt_chars:", len(prompt))
print("prompt_preview:", preview(prompt))

Log an LLM call with context

Try it — in-browser Python

Change latency_ms above 2000 and let the warning branch fire.

Output

Python runs in your browser. First run downloads the runtime.

Press Run (or Ctrl+Enter) to execute.

CPython in WebAssembly. Stdlib works. NumPy and pandas load on demand. No input(), no GPU, no network installs.

Takeaways

  • Use logging with levels and context; keep print for scratch work only.
  • Structured JSON logs with a request id make production debugging possible.
  • Secrets come from the environment or a secret manager — never from source control.