Module 4: The Easy Path — Zero-Code Auto-Instrumentation

Node.js / Express

1. Add the two packages to the dependencies in track-js/todo-api/package.json:

   "dependencies": {
     "@opentelemetry/api": "latest",
     "@opentelemetry/auto-instrumentations-node": "latest",
     "express": "^4.21.2",
     ...
   }
   

   Your API’s container runs npm install every time it starts, so there is nothing to install by hand. Nothing gets baked into an image, and no image is rebuilt.

2. Create track-js/todo-api/.env. This is the only file you create, and the application never reads it — docker-compose.yml already hands it to the container as environment (env_file:), exactly the way a deployment platform would hand environment to a process:

   OTEL_SERVICE_NAME=todo-api
   OTEL_RESOURCE_ATTRIBUTES=service.version=1.0.0,deployment.environment=workshop
   OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
   OTEL_TRACES_EXPORTER=otlp
   OTEL_METRICS_EXPORTER=otlp
   OTEL_LOGS_EXPORTER=otlp
   OTEL_METRIC_EXPORT_INTERVAL=10000
   OTEL_METRIC_EXPORT_TIMEOUT=5000
   NODE_OPTIONS=--require @opentelemetry/auto-instrumentations-node/register
   

   That last line is the one that actually turns the SDK on. It is the canonical form:

   node --require @opentelemetry/auto-instrumentations-node/register src/server.js
   

   🚨 The endpoint is otel-collector, not localhost. That’s the Collector’s service name in docker-compose.yml. Inside your API’s container, localhost is the container, and the SDK would spend all day failing to export to nothing.

   ⚠️ The --require must load before your application code, or the instrumentation has nothing left to hook. It also only patches CommonJS require() calls, which is why this application is CommonJS and not ESM. With ESM you would get HTTP spans and silently lose Express, database and logging instrumentation — a genuinely nasty failure mode, because it looks like it worked.

3. Recreate your API’s container, so it picks up the new environment and installs the new dependencies. Saving a source file is not enough this time — the container’s configuration changed, not your code:

   docker compose up -d --force-recreate todo-api-js
   

   Then open a second terminal and follow your API’s log in it. Keep it open for the rest of the day — the handouts call it your log terminal:

   docker compose logs -f todo-api-js
   

   Wait for the todo-api listening on 8081 line. The image is the same, the command is the same, your source is the same. The only new thing is the environment.

   You may also see a MetadataLookupWarning. That’s the SDK checking whether it’s running on a cloud provider, to describe where it runs. Off a cloud, the check fails; it’s harmless.

4. Generate traffic in another terminal:

   ./loadgen.sh 2m
   

Python / Flask

1. Add the packages to track-py/todo-api/requirements.txt:

   opentelemetry-distro>=0.66b1
   opentelemetry-exporter-otlp
   

   Your API’s container runs pip install every time it starts, so there is nothing to install by hand.

2. Create track-py/todo-api/.env. This is the only file you create, and the application never reads it — docker-compose.yml already hands it to the container as environment (env_file:), exactly the way a deployment platform would hand environment to a process:

   OTEL_SERVICE_NAME=todo-api
   OTEL_RESOURCE_ATTRIBUTES=service.version=1.0.0,deployment.environment=workshop
   OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
   OTEL_TRACES_EXPORTER=otlp
   OTEL_METRICS_EXPORTER=otlp
   OTEL_LOGS_EXPORTER=otlp
   OTEL_METRIC_EXPORT_INTERVAL=10000
   OTEL_METRIC_EXPORT_TIMEOUT=5000
   OTEL_PYTHON_LOG_AUTO_INSTRUMENTATION=true
   OTEL_PYTHON_LOG_CORRELATION=true
   

   Unlike the JavaScript track, no environment variable can turn the Python SDK on. You change the command instead. In docker-compose.yml, find the todo-api-py service and put opentelemetry-instrument in front of gunicorn:

       command: opentelemetry-instrument gunicorn -w 2 -b 0.0.0.0:8082 app:app
   

   That is the canonical form. Don’t add gunicorn’s --reload: the container already restarts the whole command when a file changes, and under opentelemetry-instrument gunicorn’s own reloader silently stops logs from being exported.

3. Recreate your API’s container, so it picks up the new environment and installs the new dependencies. Saving a source file is not enough this time — the container’s configuration changed, not your code:

   docker compose up -d --force-recreate todo-api-py
   

   Then open a second terminal and follow your API’s log in it. Keep it open for the rest of the day — the handouts call it your log terminal:

   docker compose logs -f todo-api-py
   

   It stays quiet for a minute while pip installs the new packages, then gunicorn prints Listening at: http://0.0.0.0:8082. The image is the same and your source is unchanged. The new things are the environment and one word in front of the command.

4. Generate traffic in another terminal:

   ./loadgen.sh 2m
   

Go / net/http

1. There is nothing to add to go.mod. On this track the instrumentation isn’t loaded when the program starts — Go has no hook for that. It is compiled into the binary, by a tool called otelc that wraps go build: it finds the libraries your code uses (net/http, database/sql, …) and adds OpenTelemetry to them while compiling. It is already installed in your API’s container.

2. Create track-go/todo-api/.env. This is the only file you create, and the application never reads it — docker-compose.yml already hands it to the container as environment (env_file:), exactly the way a deployment platform would hand environment to a process:

   OTEL_SERVICE_NAME=todo-api
   OTEL_RESOURCE_ATTRIBUTES=service.version=1.0.0,deployment.environment=workshop
   OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
   OTEL_TRACES_EXPORTER=otlp
   OTEL_METRICS_EXPORTER=otlp
   OTEL_LOGS_EXPORTER=otlp
   OTEL_METRIC_EXPORT_INTERVAL=10000
   OTEL_METRIC_EXPORT_TIMEOUT=5000
   

   Now change how the binary is built. In docker-compose.yml, find the todo-api-go service. Its command runs air, which rebuilds and restarts the server whenever you save a .go file. The --build.cmd at the end of the line is how it builds:

       command: air --tmp_dir /tmp/air --build.exclude_file "otel.instrumentation.go,otelc.runtime.go" --build.entrypoint /tmp/todo-api --build.cmd "go mod tidy && go build -o /tmp/todo-api ."
   

   Put otelc in front of go build:

       command: air --tmp_dir /tmp/air --build.exclude_file "otel.instrumentation.go,otelc.runtime.go" --build.entrypoint /tmp/todo-api --build.cmd "go mod tidy && otelc go build -o /tmp/todo-api ."
   

   That is the canonical form: otelc go build instead of go build, and nothing else changes.

3. Recreate your API’s container, so it picks up the new environment and the new build command. Saving a source file is not enough this time — the container’s configuration changed, not your code:

   docker compose up -d --force-recreate todo-api-go
   

   Then open a second terminal and follow your API’s log in it. Keep it open for the rest of the day — the handouts call it your log terminal:

   docker compose logs -f todo-api-go
   

   The first build takes a minute while otelc downloads the instrumentation; later ones take a second or two. Then you’ll see OpenTelemetry initialized and todo-api listening on 8085. The image is the same and your source is unchanged. The new things are the environment and one word in front of go build.

4. Generate traffic in another terminal:

   ./loadgen.sh 2m
   

{__name__=~"http.*"}

Node.js / Express

rate(http_server_request_duration_seconds_count{service_name="todo-api"}[1m])

Python / Flask

rate(http_server_duration_milliseconds_count{service_name="todo-api"}[1m])

Go / net/http

go_goroutine_count{service_name="todo-api"}

otelc doesn’t record HTTP metrics yet: its HTTP instrumentation produces spans, not metrics. What you do get are the Go runtime’s own metrics — goroutines, memory, garbage collection — all starting with go_. On this track the HTTP panels of the dashboards stay empty, and that is information too.

service.name:todo-api

Go / net/http

On the Go track, nothing arrives here yet. otelc adds the trace ID to each slog record, but it doesn’t send the records anywhere: your logs still only go to the container’s output, which is what your log terminal shows.

_time:15m "resource_attr:service.name":=notify-service name:="POST /notify"

trace_id:=<paste it here>

Node.js / Express

POST /todos                          ← HTTP server span, named by route
├── middleware - jsonParser          ← the web framework's own layers
├── middleware - query
└── request handler - /todos
    ├── pg-pool.connect              ← postgres instrumentation
    ├── pg.query:INSERT todo
    ├── pg.query:INSERT todo         ← one more per tag
    ├── pg.query:UPDATE todo
    └── POST                         ← outbound HTTP to notify-service
        └── POST /notify             ← and the *other service's* span, same trace

Python / Flask

POST /todos                          ← HTTP server span, named by route
├── INSERT                           ← postgres instrumentation
├── INSERT                           ← one more per tag
├── UPDATE
└── POST                             ← outbound HTTP to notify-service
    └── POST /notify                 ← and the *other service's* span, same trace

Go / net/http

POST /todos                          ← HTTP server span, named by route
├── INSERT                           ← database/sql instrumentation
├── INSERT                           ← one more per tag
├── UPDATE
└── POST                             ← outbound HTTP to notify-service
    └── POST /notify                 ← and the *other service's* span, same trace

Node.js / Express

• span_attr:http.request.method
• span_attr:url.path
• span_attr:db.system.name

Python / Flask

• span_attr:http.method
• span_attr:http.target
• span_attr:db.system

Go / net/http

• span_attr:http.request.method
• span_attr:url.path
• span_attr:db.system.name