Langfuse vs Phoenix: Self-Host LLM Tracing

Full example: Clone the working files at github.com/KingPin/sumguy-examples/llm/langfuse-vs-phoenix

Your Chatbot Said Something Weird And You Have No Idea Why

A user reports that your self-hosted assistant gave a nonsense answer. You open Jaeger. You see a 4.2 second span called POST /chat. You do not see the prompt, the completion, the token count, or the retrieved chunks that went into the context. The trace is correct and useless.

LLM tracing tools fix that, and two of them are worth self-hosting: Langfuse and Arize Phoenix. My take after running both: pick Phoenix if you want tracing and evals on a box that also runs everything else. Pick Langfuse if you need prompt management with versioning and a team that logs in. Phoenix is two containers and about half a gigabyte of idle RAM. Langfuse is six containers and over a gigabyte. Both are free to self-host, with fine print on the license that I read so you don’t have to.

I ran both stacks on a 16-core Docker host and measured them. The numbers are below, along with the traps I hit.

What LLM Tracing Gives You That Plain OTel Doesn’t

A normal trace answers “which service was slow.” An LLM trace answers “what did the model see and say, and what did it cost.” That means a few extra things stored per span:

• The full prompt and completion, not just a status code.
• Token counts (input, output, total) so you can price a request.
• The model name and parameters, so you can compare runs.
• A way to attach scores to a trace: thumbs up, an LLM-as-judge verdict, a regex check.
• Datasets of past inputs you can replay against a new prompt.

Jaeger and SigNoz can store prompts as span attributes if you push them there. Nothing renders them as a conversation, and nothing scores them. If you want the general tracing foundation first, read OpenTelemetry for Self-Hosters and SigNoz vs Jaeger. The LLM-specific layer sits on top of that.

Langfuse: The Full Platform

Langfuse v4 (I ran 4.76.0, released 2026-11-05) is a web app, a background worker, and four data stores. The official compose file lists seven services: langfuse-web, langfuse-worker, an optional langfuse-ai-gateway, Postgres, ClickHouse, Redis, and MinIO for S3-style blob storage. I dropped the gateway, which is optional and not part of tracing, and ran six.

services:
  langfuse-web:
    image: docker.langfuse.com/langfuse/langfuse:4.76.0
    depends_on: *langfuse-depends
    ports:
      - "127.0.0.1:3200:3000"
    environment:
      <<: *langfuse-env
      NEXTAUTH_SECRET: ${NEXTAUTH_SECRET}
      LANGFUSE_INIT_ORG_ID: homelab
      LANGFUSE_INIT_PROJECT_ID: demo
      LANGFUSE_INIT_PROJECT_PUBLIC_KEY: ${LANGFUSE_INIT_PROJECT_PUBLIC_KEY}
      LANGFUSE_INIT_PROJECT_SECRET_KEY: ${LANGFUSE_INIT_PROJECT_SECRET_KEY}
      LANGFUSE_INIT_USER_EMAIL: [email protected]
      LANGFUSE_INIT_USER_PASSWORD: ${LANGFUSE_INIT_USER_PASSWORD}

  langfuse-worker:
    image: docker.langfuse.com/langfuse/langfuse-worker:4.76.0
    depends_on: *langfuse-depends
    environment: *langfuse-env

The LANGFUSE_INIT_* variables create the org, project, admin user and API keys on first boot. No clicking through a signup form. Every secret in the kit is a CHANGEME in .env.example, and you generate real ones with openssl rand -hex 32.

One v4 change matters if you script against the API. Langfuse v4 runs in “events only” mode, and the old GET /api/public/traces returns a 404 on a fresh deployment with a message pointing you to OpenTelemetry ingestion. To read data back, use the v2 observations endpoint:

curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  "http://localhost:3200/api/public/v2/observations?limit=5&fields=core,basic,io,usage"

If your old dashboards or scripts hit /api/public/traces, they break on a new v4 install.

Phoenix: One Container And A Database

Phoenix (I ran server v21.0.0, released 2026-10-30) is one Python service. It uses SQLite in its working directory by default and Postgres when you set PHOENIX_SQL_DATABASE_URL. The UI and OTLP/HTTP share port 6006. OTLP/gRPC listens on 4317.

services:
  phoenix:
    image: arizephoenix/phoenix:version-21.0.0
    depends_on:
      db:
        condition: service_healthy
    ports:
      - "127.0.0.1:6106:6006"   # UI + OTLP/HTTP
      - "127.0.0.1:4417:4317"   # OTLP/gRPC
    environment:
      PHOENIX_SQL_DATABASE_URL: postgresql://postgres:${POSTGRES_PASSWORD}@db:5432/postgres
      PHOENIX_SECRET: ${PHOENIX_SECRET}
      PHOENIX_ADMIN_SECRET: ${PHOENIX_ADMIN_SECRET}

Authentication is the first gotcha. Phoenix v21 turns authentication on whenever the server binds a non-loopback address, and inside Docker it always does. My first trace post got a 401, and the log banner said “Authentication: Enabled” even though I had set nothing. Authentication turns on without either secret, so you set them to get a key, not to turn auth on. PHOENIX_ADMIN_SECRET works as a bearer token that authenticates as the first admin user, which gives scripts a key without clicking through the UI. Both secrets are optional, but the admin secret only works when PHOENIX_SECRET is also set. Both need 32 or more characters with a digit and a lowercase letter, and they must differ from each other.

Sending The Same Trace To Both

The script makes one chat completion through the OpenAI Python SDK against any OpenAI-compatible server. Point OPENAI_BASE_URL at Ollama, llama.cpp, vLLM, whatever you run.

The Langfuse path uses a drop-in wrapper. You change one import and every call becomes a traced generation. @observe adds a parent span around it:

from langfuse import get_client, observe
from langfuse.openai import OpenAI

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)

@observe(name="explain-tracing")
def ask(question: str) -> str:
    r = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": question}],
    )
    return r.choices[0].message.content

print(ask("In one sentence, what is a trace?"))
get_client().flush()

The Phoenix path is plain OpenTelemetry plus OpenInference, a set of semantic conventions for LLM spans. You register a tracer provider and patch the OpenAI SDK:

from openinference.instrumentation.openai import OpenAIInstrumentor
from phoenix.otel import register
from openai import OpenAI

provider = register(
    project_name="demo",
    endpoint="http://localhost:6106/v1/traces",
    batch=True,
    auto_instrument=False,
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

client = OpenAI(base_url=BASE_URL, api_key=API_KEY)
r = client.chat.completions.create(model=MODEL, messages=[...])
provider.force_flush()

That explicit endpoint is the second gotcha. Give register() a bare origin and no protocol=, and it picks gRPC and dials port 4317 on the host. It ignores the port in your environment variable. I remapped Phoenix to 4417 on the host, and the export failed with a connection refused error that never reached my code. Pass the full /v1/traces URL and the problem goes away.

I checked arrival through each tool’s API, not the UI. Langfuse returned a SPAN named explain-tracing with a GENERATION child holding 23 input tokens, 14 output tokens, and the completion text. Phoenix returned a ChatCompletion span with the same token counts under llm.token_count.*. One caveat: the model behind both runs was a stub OpenAI-compatible server that returns a fixed reply, because my usual local model was offline during testing. The stub proves the plumbing and the token capture. It does not say anything about trace overhead under real generation.

Feature Comparison

All of the cells above come from each project’s docs and pricing pages. I only ran the ingest-and-read-back path, not every feature. Treat the table as a map and test the one feature you care about before committing.

The two tools center on different things. Langfuse treats prompts as managed assets: you edit a prompt in the UI, label a version “production”, and your app fetches it by name. That is a real workflow for a team where someone other than the developer edits prompts. Phoenix treats the trace as the center and builds experiments around it: take a dataset, run two prompt variants, compare scores. If you are one person iterating on a RAG pipeline, Phoenix’s loop feels shorter.

OpenTelemetry fit also differs. Phoenix is OTel-first. Anything that emits OpenInference spans works, and your traces can go to Phoenix and to Jaeger at once. Langfuse started with its own SDK protocol and now accepts OTLP at POST /api/public/otel/v1/traces. Its v4 SDKs send over OpenTelemetry too.

The License Fine Print

GitHub’s API reports NOASSERTION for both repos, which is the API admitting it could not match a single license. I read the files.

Langfuse is MIT for the core. The root LICENSE says everything under ee/, web/src/ee/, worker/src/ee/ and packages/shared/src/server/ee/ falls under a separate commercial license. Langfuse’s self-hosting docs list what needs a paid key as of November 2026, including project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking, UI customization, SCIM, the org and instance management APIs, organization creator allowlisting, external media storage, and IAM-role auth for S3. The docs state that all core features and APIs run without limits in the MIT build, with the same scalability as Langfuse Cloud. Also note that the repo is now copyrighted by ClickHouse, Inc., since Langfuse joined ClickHouse.

For a home lab, none of the paid list matters. You get tracing, evals, datasets, prompt management and cost tracking for free. If you want to enforce roles per project or keep an audit trail, you are in paid territory.

Phoenix is Elastic License 2.0. ELv2 is source-available, not OSI open source. You may use, copy, modify and run it. You may not offer it to third parties as a hosted or managed service, and you may not tamper with license-key functionality. Arize’s own license page says self-hosting on your own infrastructure is free and fully permitted, and the self-hosting overview says there are no feature gates. Running it for yourself or your own company is fine. Building a “Phoenix as a service” product is not.

What It Costs To Run

I measured both stacks one at a time on a Docker 29 host with 16 CPUs. Images were pulled, volumes were wiped before each boot, and “healthy” means the Langfuse health endpoint returns 200 and the Phoenix UI returns 200. RAM comes from docker stats --no-stream, sampled three times, 10 seconds apart, after a 90 to 120 second settle.

Langfuse’s RAM bounced. Right after boot and the first trace, the web container hit 1.34 GiB and the worker 681 MiB, so the stack briefly sat near 2.4 GiB before settling. Size your box for the peak, not the idle number. Phoenix stayed flat.

Two limits apply. These are idle numbers with a single trace in the database, so they say nothing about ClickHouse or Phoenix under real ingest volume. And the Langfuse image total counts ClickHouse at 903 MB, the biggest single image, which is the price of fast analytics queries over millions of observations that a home lab will not generate.

Which One For Whom

Pick Phoenix if you run a small box, you are one or two people, and you want to see prompts, completions and token counts today. Two containers, OTel-native, nothing paywalled, and you can swap in SQLite for a single-container setup. Watch the ELv2 clause if you ever productize it.

Pick Langfuse if a non-developer edits prompts, you want versioned prompts with labels fetched at runtime, or several people share one instance. The extra four containers buy you a real prompt registry and ClickHouse headroom. Budget 1.2 GiB idle, 2.4 GiB at the peak I saw, and a bit of care on upgrades because six services have six version pins.

Pick neither if you only want to know which service is slow. Your existing OpenTelemetry backend does that. An LLM tracing tool earns its RAM when you start asking what the model said and whether it was any good.

Either one puts the bad completion on screen at 2 AM, and that is the job.

Common Questions

Can I run Langfuse without ClickHouse?

No. Langfuse v3 and v4 both need ClickHouse for trace storage, alongside Postgres, Redis and S3-compatible storage. The official compose file includes all of them. If you want fewer moving parts, Phoenix runs on SQLite or a single Postgres database.

Is Arize Phoenix really free to self-host?

Yes. Phoenix uses the Elastic License 2.0, and Arize’s license page says self-hosting on your own infrastructure is free and fully permitted. The one restriction is that you cannot sell Phoenix to others as a hosted or managed service. Running it for your own team is allowed.

Does Phoenix work with Ollama and llama.cpp?

Yes, through the OpenAI-compatible endpoints those servers expose. Phoenix traces the OpenAI Python SDK via the OpenInference instrumentation, so you point OPENAI_BASE_URL at Ollama or llama.cpp and the spans arrive as usual. Langfuse works the same way through its drop-in OpenAI wrapper.

Can I send traces to Phoenix and Jaeger at the same time?

Yes. Phoenix ingests standard OTLP, so an OpenTelemetry Collector can fan the same spans out to Phoenix and to Jaeger or SigNoz. Langfuse also accepts OTLP over HTTP at /api/public/otel/v1/traces. The Collector exporter for Langfuse must add the header x-langfuse-ingestion-version: 4, and then the same fan-out works for it.

Which Langfuse Python SDK version works with a v4 server?

Langfuse Python SDK 5.x is the safe pick for a v4 server. The kit pins langfuse==5.2.0 against server 4.76.0, and that pair traced the demo with no changes. Langfuse Python SDK v3 and later export over OpenTelemetry, the ingestion path v4 is built around. Read the Langfuse SDK upgrade guide before pointing an older v2 SDK at a v4 server.

Related Reading

• OpenTelemetry for Self-Hosters: Traces, Metrics, Logs Without the Datadog Bill
• SigNoz vs Jaeger for Tracing
• OpenTelemetry Collector: One Pipeline to Rule Them All
• RAGAS: Evaluating RAG Without Vibes
• OpenRouter vs LiteLLM