Self-hosted deployment - v3 Preview - Open Source LLM Observability

Components

Langfuse consists of multiple storage components and two Docker containers:

• Langfuse Web: The main web application serving the Langfuse UI and APIs.
• Langfuse Worker: A worker that asynchronously processes events.
• Postgres: The main database for transactional workloads.
• Redis: A fast in-memory data structure store. Used for queue and cache operations.
• S3/Blob Store: Object storage to persist all incoming events, multi-modal inputs, and large exports.
• Clickhouse: High-performance OLAP database which stores traces, observations, and scores.

See the chart below for an overview of the components and their interactions:

Postgres Database

Langfuse requires a persistent Postgres database to store its state. You can use a managed service on AWS, Azure, or GCP, or host it yourself. At least version 12 is required.

Redis

Langfuse uses Redis for caching and queue operations. You can use a managed service on AWS, Azure, or GCP, or host it yourself. At least version 7 is required and the instance must have maxmemory-policy=noeviction configured. You may use Valkey instead of Redis, but there is no active support from the Langfuse team as of now. See Redis for more details on how to connect Redis to Langfuse.

S3/Blob Store

Langfuse requires an S3-compatible blob store to persist all incoming events, multi-modal inputs, and large exports. You can use a managed service on AWS, Azure, or GCP, or host it yourself using MinIO. See S3/Blob Store for more details on how to connect a blob store to Langfuse.

Clickhouse

Langfuse uses Clickhouse as an OLAP database to store traces, observations, and scores. You can use the managed service by Clickhouse Cloud, or host it yourself. See ClickHouse for more details on how to connect ClickHouse to Langfuse.

Deploying Langfuse

Deploy the application container to your infrastructure. You can use managed services like AWS ECS, Azure Container Instances, or GCP Cloud Run, or host it yourself.

During the container startup, all database migrations will be applied automatically. This can be optionally disabled via environment variables.

Run Langfuse Web

docker run --name langfuse-web \
  -e DATABASE_URL=postgresql://hello \
  -e NEXTAUTH_URL=http://localhost:3000 \
  -e NEXTAUTH_SECRET=mysecret \
  -e SALT=mysalt \
  -e ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000 \ # generate via: openssl rand -hex 32
  -e CLICKHOUSE_URL=http://clickhouse:8123 \
  -e CLICKHOUSE_USER=clickhouse \
  -e CLICKHOUSE_PASSWORD=clickhouse \
  -e CLICKHOUSE_MIGRATION_URL=clickhouse://clickhouse:9000 \
  -e REDIS_HOST=localhost \
  -e REDIS_PORT=6379 \
  -e REDIS_AUTH=redis \
  -e LANGFUSE_S3_EVENT_UPLOAD_ENABLED=true \
  -e LANGFUSE_S3_EVENT_UPLOAD_BUCKET=my-bucket \
  -e LANGFUSE_S3_EVENT_UPLOAD_REGION=us-east-1 \
  -e LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
  -e LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=bPxRfiCYEXAMPLEKEY \
  # Temp env variables during migration
  -e LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES=true \
  -e LANGFUSE_ASYNC_INGESTION_PROCESSING=true \
  -e LANGFUSE_ASYNC_CLICKHOUSE_INGESTION_PROCESSING=true \
  -e LANGFUSE_READ_DASHBOARDS_FROM_CLICKHOUSE=true \
  -e LANGFUSE_READ_FROM_POSTGRES_ONLY=false \
  -e LANGFUSE_RETURN_FROM_CLICKHOUSE=true \
  -p 3000:3000 \
  -a STDOUT \
langfuse/langfuse:3.0.0-rc.1

Run Langfuse Worker

docker run --name langfuse-worker \
  -e DATABASE_URL=postgresql://hello \
  -e SALT=mysalt \
  -e ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000 \ # generate via: openssl rand -hex 32
  -e CLICKHOUSE_URL=http://clickhouse:8123 \
  -e CLICKHOUSE_USER=clickhouse \
  -e CLICKHOUSE_PASSWORD=clickhouse \
  -e REDIS_HOST=localhost \
  -e REDIS_PORT=6379 \
  -e REDIS_AUTH=redis \
  -e LANGFUSE_S3_EVENT_UPLOAD_ENABLED=true \
  -e LANGFUSE_S3_EVENT_UPLOAD_BUCKET=my-bucket \
  -e LANGFUSE_S3_EVENT_UPLOAD_REGION=us-east-1 \
  -e LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
  -e LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=bPxRfiCYEXAMPLEKEY \
  # Temp env variables during migration
  -e LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES=true \
  -e LANGFUSE_ASYNC_INGESTION_PROCESSING=true \
  -e LANGFUSE_ASYNC_CLICKHOUSE_INGESTION_PROCESSING=true \
  -p 3030:3030 \
  -a STDOUT \
langfuse/langfuse-worker:3.0.0-rc.1

Recommended sizing

For production environments, we recommend to use at least 2 CPUs and 4 GB of RAM for all containers. You should have at least two instances of the Langfuse Web container for high availability. For auto-scaling, we recommend to add instances once the CPU utilization exceeds 50% on either container.

Environment Variables

Langfuse accepts additional environment variables to fine-tune your deployment. You can use the same environment variables for the Langfuse Web and Langfuse Worker containers.

Authentication

Email/Password

Email/password authentication is enabled by default. Users can sign up and log in using their email and password.

To disable email/password authentication, set AUTH_DISABLE_USERNAME_PASSWORD=true. In this case, you need to set up SSO instead.

If you want to provision a default user for your Langfuse instance, you can use the LANGFUSE_INIT_* environment variables.

Password Reset

• If transactional emails are configured on your instance via the SMTP_CONNECTION_URL and EMAIL_FROM_ADDRESS environments, users can reset their password by using the “Forgot password” link on the login page.

• If transactional emails are not set up, passwords can be reset by following these steps:

1. Update the email associated with your user account in database, such as by adding a prefix.
2. You can then sign up again with a new password.
3. Reassign any organizations you were associated with via the organization_memberships table in database.
4. Finally, remove the old user account from the users table in database.

SSO

To enable OAuth/SSO provider sign-in for Langfuse, add the following environment variables:

Use *_ALLOW_ACCOUNT_LINKING to allow merging accounts with the same email address. This is useful when users sign in with different providers or email/password but have the same email address. You need to be careful with this setting as it can lead to security issues if the emails are not verified.

Need another provider? Langfuse uses Auth.js, which integrates with many providers. Add a feature request on GitHub if you want us to add support for a specific provider.

Additional configuration

Headless Initialization

By default, you need to create a user account, organization and project via the Langfuse UI before being able to use the API. You can find the API keys in the project settings within the UI.

If you want to automatically initialize these resources, you can optionally use the following LANGFUSE_INIT_* environment variables. When these variables are set, Langfuse will automatically create the specified resources on startup if they don’t already exist. This allows for easy integration with infrastructure-as-code and automated deployment pipelines.

The different resources depend on each other in the following way. You can e.g. intialize an organization and a user wihtout having to also initialize a project and API keys, but you cannot initialize a project without also initializing an organization.

Organization
├── Project      (part of organization)
│   └── API Keys (set for project)
└── User         (owner of organization)

Troubleshooting:

• If you use LANGFUSE_INIT_* in Docker Compose, do not double-quote the values (GitHub issue).
• The resources depend on one another (see note above). For example, you must create an organization to initialize a project.

Configuring the Enterprise Edition

The Enterprise Edition (compare versions) of Langfuse includes additional optional configuration options that can be set via environment variables.

Health and Readiness Check Endpoint

Langfuse web includes a health check endpoint at /api/public/health and a readiness check endpoint at /api/public/ready and the Langfuse worker a health check endpoint at /api/health. The health check endpoint indicates if the application is alive and the readiness check endpoint indicates if the application is ready to serve traffic.

Access the health and readiness check endpoints:

curl http://localhost:3000/api/public/health
curl http://localhost:3000/api/public/ready
curl http://localhost:3030/api/health

The potential responses from the health check endpoint are:

• 200 OK: Both the API is functioning normally and a successful connection to the database was made.
• 503 Service Unavailable: Either the API is not functioning or it couldn’t establish a connection to the database.

The potential responses from the readiness check endpoint are:

• 200 OK: The application is ready to serve traffic.
• 500 Internal Server Error: The application received a SIGTERM or SIGINT and should not receive traffic.

Applications and monitoring services can call this endpoint periodically for health updates.

Per default, the Langfuse web healthcheck endpoint does not validate if the database is reachable, as there are cases where the database is unavailable, but the application still serves traffic. If you want to run database healthchecks, you can add ?failIfDatabaseUnavailable=true to the healthcheck endpoint.

Encryption

Encryption in transit (HTTPS)

For encryption in transit, HTTPS is strongly recommended. Langfuse itself does not handle HTTPS directly. Instead, HTTPS is typically managed at the infrastructure level. There are two main approaches to handle HTTPS for Langfuse:

1. Load Balancer Termination: In this approach, HTTPS is terminated at the load balancer level. The load balancer handles the SSL/TLS certificates and encryption, then forwards the decrypted traffic to the Langfuse container over HTTP. This is a common and straightforward method, especially in cloud environments.

• Pros: Simplifies certificate management as it is usually a fully managed service (e.g. AWS ALB), offloads encryption overhead from application servers.
• Cons: Traffic between load balancer and Langfuse container is unencrypted (though typically within a secure network).

2. Service Mesh Sidecar: This method involves using a service mesh like Istio or Linkerd. A sidecar proxy is deployed alongside each Langfuse container, handling all network traffic including HTTPS.

• Pros: Provides end-to-end encryption (mutual TLS), offers advanced traffic management and observability.
• Cons: Adds complexity to the deployment, requires understanding of service mesh concepts.

Once HTTPS is enabled, you can configure add LANGFUSE_CSP_ENFORCE_HTTPS=true to ensure browser only allow HTTPS connections when using Langfuse.

Encryption at rest (database)

All Langfuse data is stored in your Postgres database, Clickhouse, Redis, or S3/Blob Store. Database-level encryption is recommended for a secure production deployment and available across cloud providers.

The Langfuse team has implemented this for Langfuse Cloud and it is fully ISO27001, SOC2 Type 2 and GDPR compliant (security page).

Additional application-level encryption

In addition to in-transit and at-rest encryption, sensitive data is also encrypted or hashed at the application level.

Deployment Guides

The Langfuse team and our community maintain a collection of deployment guides to illustrate how you can run Langfuse in various environments. This section is work in progress and relies on community contributions. If you have successfully deployed Langfuse on a specific platform, consider contributing a guide either via a GitHub PR/Issue or by reaching out to the maintainers. Please also let us know if one of these guides does not work anymore or if you have a better solution.

• Docker Compose
• Kubernetes (Helm)

Support

If you experience any issues, please join us on Discord or contact the maintainers at support@langfuse.com.

For support with production deployments, the Langfuse team provides dedicated enterprise support. To learn more, reach out to enterprise@langfuse.com or schedule a demo.

Alternatively, you may consider using Langfuse Cloud, which is a fully managed version of Langfuse. You can find information about its security and privacy here.

FAQ

• Are prebuilt ARM images available?
• Are older versions of the SDK compatible with newer versions of Langfuse?
• I cannot connect to my docker deployment, what should I do?
• I have forgotten my password
• How can I restrict access on my self-hosted instance to internal users?
• How to manage different environments in Langfuse?
• Can I deploy multiple instances of Langfuse behind a load balancer?
• What kind of telemetry does Langfuse collect?

GitHub Discussions