Lightdash is an open-source analytics platform licensed under MIT, designed for teams looking to provide self-service reporting to their organization. Positioning itself as an open-source alternative to Looker, Lightdash allows users to explore, visualize, and report on data without advanced technical knowledge. It’s especially popular for modern data environments due to its direct database connectivity while using a semantic layer to define and transform data.
How does Lightdash Work?
When you build a report in Lightdash, you’re leveraging a dynamic SQL generation engine that combines real-time data access, a powerful semantic layer, and intuitive visualization tools. Here’s how it all works behind the scenes:
1. Direct Database Connection
Lightdash connects directly to your database, executing SQL queries in real time. This eliminates the need for duplicative data extraction or storage, ensuring you’re always working with live, up-to-date data. We know you hate outdated snapshots and manual refreshes as much as we do — your reports are as current as your database.
2. dbt: The Semantic Layer
At the core of Lightdash is a semantic layer powered by dbt (data build tool). This layer standardizes how data is structured and defines key metrics, ensuring consistency across your organization.
- Dynamic SQL Queries: Lighdash dynamically generates SQL queries for users when they add fields and filters to their reports. No technical knowledge is required.
- Metric Consistency: A well-defined semantic layer means no more debates like, “Wait, how did you calculate revenue?” Everyone stays on the same page, and data teams can easily audit queries for accuracy.
This approach empowers stakeholders to focus on insights, even without a technical background, while data teams maintain trust and clarity in reporting.
3. Self-Serve Analytics
Once the query results are returned, non-technical users can dive in. They can view the data in a table or create compelling visualizations without needing extensive analytics expertise.
- Flexible Dashboards: Combine multiple reports into a single dashboard, with each widget running its own independent query. No more restrictive, single-source dashboards, or costly snapshot joins …. looking at you Tableau.
- Automated Alerts: Keep stakeholders in the loop with scheduled alerts and report distributions, ensuring everyone stays informed without needing to check in manually.
By bridging the gap between technical and non-technical users, Lightdash makes self-serve analytics a reality, unlocking faster, more confident decision-making across your organization.
Understanding Hosting Options
When looking to implement Lightdash, there are a few hosting methods to consider:
- Lightdash Cloud, a solution provided by the developers themselves
- Self-Hosting
- Partnering with Driftwave (us), for a customized implementation experience
While Lightdash offers a managed cloud service, self-hosting is a powerful option for users who prioritize customization, data sovereignty, or hands-on control. In this guide, we’ll explain the concept of self-hosting, its benefits, considerations, and provide step-by-step instructions for deploying Lightdash on your own hardware.
Considerations for Self-Hosting
Self-hosting Lightdash can provide significant benefits, including greater control, cost efficiency, and customization. However, it also comes with challenges that businesses need to carefully evaluate.
1. Plan Ahead to Avoid Infrastructure Headaches
Self-hosting demands the right infrastructure. This includes physical or cloud servers, reliable internet connectivity, and scalable storage. To set yourself up for success:
- Assess your existing infrastructure to ensure it meets the software’s resource needs.
- Implement redundancy measures to minimize downtime.
- Monitor server health and usage metrics regularly to avoid slowdowns.
For instance, hosting analytics platforms like Metabase or Lightdash requires not only sufficient server capacity but also consistent performance for large datasets. By planning ahead, you can prevent bottlenecks and keep your operations running smoothly.
2. Stay on Top of Maintenance and Updates
When self-hosting, maintenance and updates are your responsibility. Staying proactive in this area ensures optimal performance and security. To manage this effectively:
- Assign a team or individual to monitor updates and patches.
- Schedule regular maintenance to keep the software running smoothly.
- Anticipate compatibility challenges and test updates in a staging environment.
Proactively addressing maintenance prevents the risks of neglect, such as security vulnerabilities or system instability.
3. Evaluate the Full Cost of Ownership
Self-hosting can offer cost savings, but only if you account for all expenses. To avoid surprises:
- Calculate upfront costs for hardware and software licenses.
- Factor in ongoing labor for maintenance, support, and upgrades.
By understanding the true Total Cost of Ownership (TCO), you can make an informed decision about whether self-hosting is the right fit for your organization.
Option 1: Getting Started with Docker Compose
Using Docker Compose is the simplest way to set up a Lightdash instance. While this setup isn’t production-ready, it’s perfect for testing and exploring Lightdash’s capabilities.
Requirements
To follow this guide, you’ll need:
- The Git CLI or the Github CLI: To clone the repository.
- Basic Docker knowledge: Familiarity with containers and Docker commands.
- Docker installed and running: Ensure Docker is installed and the service is running on your machine.
Take a look at the docker-compose.yml file in the Lightdash repository. This file is a configuration that runs three components (called “services”) using Docker containers, which are lightweight virtual environments. These components are:
- Lightdash: The main app for analytics and dashboards.
- PostgreSQL: A database to store Lightdash’s data.
- Headless Browser: Used for generating reports or visualizations.
Each service runs in its own isolated container but works together. Now let’s get Lightdash running on your local machine.
Step 1: Clone the Github Repository
You can clone the Lightdash repository using either the Github CLI or Git CLI:
Using the Github CLI:
gh repo clone lightdash/lightdash
Using the Github CLI:
git clone https://github.com/lightdash/lightdash.git
Step 2: Navigate to the Lightdash Directory
After cloning the repository, you need to navigate to the folder where the files were saved.
By default, the repository will be cloned into a folder named lightdash in your current working directory. This means you can the change directory with simply:
cd lightdash
For example, if you ran:
git clone https://github.com/lightdash/lightdash.git /path/to/desired/location
Replace path/to/desired/location with the path you provided and navigate there:
cd /path/to/desired/location
Step 3: Start Lightdash
To start Lightdash, run the following command from the root directory of the repository:
docker compose up
You will notice that this command will:
- Pull the latest images for all three containers (Lightdash, Postgres, and the Headless Browser)
- Perform any necessary migrations
- Start Lightdash on http://localhost:8080/
Step 4: Create Your Account and Connect Your Data
Once Lightdash is running, open your browser and navigate to http://localhost:8080/. You’ll see the login screen where you can
- Create your Lightdash account.
- Connect your dbt project to start exploring your analytics data.
Option 2: Getting Started with Kubernetes
Deploying Lightdash on Kubernetes provides a scalable and flexible environment for your analytics platform. While we are deploying Lightdash locally, the following steps can be applied to your production Kubernetes cluster.
Requirements
- The Git CLI or the Github CLI
- Familiarity with Kubernetes and Docker
- Docker installed and running
kubectl and helm installed to interact with your Kubernetes instance
Step 1: Add the Lightdash Helm Repository
kubectl create namespace lightdash
Step 2: Create a Namespace for Lightdash
A namespace in Kubernetes is a way to organize and isolate resources. By creating a namespace called lightdash, all resources (pods, services, secrets) related to Lightdash will stay grouped and separate from other applications running in your cluster.
kubectl create namespace lightdash
Step 3: Configure Lightdash with a values.yaml File
- secrets.LIGHTDASH_SECRET:
- Used to encrypt data stored in the database.
- Losing this secret means you won’t be able to decrypt your data, rendering it inaccessible.
- service.type:
- Defines how Lightdash is accessible:
- ClusterIP: Internal access within the Kubernetes cluster (default).
- LoadBalancer or NodePort: Exposes Lightdash outside the cluster for external users.
- Choose based on whether you need internal-only access or external availability.
- Defines how Lightdash is accessible:
- SITE_URL:
- The URL at which Lightdash will be available.
- This ensures links and redirects within Lightdash work correctly. You can update this later if needed.
# values.yaml
secrets:
LIGHTDASH_SECRET: your_secure_secret
configMap:
SITE_URL: https://lightdash.mycompany.com
service:
type: NodePort
Step 4: Install Lightdash with Helm
# bash
helm install lightdash lightdash/lightdash -n lightdash -f values.yaml
Step 5: Create Your Account and Connect Your Data
Once Lightdash is deployed, you can create your account and connect your data just as described in the Docker Compose section above.
Open your browser and navigate to the URL specified in the SITE_URL parameter (or the external URL configured with LoadBalancer or NodePort).
Transitioning from Your Machine to a Production Environment
After getting Lightdash running on your local machine, the next step is transitioning to a production-ready setup. Here are three paths you can take, depending on your requirements and resources:
1. Do-It-Yourself
For those comfortable managing infrastructure, you can deploy Lightdash using:
- A Virtual Machine in the Cloud: Set up a server with your preferred cloud provider (e.g., AWS, GCP, or Azure)
- Managed Kubernetes Providers: Services like Amazon EKS, Google GKE, or Azure AKS simplify Kubernetes management while giving you scalability and flexibility.
- On-Premises Deployment: For businesses with specific compliance or security requirements, you can deploy Lightdash on your in-house infrastructure.
Each option provides full control over your deployment, but you’ll be responsible for maintaining uptime, scalability, backups, and security.
2. Lightdash Cloud
If you’d rather focus on analytics and leave infrastructure management to the experts, one option is Lightdash Cloud.
With Lightdash Cloud is managed by the core development team and offers:
- Hassle-free hosting and maintenance.
- Automatic updates to the latest versions.
- Built-in scalability to meet your growing data needs.
This option is ideal for teams that want to get up and running quickly without worrying about technical overhead.
3. Partnering with Driftwave
For a flexible, vendor-neutral approach, Driftwave offers managed hosting for open-source analytics tools, including Lightdash. With Driftwave, you’ll benefit from everything Lightdash Cloud plus:
- No per-user fees, ensuring cost efficiency as your team grows.
- Full exportability of your instance, so you retain control and avoid vendor lock-in.
- Expert guidance on deploying and managing Lightdash alongside other tools in the modern data stack.
Driftwave bridges the gap between DIY control and managed convenience, making it an excellent choice for small and medium-sized businesses looking to scale their analytics capabilities.
Contact us today to discuss your BI needs and explore how Driftwave can help you accelerate your business growth. Right now we are offering 3 months of Lightdash hosting for free for organizations already using dbt in their data pipelines.