]
This commit is contained in:
Petar Cubela
300
archive/discopharma/20250320-manual-project.md
Normal file
300
archive/discopharma/20250320-manual-project.md
Normal file
@@ -0,0 +1,300 @@
|
||||
---
|
||||
title: "Metabase - Setup Manual"
|
||||
author: Petar Cubela
|
||||
date: March 20, 2025
|
||||
geometry: margin=1.5cm
|
||||
output: pdf_document
|
||||
---
|
||||
|
||||
## Intro
|
||||
|
||||
Setting up a Metabase instance via Docker with a PostgreSQL application database and a secure web connection via https mediated by a public facing reverse proxy (nginx) and commercial TLS/SSL certificates.
|
||||
|
||||
### Goals and Requirements
|
||||
|
||||
### Software
|
||||
|
||||
- Google Cloud Platform (GCP)
|
||||
- [Debain 12 (OS)](https://www.debian.org/download)
|
||||
- [Docker (Containerization Platform)](https://docs.docker.com/engine/install/debian/)
|
||||
- [NGINX (Web Server, Reverse Proxy)](https://docs.nginx.com/)
|
||||
- [Postgres (as Container)](https://hub.docker.com/_/postgres)
|
||||
- [Metabase (as Container)](https://hub.docker.com/r/metabase/metabase)
|
||||
|
||||
## VM Specs
|
||||
|
||||
### Metabase Server
|
||||
|
||||
- Name: Metabase Server
|
||||
- OS: Debian 12
|
||||
- hostname: mb-prod
|
||||
- IP Address: `10.156.0.6/24`
|
||||
- CPU: 2 core
|
||||
- RAM: 2 GB (2048 MB)
|
||||
- Storage: depends (30 GB)
|
||||
- DNS entry: none
|
||||
- Note: for every 20 concurrent users: needs 1CPU and 2GB of RAM more
|
||||
|
||||
### Reverse Proxy
|
||||
|
||||
- Name: Reverse Proxy
|
||||
- OS: Debian 12
|
||||
- hostname: rproxy
|
||||
- IP Address: `10.156.0.7/24` + `<PUBLIC IP>` address (only activated in the end)
|
||||
- CPU: 1 core
|
||||
- RAM: 1 GB (1024 MB)
|
||||
- Storage: depends (16 GB)
|
||||
- DNS entry: metabase.discopharma.de -> `<PUBLIC IP>`
|
||||
- Note: for every concurrent users: needs 1CPU and 2GB of RAM more
|
||||
|
||||
|
||||
### Firewall
|
||||
|
||||
I list all necessary communications and respective ports needed:
|
||||
|
||||
Abbreviations:
|
||||
- Metabse: mb-prod = `10.156.0.6`
|
||||
- Metabse Dev: mb-dev = `10.156.0.8`
|
||||
- ReverseProxy: rp = `10.156.0.7`
|
||||
|
||||
|
||||
| Source | SourcePort | Destination | DestPort | Description |
|
||||
| ------------- | ----------------------- | --------------- | ----------------------- | ------------------------------------------------------------------------------- |
|
||||
| mb-prod | 3306/tcp | db | 3306/tcp | 3306 is the standard mysql port. Communication of mb-prod to db |
|
||||
| rp | 3000/tcp </br> 3000/udp | mb-prod | 3000/tcp </br> 3000/udp | 3000 is the metabase web port. Reverse Proxy sends request via this port to mb. |
|
||||
| rp | 3000/tcp </br>3000/udp | mb-dev | 3000/tcp </br> 3000/udp | 3000 is the metabase web port. Reverse Proxy sends request via this port to mb. |
|
||||
| OPEN INTERNET | any | PUBLIC IP of rp | 443/tcp | 443 is the https port to communicate to rp over internet |
|
||||
|
||||
### Network Diagram
|
||||
|
||||
|
||||
|
||||
## Metabase Application Server and Database
|
||||
|
||||
### Administration
|
||||
|
||||
#### Update
|
||||
|
||||
In order to update the metabase containers change to the `~/metabase/` folder (where `compose.yml` file resides) and use the following command:
|
||||
```bash
|
||||
docker compose pull && docker compose up -d
|
||||
```
|
||||
Monitor the container logs to see if there are any errors by using the command:
|
||||
```bash
|
||||
docker compose logs -f
|
||||
```
|
||||
The `docker compose pull` command searches for images which are specified by a tag in the image variable in the `compose.yml` file:
|
||||
`image: metabase/metabase:latest`
|
||||
`latest` is here the tag and can also be changed to a version number which can be extracted from the [docker-hub](https://hub.docker.com/r/metabase/metabase/tags).
|
||||
|
||||
To simplify the process I wrote a simple bash script which updates the container images and removes old container images. The script is in the folder `/home/lukas_discopharma_de/scripts/metabase-update.sh`.
|
||||
The update has to be done manually.
|
||||
|
||||
#### Backup
|
||||
|
||||
There is a script `/home/lukas_discopharma_de/db-backup.sh` which creates a database dump from the postgres instance running in the container and places the dump into the folder at `/home/lukas_discopharma_de/backup-db` including the current date in the filename.
|
||||
The scripts runs weekly mondays at 2 a.m. via a cronjob. You should secure the backups/dumps to a secure location.
|
||||
|
||||
|
||||
### Development Instance
|
||||
|
||||
Go step-by-step through the installation and setup of a development metabase instance.
|
||||
|
||||
#### 1. Setup VM
|
||||
|
||||
Setup the a new VM with specs as described in the [VM specs](#vm-specs) section. The OS we are using is Debian 12. The private ip address can be chosen as `10.156.0.8`
|
||||
|
||||
#### 2. Update pkgs and install docker and compose
|
||||
|
||||
After Installation of the OS perform a pkg update:
|
||||
```bash
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
```
|
||||
|
||||
In order to install docker engine we will follow the official [documentation](https://docs.docker.com/engine/install/debian/).
|
||||
1. Set up Docker's `apt` repository
|
||||
```bash
|
||||
# Add Docker's official GPG key:
|
||||
sudo apt-get update
|
||||
sudo apt-get install ca-certificates curl
|
||||
sudo install -m 0755 -d /etc/apt/keyrings
|
||||
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
|
||||
sudo chmod a+r /etc/apt/keyrings/docker.asc
|
||||
|
||||
# Add the repository to Apt sources:
|
||||
echo \
|
||||
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
|
||||
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
|
||||
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
sudo apt-get update
|
||||
```
|
||||
|
||||
2. Install the Docker packages (which includes docker compose)
|
||||
```bash
|
||||
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||
```
|
||||
3. Verify that the installation is successful by running the `hello-world` image
|
||||
```bash
|
||||
sudo docker run hello-world
|
||||
```
|
||||
|
||||
|
||||
It is possible to manage Docker as a non-root user. It the next steps we describe how to achieve this.
|
||||
We need to create a `docker` group and add to user we wish to use:
|
||||
1. Create the `docker` group
|
||||
```bash
|
||||
sudo groupadd docker
|
||||
```
|
||||
2. Add your user to the `docker` group.
|
||||
```bash
|
||||
sudo usermod -aG docker $USER
|
||||
```
|
||||
3. Log out and log back in so that your group membership is re-evaluated
|
||||
4. Verify that you can run `docker` commands without `sudo`
|
||||
```bash
|
||||
docker run hello-world
|
||||
```
|
||||
|
||||
#### 3. Create folder and compose file
|
||||
|
||||
After getting Docker Engine to work we can setup the necessary files and folders for the metabase container.
|
||||
Create a metabase folder for the docker compose files in your home folder:
|
||||
``` bash
|
||||
mkdir -p ~/metabase/plugins
|
||||
```
|
||||
|
||||
In addition create two files where the database user name and password will be placed:
|
||||
```bash
|
||||
touch ~/metabase/{db_user.txt,db_password.txt}
|
||||
```
|
||||
|
||||
Create a `compose.yml` file which will be used to spin up the containers:
|
||||
|
||||
```yaml
|
||||
---
|
||||
services:
|
||||
metabase:
|
||||
image: metabase/metabase:latest
|
||||
container_name: mb-dev
|
||||
hostname: mb-dev
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /dev/urandom:/dev/random:ro
|
||||
- ./plugins:/plugins
|
||||
ports:
|
||||
- 3000:3000
|
||||
environment:
|
||||
JAVA_TIMEZONE: Europe/Berlin
|
||||
MB_DB_TYPE: postgres
|
||||
MB_DB_DBNAME: metabase
|
||||
MB_DB_PORT: 5432
|
||||
MB_DB_USER_FILE: /run/secrets/db_user
|
||||
MB_DB_PASS_FILE: /run/secrets/db_password
|
||||
MB_DB_HOST: postgres
|
||||
networks:
|
||||
- metanet1
|
||||
secrets:
|
||||
- db_password
|
||||
- db_user
|
||||
healthcheck:
|
||||
test: curl --fail -I http://localhost:3000/api/health || exit 1
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
postgres:
|
||||
image: postgres:latest
|
||||
container_name: postgres-dev
|
||||
hostname: postgres-dev
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER_FILE: /run/secrets/db_user
|
||||
POSTGRES_DB: metabase
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
|
||||
networks:
|
||||
- metanet1
|
||||
secrets:
|
||||
- db_password
|
||||
- db_user
|
||||
networks:
|
||||
metanet1:
|
||||
driver: bridge
|
||||
secrets:
|
||||
db_password:
|
||||
file: db_password.txt
|
||||
db_user:
|
||||
file: db_user.txt
|
||||
```
|
||||
|
||||
Choose a name for the database user and place it in the `db_user.txt` file, e.g.:
|
||||
```bash
|
||||
echo "metabase" > db_user.txt
|
||||
```
|
||||
|
||||
and accordingly for the password:
|
||||
```bash
|
||||
echo "SecurePass" > db_password.txt
|
||||
```
|
||||
|
||||
Change the permissions of the files such that they are read-only for your own user:
|
||||
```bash
|
||||
chmod 400 db_*.txt
|
||||
```
|
||||
|
||||
#### 4. Pull images and start container
|
||||
|
||||
The pull of the container images and the start of the containers can be simply done by one command. Change the working directory to the metabase folder,
|
||||
```bash
|
||||
cd ~/metabase
|
||||
```
|
||||
and execute the command:
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
During the startup the log files for the containers should be monitored for possible errors by using the command:
|
||||
```bash
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
If you see now errors and if you have the possibility to reach the server you can visit the metabase instance using the URL `http://<private-ip-of-server>:3000`. Port 3000 has to be open and you have to be able to reache the server via its private ip address.
|
||||
|
||||
## Reverse Proxy
|
||||
|
||||
The software which is used on the reverse proxy server is called `nginx`. This is a standard common web server/reverse proxy. Its configuration files reside in the folder `/etc/nginx/` and its log files can be found in `/var/logs/nginx/`.
|
||||
The configuration file which accomplishes the reverse proxying for your metabase instance is `/etc/nginx/sites-available/metabase.conf`:
|
||||
```conf
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
listen 443 ssl http2;
|
||||
listen [::]:443 ssl http2;
|
||||
|
||||
server_name metabase.discopharma.de;
|
||||
|
||||
ssl_certificate /etc/nginx/ssl/discopharma.de/discopharma_fullchain.cer;
|
||||
ssl_certificate_key /etc/nginx/ssl/discopharma.de/discopharma_private.key;
|
||||
|
||||
|
||||
|
||||
if ($ssl_protocol = "") {
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://10.156.0.6:3000;
|
||||
proxy_set_header HOST $host;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
In order to reverse proxy traffic to a development instance you can proceed as follows:
|
||||
1. Create a nginx configuration file for the dev metabase instance by copying the existing config: `cp /etc/nginx/sites-available/metabase.conf /etc/nginx/sites-available/mb-dev.conf`
|
||||
2. Open the new file using any text editor `nano /etc/nginx/sites-available/mb-dev.conf` and edit the `server_name` and `proxy_pass` variables to reflect your new dev instance, e.g.: `server_name mb-dev.discopharma.de;` (the corresponding dns entry for `mb-dev.discopharma.de` has to point to the public ip of the reverse proxy) and `proxy_pass http:<private-ip-of-server>:3000;`
|
||||
3. Create a symbolic link (nignx reads the config files in `sites-enabled`):
|
||||
```bash
|
||||
ln -sf /etc/nginx/sites-available/mb-dev.conf /etc/nginx/sites-enabled/
|
||||
```
|
||||
4. Restart the `nignx` service: `systemctl restart nginx`
|
||||
5. Setup your google firewall such that the reverse proxy can reach your dev metabase instance via port 3000.
|
||||
6. Visit `https://mb-dev.discopharma.de`. The homepage should working ssl certificates which are configured in the `nginx` configuration file for mb-dev.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user