CubelaPetar/notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

new notes

Browse Source
This commit is contained in:
Petar Cubela
2025-03-18 14:23:17 +01:00
parent e6c2775f5f
commit 6c47451c60
58 changed files with 1648 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
projects/discopharma/20250310-Next_Steps.md Normal file
View File

@@ -0,0 +1,149 @@
## Goal
Setup a metabase instance via docker with https support and a professional Deployment Pipeline
## Questions
- Separate Reverse Proxy or local Web Server enough??
- Exisiterende SSL Zertifikate nutzen?
- Kriege ich irgendwie Zugang?
### 20250311
- How many users?
- What is the old db software? Maybe we can reuse it? Are there backups of the old database ?
- DNS Verwaltung
- is the metabase version a requirement?
## Meeting-20250311
Teilnehmer: Lukas Maas, Milos Nikolic, Petar Cubela
### Answers
- DB: MySQL. Backup dump exist.
- Version needs to be 0.49.18
- 20 people
- Existing certs
- Use Reverse Proxy
- I will get access to the machines
### My Time/ Steps
1. Databse Instance MySQL (0.5h -1h)
2. Metabase (.50 h)
3. VM R2verse Proxy (.50 h)
4. Find and Test the recreation of the data/dashboard database (metabase.db/) (1-2h)
5. write overwivew network setup (ip address, open ports in firewall, metabase.discopharma.de -> public ip ) (1h)
6. Recreate in discopharma setup: (2-3h)
1. dns setup properly
2. network setup properly
3. creation of the VMs (oeither discopharma or me)
4. Installation process (db exist, docker deployment of metabase, reverse proxy)
5. Test
## List of requirements regarding Metabase deployment (discopharma)
1. Find or create backup of Metabase Dashboard data within Docker image on the old machine (marketplace image that was compromised, or a previous image of it)
2. Solution architecture that obeys to best practices of security, so that
- DISCO employees can connect to a DISCO-internal metabase application using a web browser and the URL “metabase.discopharma.de”
- The application is not exposed to the public
- All connections to the application are encrypted (https)
3. Solution architecture that includes a
- Productive instance (highest priority)
- Development/sandbox instance (lower priority)
- A process to deploy upgrades of the application (lower priority)
4. Metabase version 0.49.18
## Requirements
- properly configured and firewalled google cloud; VMs should only be able to communicate via private IPs!
- VM in google cloud for the metabase instance; Public IP address, port 80 and 443 forwarded; 1 cores, 2GB RAM (depends on user number)
- VM in google cloud for the metabase database instance; Private IP address; 1 cores, 1GB RAM (depends on user number); PostgreSQL
- Use existing SSL certs(??) with web server/reverse proxy like nginx/traefik/etc
## Software
- Debian 12
- Docker
- Metabase
- PostgreSQL
- Traefik/Nginx (depends)
## Notes
### 20250311
- <https://www.metabase.com/learn/metabase-basics/administration/administration-and-operation/metabase-in-production#metabase-application-server-size>
- Run separate database (PostgreSQL) and application server instances
#### Metabase application server size
- Metabase needs at least 1 core and 1GB of RAM
- For every 20 concurrent people it needs 1CPU and 2GB of RAM
#### Metabase application database server size
- Database needs at least 1 core and 2GB of RAM
- For every 40 concurrent people it needs 1CPU and 1GB of RAM
## docker-compose.yml example
```yml
services:
metabase:
image: metabase/metabase:latest
container_name: metabase
hostname: metabase
restart: unless-stopped
volumes:
- /dev/urandom:/dev/random:ro
- "./metabase-db:/metabase.db"
- ./plugins:/plugins
ports:
- 3000:3000
environment:
JAVA_TIMEZONE: Europe/Berlin
MB_DB_FILE=/metabase.db
MB_DB_TYPE: postgres
MB_DB_DBNAME: metabase
MB_DB_PORT: 5432
MB_DB_USER_FILE: /run/secrets/db_user
MB_DB_PASS_FILE: /run/secrets/db_password
MB_DB_HOST: postgres
networks:
- metanet1
secrets:
- db_password
- db_user
healthcheck:
test: curl --fail -I http://localhost:3000/api/health || exit 1
interval: 15s
timeout: 5s
retries: 5
postgres:
image: postgres:latest
container_name: postgres
hostname: postgres
restart: unless-stopped
environment:
POSTGRES_USER_FILE: /run/secrets/db_user
POSTGRES_DB: metabase
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
networks:
- metanet1
secrets:
- db_password
- db_user
networks:
metanet1:
driver: bridge
secrets:
db_password:
file: db_password.txt
db_user:
file: db_user.txt
```

65
projects/discopharma/20250311-metabase-environment.md Normal file
View File

@@ -0,0 +1,65 @@
## VM Ressources and Setup
The listed IP Addresses are only example values here and can be chosen on your judgement. Important is that the machines can communicate with each other.
### MySQL Database
- Name: MySQL Database
- OS: Debian 12
- hostname: db.discopharma.de (unimportant)
- IP Address: 10.156.0.5/24
- CPU: 1 core
- RAM: 2 GB (2048 MB)
- Storage: depends (30 GB)
- DNS entry: none
- Note: for every 40 concurrent users: needs 1CPU and 1GB of RAM more
### Metabase Server
- Name: Metabase Server
- OS: Debian 12
- hostname: mb.discopharma.de (unimportant)
- IP Address: 10.156.0.6/24
- CPU: 1 core
- RAM: 1 GB (1024 MB)
- Storage: depends (30 GB)
- DNS entry: none
- Note: for every 20 concurrent users: needs 1CPU and 2GB of RAM more
### Reverse Proxy
- Name: Reverse Proxy
- OS: Debian 12
- hostname: rproxy.discopharma.de (unimportant)
- IP Address: 10.156.0.7/24 + \<PUBLIC IP\> address (only activated in the end)
- CPU: 1 core
- RAM: 1 GB (1024 MB)
- Storage: depends (16 GB)
- DNS entry: metabase.discopharma.de -> \<PUBLIC IP\>
- Note: for every concurrent users: needs 1CPU and 2GB of RAM more
## SSL/TSL certificates
- we need the discopharma wildcard certificate placed on the Reverse Proxy
- usually two files enough called `privkey.pem` and `fullchain.pem`
- you can put all the cert files on the reverse proxy and we will then use only the needed ones or convert them in the process if necessary
## Firewall Setup
I list all necessary communications and respective ports needed:
(Abbreviations:
- Databse: db = 10.156.0.5
- Metabse: mb = 10.156.0.6
- ReverseProxy: rp = 10.156.0.7)
| Source | SourcePort | Destination | DestPort | Description |
| ------------- | ----------------- | --------------- | ----------------- | ------------------------------------------------------------------------------------------- |
| mb | 3306/tcp | db | 3306/tcp | 3306 is the standard mysql port. Communication of mb to db |
| rp | 3000/tcp,3000/udp | mb | 3000/tcp,3000/udp | 3000 is the metabase web port (arbitrary). Reverse Proxy sends request via this port to mb. |
| OPEN INTERNET | any | PUBLIC IP of rp | 443/tcp | 443 is the https port to communicate to rp over internet |
You could also limit the access to the public ip such that only your company ip can reach it. The 443 port should be opened as the last thing when everything is done.
When the VMs are in the same private network, they should be able to openly communicate with each other; the first two entries in the table should be already open.

19
projects/discopharma/20250312-metabase-deployment.md Normal file
View File

@@ -0,0 +1,19 @@
## Metabase Instance
### Requirements
- [x] unattended-updates
- [x] docker
### Database
- name: metabase
- user: metabase
- pass: /E^bOu|<C{Y{bZu
### Reverse Proxy
- [x] unattended-updates
- [x] fail2ban
- [x] nginx

11
projects/discopharma/20250317-finishing-meeting.md Normal file
View File

@@ -0,0 +1,11 @@
## To do's:
- Cloud SQL dump load and user mgmt (Miloš)
- Docker licensing (Lukas)
- backup procedure for MB application db (Petar)
- Documentation/ manual (Petar)
- For example,
- how deployment works,
- what docker image to select
- how the routing in the reverse proxy is done