- Create a synapse Docker Image
- Configuring synapse
- Running it
Matrix is intriguing to me from both a community and technological aspect, but for this purpose, I wanted to run my own community for friends and family. What this entry goes into is the configuration, from setup of the initial digital ocean droplet/VM to the deployment of synapse on Kubernetes. TURN setup will be covered in a future article; so if you’re looking for that, come back for updates! :D
What does the Matrix protocol provide? A very high level overview compared to IRC:
- Offline chat logs (sync on connect, like slack/discord/others)
- Voice/Video chat (this involves the aforementioned TURN server)
- End-to-end encryption of channels and private messages
- User accounts & roles (with multiple back-ends to integrate with your existing environments)
- Federation with other servers
And a lot more. Matrix and synapse are quite the bears to get going primarily because of all the features it supports.
Synapse is the premier software that powers Matrix protocol servers. It is primarily written in python and uses YAML configuration, as well as a database and file storage. In this article we will account for most of the basic storage areas as well as touch on network management a little.
About this Tutorial
I expect this to change over time and I will revise it or rewrite it as needed. This document is current as of Aug 6, 2020.
Digital Ocean will be providing our VM today for our experiment. Some of the context around installation will be related to that. Rest assured, most of this article will be applicable to anyone looking to set up synapse in Kubernetes, no specific cloud technology is used.
Our installation is a Ubuntu 20.04 with DO’s basic plan $40/mo option, which is currently rated at 8GB ram, 4 vcpu, and 160GB storage.
My host will be named
int throughout the tutorial.
This is a hobby server, and while appropriate measures are taken to ensure security, uptime and scalability are kind of thrown out the door. If want to scale synapse, that’s going to take more effort.
Upgrade the host
Get all the latest dependencies:
apt update && apt dist-upgrade -y && reboot
For this tutorial, matrix only needs port 443 ingress; egress should be unlimited. We have disabled traefik in our example to enable our load balancer and hard-code the IP.
We are going to use k3s to set up our installation of Kubernetes since it is great for single-node installations. To do this, you can follow the instructions on the website; or simply:
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik" sh -
After a minute or two, you should be able to
k3s kubectl get nodes to see the
“cluster” in action. Here’s what ours looks like:
root@int:~# curl -sfL https://get.k3s.io | sh - [INFO] Finding release for channel stable [INFO] Using v1.18.6+k3s1 as release [INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v1.18.6+k3s1/sha256sum-amd64.txt [INFO] Downloading binary https://github.com/rancher/k3s/releases/download/v1.18.6+k3s1/k3s [INFO] Verifying binary download [INFO] Installing k3s to /usr/local/bin/k3s [INFO] Creating /usr/local/bin/kubectl symlink to k3s [INFO] Creating /usr/local/bin/crictl symlink to k3s [INFO] Creating /usr/local/bin/ctr symlink to k3s [INFO] Creating killall script /usr/local/bin/k3s-killall.sh [INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh [INFO] env: Creating environment file /etc/systemd/system/k3s.service.env [INFO] systemd: Creating service file /etc/systemd/system/k3s.service [INFO] systemd: Enabling k3s unit Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service. [INFO] systemd: Starting k3s root@int:~# k3s kubectl get nodes NAME STATUS ROLES AGE VERSION int Ready master 28s v1.18.6+k3s1
Create a directory for local (stateful) storage
/data, for the purposes of lining up with our manifests.
Create a synapse Docker Image
First order of business is to create a docker image. We can do that by starting with an ubuntu image; there are good packages for it provided by the matrix.org folks that we can leverage.
In this image we do a few small things:
- First, we set the timezone and set the apt frontend to not prompt us during installation. We are going to muck with most of the configuration anyway, and the tzinfo package will prompt mid-build (and lock it up) if this is not set.
- Second, we get our keyring from matrix.org and install it and the package we need to run synapse.
- Finally, we set the command to not daemonize so it occupies PID 1, and point
it at our configuration, which will be in
- We do not copy any data into the image.
FROM ubuntu:latest ENV TZ=America/Los_Angeles DEBIAN_FRONTEND=noninteractive RUN apt-get update -qq && apt-get install -y lsb-release wget apt-transport-https RUN wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg RUN echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" | \ tee /etc/apt/sources.list.d/matrix-org.list RUN apt-get update -qq && apt-get install -y matrix-synapse-py3 CMD ["synctl", "--no-daemonize", "start", "/data/synapse/homeserver.yaml" ]
Building the image
Place the contents of the
Dockerfile in a file in an empty directory. Since
setting up a registry is beyond the scope for this article, we’ll assume our
images are to be published to Docker Hub. Let’s call
synapse, but the full name will be dependent on what account you
use on Docker Hub.
docker build -t erikh/synapse . && docker push erikh/synapse:latest
Extracting the Configuration & Generating a Signing Key
So, one thing that would be incredibly useful to you, is to have the latest annotated configuration file available for reading. Especially later; I will give you an abridged configuration file, you should be aware of all the options.
Additionally, you need to generate a fresh signing key for your chat server so it can perform encrypted operations with the clients. Please note that one will be in the image, as this is public (we just published it to docker hub!) you will not want to use it.
To do all this, we’ll bind mount
/tmp/config in the container to
outside of it, and then run our command that points everything at that
directory. The configuration and signing key will be in
/tmp/config on your
I know this command is a mouthful; but what it’s doing here is running our image, running the python in the virtualenv provided by the package we installed, then running our key/config generation tool to populate the directory. Since the directory is bind-mounted, we’ll see the changes on the host.
docker run \ -v /tmp/config:/tmp/config \ -it erikh/synapse \ /opt/venvs/matrix-synapse/bin/python \ -m synapse.app.homeserver \ --keys-directory /tmp/config --generate-keys \ --generate-config -H chat.foo.com \ --report-stats no \ -c /tmp/config/homeserver.yaml
You should have, among other things, a
homeserver.yaml and a
<domain>.signing.key in the newly created
/tmp/config directory in your
current working directory. This is your primary configuration and signing key.
Copy the key for later use.
Synapse has a LOT of configuration knobs and we’ll only be covering a small amount here; the ones that pertain to our infrastructure.
Here, we’re calling our server
chat.foo.com – users will point their
Element clients at this endpoint to talk to others. It is
not a federated server that talks to matrix.org or other matrix servers.
We’re also using Postgres and Redis here to solve some of the state management that is required by synapse.
Main Configuration File
This is only a very small part of the synapse configuration; please review the example set above you extracted from the image for a LOT more to configure!
Also, be sure to edit the parts that say
<shared secret>, as they need
generated passwords. For each field, try something like
apg to generate it:
apt-get install apg -y apg -a 1 -m 64
If you make any adjustments to the
listeners section, the Kubernetes service
load balancer manifest will also need to be edited. Same goes for changing any
of the paths, with regard to storage.
server_name: "chat.foo.com" public_baseurl: "https://chat.foo.com/" use_presence: true default_room_version: "6" limit_profile_requests_to_users_who_share_rooms: true pid_file: /var/run/homeserver.pid # NOTE: this is your TCP listener configuration. You'll want to # read the docs if you modify this portion in particular. listeners: - port: 8448 type: http tls: true x_forwarded: true resources: - names: [client] tls_certificate_path: "/data/synapse/tls/fullchain.pem" tls_private_key_path: "/data/synapse/tls/privkey.pem" acme: enabled: false send_federation: false federation_ip_range_blacklist: - "0.0.0.0/0" database: name: psycopg2 args: user: synapse password: synapse database: synapse host: postgres # connection pool parameters cp_min: 5 cp_max: 10 # NOTE: this is a separate configuration file; more on this below log_config: "/data/synapse/log.config" media_store_path: "/data/synapse/media" max_upload_size: 100M max_image_pixels: 32M enable_registration: true session_lifetime: 2w auto_join_rooms: - "#general:chat.foo.com" # NOTE: generate these with a tool like `apg` macaroon_secret_key: "<shared secret>" form_secret: "<shared secret>" signing_key_path: "/data/synapse/chat.foo.com.signing.key" key_refresh_interval: 1d trusted_key_servers:  push: include_content: true encryption_enabled_by_default_for_room_type: all enable_group_creation: true group_creation_prefix: "community/" user_directory: enabled: true search_all_users: false redis: enabled: true host: redis port: 6379 url_preview_enabled: true max_spider_size: 10M url_preview_accept_language: - en-US url_preview_ip_range_blacklist: - "127.0.0.0/8" - "10.0.0.0/8" - "172.16.0.0/12" - "192.168.0.0/16" - "100.64.0.0/10" - "169.254.0.0/16" - "::1/128" - "fe80::/64" - "fc00::/7" password_config: pepper: "<shared secret>" policy: enabled: true minimum_length: 8 require_digit: true require_symbol: true require_lowercase: true require_uppercase: true report_stats: false
Configuring the logger
We primarily want the logger to spit to stdout, saving our storage for other things. This configuration gets us mostly there.
version: 1 formatters: precise: format: "%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s" filters: context: (): synapse.logging.context.LoggingContextFilter request: "" handlers: console: class: logging.StreamHandler formatter: precise filters: [context] loggers: synapse.storage.SQL: # beware: increasing this to DEBUG will make synapse log sensitive # information such as access tokens. level: INFO root: level: INFO handlers: [console] disable_existing_loggers: false
Create a Certificate
If you have your own certs that were generated in some other way, just put them
certbot to generate our cert, and copy our files directly into the
/data/synapse/tls directory so they can be referenced. A better implementation
would be to embed these as disposable secrets.
Be sure to enable port 80 in your firewall!
apt install certbot -y certbot certonly -d chat.foo.com --standalone cd /etc/letsencrypt/live/chat.foo.com cp fullchain.pem privkey.pem /data/synapse/tls
Once all the files are in place, we can launch our set of services. I’ll spare
you the line-by-line; you should just be able to
kubectl apply -f this YAML
with one edit: to the public IP of the load balancer at the bottom of the text.
By this point, you should have:
- A docker image
- Configuration created
- Signing key generated & copied
- Certificates generated
- Firewall accepting port 443 (https) on the local IP.
/data tree should look something like:
/data/postgres /data/synapse /data/synapse/homeserver.yaml /data/synapse/log.config /data/synapse/chat.foo.com.signing.key /data/synapse/tls /data/synapse/tls/fullchain.pem /data/synapse/tls/privkey.pem
Edit and apply the configuration
You should not apply this verbatim. Edit it!
intshould be changed to your Node name
- IP in load balancer should be changed
- Images should be changed to use something other than mine! :D
The configuration creates a namespace called
matrix and stuffs a number of
details (pods, services, etc) underneath it. It also creates PVs for the mountpoints of
You can continue launching this likely get a working result! :D You can find a gist here for downloading.
Assuming you’ve accomplished all of the above; you should see this at the
https:// URL of your matrix server:
Congratulations! Point your Element client at it and have a blast!