Enterprise
This guide will cover how to run DoltLab in Enterprise mode and use exclusive features not covered in the Administrator's Guide.
To start DoltLab in Enterprise mode, when running the installer supply the following additional arguments:
The values for these arguments will be provided to you by our DoltLab team. The following contents on this page covers how to configure various Enterprise features for your DoltLab instance.
Use custom logo on DoltLab instance
DoltLab Enterprise allows administrators to customize the logo used across their DoltLab instance. At the time of this writing, custom logos custom logos must have a maximum height of 24px
and a maximum width of 112px
. If a custom logo is used on DoltLab, the footer of the DoltLab instance will display the text "Powered by DoltLab" below the custom logo.
You can use a custom logo on DoltLab by running the installer
with the argument --custom-logo=/path/to/custom/logo.png
.
Customize automated emails
DoltLab Enterprise allows administrators to customize the automated emails their DoltLab instance sends to its users.
Custom emails can be configured with the installer
by supplying the argument --custom-email-templates=true
. The installer will generate the email template files at ./doltlabapi/templates/email
which match the files described below. You can customize these files and they will be used by DoltLab. Each file is named according to use-case. The names and paths of these files should NOT be changed.
collabInvite.txt
sent to invite user to be a database collaborator.invite.txt
sent to invite a user to join an organization.issueComment.txt
sent to notify user that an issue received a comment.issueState.txt
sent to notify user that an issue's state has changed.pullComment.txt
sent to notify user that a pull request received a comment.pullCommits.txt
sent to notify user that a pull request received a commit.pullReview.txt
sent to notify user that a pull request review's state has changed.pullState.txt
sent to notify user that a pull request's state has changed.recoverPassword.txt
sent to provide user with a link to reset their password.resetPassword.txt
sent to notify a user that their password has been reset.verify.txt
sent to a user to verify their email account.
To alter the text within one of the above files, we recommend only changing the hardcoded text between the Actions and replacing the use of {{.App}}
, which normally evaluates to "DoltLab", with the name of your company or team.
You should not change any template definitions, indicated with {{define "some-template-name"}}
syntax, within these files as doltlabapi
relies on these specific definitions.
To better illustrate how to modify these files, let's look at an example. Here is the default verify.txt
template:
Above, three templates are defined verifySubject
, verifyHTML
, and verifyText
. We will not add or remove any of these templates and we won't change their names, but we will replace the {{.App}}
field with the name of our company, Acme, Inc.'s DoltLab instance, "AcmeLab". We'll also modify the hardcoded text to be specific to our DoltLab instance's users.
After replacing {{.App}}
with "AcmeLab", our file looks like:
Lastly, let's customize this email with the contact information of our AcmeLab admin, in case users have any questions. We want to add the same information to the verifyHTML
template and the verifyText
template so that it appears for either supported email format:
Once we save our edits, we can restart our DoltLab instance for the changes to take affect.
Customize DoltLab colors
DoltLab Enterprise allows administrators to customize the color of certain assets across their DoltLab instance.
For configuring custom colors, the installer
accepts the following arguments corresponding to the custom color you want to override:
Add Super Admins to a DoltLab instance
DoltLab Enterprise allows administrators to specify users who will be "super admins" on their DoltLab instance.
A DoltLab "super admin" is a user granted unrestricted access and the highest possibly permission level on all organizations, teams, and databases on a DoltLab instance. This allows these users to write to any database, public or private, merge pull-requests, delete databases and add or remove organization/team members. By default there are no "super admins" registered on a DoltLab instance, including the default user admin
.
Super admins can be configured using the installer
with the argument --super-admin-email
. This argument can be supplied multiple times, for example:
Configure SAML Single-Sign-On
DoltLab Enterprise supports SAML single-sign-on. To configure your DoltLab instance to use single-sign-on, you will first need an Identity Provider (IP) to provide you with a metadata descriptor.
For example, Okta, a popular IP, provides an endpoint for downloading the metadata descriptor for a SAML application after you register an application on their platform.
During registration, Okta will ask you for the "Single Sign On Url" and an "Audience Restriction" for the application.
Use the domain/host IP address of your DoltLab instance followed by /sso/callback
for the "Single Sign On Url", and use that same domain/host IP address followed by just "/sso" for the "Audience Restriction". Since this example will be for https://doltlab.dolthub.com
, we'll use https://doltlab.dolthub.com/sso/callback
and https://doltlab.dolthub.com/sso
respectively.
Be sure to also set "Name ID Format" to "Persistent".
Then, download the metadata Okta provides for this application to your DoltLab host.
Next, run the installer
with following arguments included:
When SAML single-sign-on is configured, you will see the SAML option on the sign-in page:
Next, as user admin
, login to your DoltLab instance and navigate to Profile
> Settings
> SSO
.
On this tab you will see the following:
Assertion Consumer Service Url
displays the url where Okta should send the SAML assertion.
Entity ID/Login Url
displays the url users can use to login to DoltLab using the IP, but they can now simply use the option available on the sign-in page.
IP Metadata Descriptor
is a metadata descriptor for this DoltLab instance, and can be downloaded and supplied to the IP if it requires service providers to upload metadata.
Certificate
can be downloaded if you want to add a signature certificate to the IP to verify the digital signatures.
Your Enterprise instance will now use single-sign-on through your IP for user login and account creation.
Automated Remote Backups
DoltLab Enterprise supports automated database backups for DoltLab's application Dolt server. To backup database data of all the Dolt databases hosted on your DoltLab instance, we recommend taking regular snapshots of the host's filesystem.
To configure your DoltLab instance to automatically back up its Dolt database server, first, provision either a GCP bucket or and AWS S3 bucket and Dynamo DB table. You will need these to resources to create a remote backup. Oracle Cloud Infrastucture (OCI) storage buckets may be used as well.
Dolt supports a backup command which can be used to create backups of a Dolt instance.
Let's walk through setting up automated backups using an AWS remote backup first.
AWS Remote Backup
Dolt can use an AWS Remote as a backup destination, but requires that two resources be provisioned. As stated in this helpful blog post, "AWS remotes use a combination of Dynamo DB and S3. The Dynamo table can be created with any name but must have a primary key with the name db
."
For our example, let's create an AWS S3 bucket called test-doltlab-application-db-backups
.
Let's also create a Dynamo DB table in the same AWS region, and call it test-doltlab-backup-application-db-manifest
. Notice its uses the required partition key (primary key) db
.
The AWS remote url for our DoltLab instance which is determined by the template aws://[dolt_dynamo_table:dolt_remotes_s3_storage]/backup_name
, will be aws://[test-doltlab-backup-application-db-manifest:test-doltlab-application-db-backups]/my_doltlab_backup
.
We've also granted read and write access for these resources to an IAM role called DoltLabBackuper
.
It's now time to update our DoltLab instance configuration to automatically backup it's Dolt server data to our AWS remote.
First, ensure that the AWS credentials on the DoltLab host can be used to assume the role DoltLabBackuper
. Create a AWS config file that contains:
Then use the AWS CLI to confirm this profile can be used on your DoltLab host:
Next, run the installer
with the following arguments to configure the AWS backup:
DoltLab will use a combination of Prometheus and Alertmanager to notify you if your regularly scheduled backup fails for some reason. You'll need to edit the Alertmanager configuration file generated by the installer
at ./alertmanager/alertmanager.yaml
and include your SMTP authentication information in the global
section. The other sections do not need to be edited:
For more configuration options, please consult the AlertManager documentaion.
Finally, start DoltLab using the ./start.sh
script. DoltLab will create the first backup when started, and by default, will create backups at midnight each night. You will see your backups stored in your S3 bucket, and the manifest stored in your DynamoDB table.
Your DoltLab's Dolt server is now automatically backing up to your AWS remote.
GCP Remote Backup
To backup DoltLab's Dolt server to a GCP remote, first create a bucket in GCP. This will be the only required resource needed.
Next, add GCP JSON credentials to your DoltLab host. You can find information about GCP credentials here.
Following the Dolt's url template for GCP remotes as outlined in this blog, the remote url we will use for this bucket will be gs://test-doltlab-application-db-backup/my_doltlab_backup
.
Run the installer
with the following arguments to create automated GCP backups:
Finally, edit the ./alertmanager/alertmanager.yaml
file generated by the installer
, as shown in the AWS backups section, to receive notifications of backup failures.
Once you start your Enterprise instance with ./start.sh
, it will now automatically back up its application Dolt server to your GCP bucket.
OCI Remote Backup
To backup DoltLab's Dolt server to an OCI remote, first create a bucket in OCI. This will be the only required resource needed.
Next, install the oci
CLI tool on your DoltLab host, and run oci setup config
to create a configuration file with credentials authorized to access your bucket. You can find information about creating an config file here.
oci setup config
will create a config file and private key file that you will then need to mount into the doltlabdb
container.
First, edit the generated config file so that the key_file
field contains the absolute path of where the generate key file will be mounted in the doltlabdb
container.
In the above example, we've changed key_file
to point to /oci_private_key.pem
, where DoltLab will mount the private key file. Save these changes.
Following the Dolt's url template for OCI remotes as outlined in this blog, the remote url we will use for this bucket will be oci://test-doltlab-application-db-backup/my_doltlab_backup
.
Next, run the installer
with the following arguments to configure the OCI backups:
Finally, edit the ./alertmanager/alertmanager.yaml
file generated by the installer
, as shown in the AWS backups section, to receive notifications of backup failures.
Once you start your Enterprise instance with ./start.sh
, it will now automatically back up its application Dolt server to your OCI bucket.
Deploy DoltLab across multiple hosts
DoltLab's services can be deployed across multiple hosts which allow DoltLab's services to be scaled independently.
The following guide will walkthrough deploying a DoltLab instance whose set of services each run on separate host machines.
The diagram above depicts the multi-host architecture for DoltLab. Each independent service runs on a distinct host, and is served behind it's own reverse proxy. Both the service and the proxy run via Docker compose, and are easily configured using the installer
. At the time of this writing, multi-host deployments are only available over http
. For https
support, please file an issue in our issues repository.
To get started with a DoltLab multi-host deployment, you'll need to provision a host per DoltLab service. Here are our hardware recommendations for each service:
doltlabdb
, DoltLab's Dolt database, requires a host with at least 2 CPU, 4GB of memory, and 15GB of disk.doltlabfileserviceapi
, DoltLab's service for managing user uploads, requires a host with at least 2 CPU, 4GB of memory, and 50GB of disk.doltlabremoteapi
, DoltLab's service for managing database data, requires a host with at least 2 CPU, 4GB of memory, and 200GB of disk.doltlabapi
, DoltLab's main API, requires a host with at least 4 CPU, 16GB of memory, and 50GB of disk.doltlabgraphql
, DoltLab's data transformation layer, requires a host with at least 2 CPU, 4GB of memory, and 10GB of disk.doltlabui
, DoltLab's frontend, requires a host with at least 2 CPU, 4GB of memory, and 10GB of disk.
When provisioning the hosts for each service, change the networking settings on each host so that the ports specified below are open for ingress. You will also need to ensure that each host has a publicly reachable IP address:
doltlabdb
, port3306
.doltlabfileserviceapi
, port4321
.doltlabremoteapi
, ports50051
and100
.doltlabapi
, ports9443
and9444
.doltlabgraphql
, port9000
.doltlabui
, port80
.
Before continuing you will need to document the public IP of each host associated with each DoltLab service, as it will be referenced in different places throughout this process.
For this walkthrough I will use the following IP's for each DoltLab service:
doltlabdb
,52.43.136.146
.doltlabfileserviceapi
,34.221.204.184
.doltlabremoteapi
,34.222.48.69
.doltlabapi
,35.91.149.175
.doltlabgraphql
,35.91.70.84
.doltlabui
,35.94.142.32
.
Once the hosts are provisioned and running, you will need to download the latest version of DoltLab on each host, and install unzip
:
You can then unzip DoltLab's contents and install it's dependencies using the installation script generated by the installer
.
Again, you should complete this process on each host before moving on to the subsequent steps.
Next, we will use the installer
to configure the separate services on each host in the following order:
doltlabdb
doltlabapi
doltlabremoteapi
doltlabfileserviceapi
doltlabgraphql
doltlabui
doltlabdb
Starting with your doltlabdb
host and from within the unzipped doltlab
directory, ensure that Docker can run without the sudo
command by running:
Next, run the installer
with the --doltlabdb-only
argument. Be sure to also supply the arguments for DoltLab Enterprise mode, as those are required as well.
This will produce output like the following:
Running the installer
will generate a password for the dolthubapi
user of DoltLab's application database. DoltLab's main API will need to connect to the application database as this user, so you will need to make note of this generated password.
You can find the generated password at ./secrets/dolt_dolthubapi_password.priv
:
Now start doltlabdb
by running ./start.sh
:
You can see the running services by running docker ps
:
doltlabapi
Now connect to your doltlabapi
host, cd
into the doltlab
directory, and ensure you can use docker
without sudo
by running:
Run the installer
with the --doltlabapi-only
flag and the other required arguments in order to configure the doltlabapi
instance.
--host
, is required and should be the IP address of the doltlabapi
host.
--doltlabdb-host
is required and is the IP address of the doltlabdb
host.
--doltlabdb-port
, is required and should be the value 3306
. This port value should not be changed.
--doltlabremoteapi-host
, is required and is the IP address of the doltlabremoteapi
host.
--doltlabremoteapi-port
, is required and should be the value 50051
. This port value should not be changed.
--doltlabremoteapi-file-server-port
, is required and should be the value 100
. This port value should not be changed.
--doltlabfileserviceapi-host
, is required and is the IP address of the doltlabfileserviceapi
host.
--doltlabfileserviceapi-port
, is required and should be the value 4321
. This port value should not be changed.
--doltlabui-host
, is required and is the IP address of the doltlabui
host.
--doltlabui-port
, is required and should be the value 80
. This port value should not be changed.
--doltlabdb-dolthubapi-password
, is required and should be dolthubapi
password generated by the installer
on the doltlabdb
host.
--smtp-host
, is optional and is the host name of an SMTP server. It is only required if users other than admin
will be using the DoltLab instance. See connecting DoltLab to an SMTP server for more information.
--smtp-port
, is optional and is the port of an SMTP server. It is only required if users other than admin
will be using the DoltLab instance. See connecting DoltLab to an SMTP server for more information.
--smtp-auth-method
, is optional and is the authentication method supported by the SMTP server. It is only required if users other than admin
will be using the DoltLab instance. See connecting DoltLab to an SMTP server for more information.
--smtp-username
, is required for authentication method plain
, and is the username used to connect to the SMTP server.
--smtp-password
, is required for authentication method plain
, and is the password used to connect to the SMTP server.
--no-reply-email
, is optional and is the email used to send automated DoltLab emails. It is only required if users other that admin
will be using the DoltLab instance. See connecting DoltLab to an SMTP server for more information.
--default-user-email
, is optional and is the email address to associate with the default user admin
.
After running the installer
, you will see output like the following:
The installer
will tell you how to find the default user password so it can be used to login to the DoltLab instance once all services are deployed.
You can now run ./start.sh
:
Running docker ps
will show the running services:
doltremoteapi
Now connect to your doltlabremoteapi
host, cd
into the doltlab
directory, and ensure you can use docker
without sudo
by running:
Run the installer
with the --doltlabremoteapi-only
flag and the other required arguments in order to configure the doltlabremoteapi
instance.
--host
, is required and should be the IP address of the doltlabremoteapi
host.
--doltlabapi-host
, is required and should be the IP address of the doltlabapi
host.
--doltlabapi-port
, is required and the value should be 9443
. This port value should not be changed.
You can now run ./start.sh
:
Running docker ps
will show the running services:
doltlabfileserviceapi
Now connect to your doltlabfileserviceapi
host, cd
into the doltlab
directory, and ensure you can use docker
without sudo
by running:
Run the installer
with the --doltlabfileserviceapi-only
flag and the other required arguments in order to configure the doltlabfileserviceapi
instance.
--doltlabapi-host
, is required and should be the IP address of the doltlabapi
host. --doltlabapi-port
, is required and should be the value 9443
. This port value should not be changed. --doltlabremoteapi-host
, is required and should be the IP address of the doltlabremoteapi
host. --doltlabremoteapi-port
, is required and should be the value 50051
. This port value should not be changed. --doltlabui-host
, is required and should be the IP address of the doltlabui
host.
You can now run ./start.sh
:
Running docker ps
will show the running services:
doltlabgraphql
Now connect to your doltlabgraphql
host, cd
into the doltlab
directory, and ensure you can use docker
without sudo
by running:
Run the installer
with the --doltlabgraphql-only
flag and the other required arguments in order to configure the doltlabgraphql
instance.
--doltlabapi-host
, is required and should be the IP address of the doltlabapi
host. --doltlabapi-port
, is required and should be the value 9443
. This port value should not be changed.
You can now run ./start.sh
:
Running docker ps
will show the running services:
doltlabui
Now connect to your doltlabui
host, cd
into the doltlab
directory, and ensure you can use docker
without sudo
by running:
Run the installer
with the --doltlabui-only
flag and the other required arguments in order to configure the doltlabui
instance.
--doltlabapi-host
, is required and should be the IP address of the doltlabapi
host. --doltlab-csv-port
, is required and should be the value 9444
. This port value should not be changed. --doltlabgrapqhl-port
, is required and should be the value 9000
. This port value should not be changed. --doltlabgraphql-host
, is required and should be the IP address of the doltlabgraphql
host.
You can now run ./start.sh
:
Running docker ps
will show the running services:
You are now running DoltLab in multi-host deployment configuration. To view the homepage of your DoltLab instance, navigate to http://${HOST_IP}
, where HOST_IP
is the IP address of the doltlabui
host.
Last updated