Commit e3ff4101 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files

Merge branch '2799-mysql-to-postgresql-docs' into 'master'

Document how to use pgloader

Closes gitlab-ee#2799

See merge request gitlab-org/gitlab-ce!13109
*** NOTE: These instructions should be considered deprecated. In GitLab 10.0 we will be releasing new migration instructions using [pgloader](http://pgloader.io/).
---
last_updated: 2017-10-05
---
# Migrating GitLab from MySQL to Postgres
*Make sure you view this [guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/mysql_to_postgresql.md#migrating-gitlab-from-mysql-to-postgres) for the most up to date instructions.*
# Migrating from MySQL to PostgreSQL
If you are replacing MySQL with Postgres while keeping GitLab on the same server all you need to do is to export from MySQL, convert the resulting SQL file, and import it into Postgres. If you are also moving GitLab to another server, or if you are switching to omnibus-gitlab, you may want to use a GitLab backup file. The second part of this documents explains the procedure to do this.
> **Note:** This guide assumes you have a working Omnibus GitLab instance with
> MySQL and want to migrate to bundled PostgreSQL database.
## Export from MySQL and import into Postgres
## Prerequisites
Use this if you are keeping GitLab on the same server.
First, we'll need to enable the bundled PostgreSQL database with up-to-date
schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data
from the old MySQL database to the new PostgreSQL one.
```
sudo service gitlab stop
Here's what you'll need to have installed:
# Update /home/git/gitlab/config/database.yml
- pgloader 3.4.1+
- Omnibus GitLab
- MySQL
git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab
cd mysql-postgresql-converter
mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p
python db_converter.py gitlabhq_production.mysql gitlabhq_production.psql
ed -s gitlabhq_production.psql < move_drop_indexes.ed
## Enable bundled PostgreSQL database
# Import the database dump as the application database user
sudo -u git psql -f gitlabhq_production.psql -d gitlabhq_production
1. Stop GitLab:
# Install gems for PostgreSQL (note: the line below states '--without ... mysql')
sudo -u git -H bundle install --without development test mysql --deployment
``` bash
sudo gitlab-ctl stop
```
sudo service gitlab start
```
1. Edit `/etc/gitlab/gitlab.rb` to enable bundled PostgreSQL:
## Converting a GitLab backup file from MySQL to Postgres
**Note:** Please make sure to have Python 2.7.x (or higher) installed.
```
postgresql['enable'] = true
```
GitLab backup files (`<timestamp>_gitlab_backup.tar`) contain a SQL dump. Using the lanyrd database converter we can replace a MySQL database dump inside the tar file with a Postgres database dump. This can be useful if you are moving to another server.
1. Edit `/etc/gitlab/gitlab.rb` to use the bundled PostgreSQL. Please check
all the settings beginning with `db_`, such as `gitlab_rails['db_adapter']`
and alike. You could just comment all of them out so that we'll just use
the defaults.
```
# Stop GitLab
sudo service gitlab stop
1. [Reconfigure GitLab] for the changes to take effect:
``` bash
sudo gitlab-ctl reconfigure
```
1. Start Unicorn and PostgreSQL so that we could prepare the schema:
``` bash
sudo gitlab-ctl start unicorn
sudo gitlab-ctl start posgresql
```
1. Run the following commands to prepare the schema:
``` bash
sudo gitlab-rake db:create db:migrate
```
1. Stop Unicorn in case it's interfering the next step:
# Create the backup
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
``` bash
sudo gitlab-ctl stop unicorn
```
# Note the filename of the backup that was created. We will call it
# TIMESTAMP_gitlab_backup.tar below.
After these steps, you'll have a fresh PostgreSQL database with up-to-date schema.
# Move the backup file we will convert to its own directory
sudo -u git -H mkdir -p tmp/backups/postgresql
sudo -u git -H mv tmp/backups/TIMESTAMP_gitlab_backup.tar tmp/backups/postgresql/
## Migrate data from MySQL to PostgreSQL
# Create a separate database dump with PostgreSQL compatibility
cd tmp/backups/postgresql
sudo -u git -H mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p
Now, you can use pgloader to migrate the data from MySQL to PostgreSQL:
# Clone the database converter
sudo -u git -H git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab
1. Save the following snippet in a `commands.load` file, and edit with your
database `username`, `password` and `host`:
# Convert gitlabhq_production.mysql
sudo -u git -H mkdir db
sudo -u git -H python mysql-postgresql-converter/db_converter.py gitlabhq_production.mysql db/database.sql
sudo -u git -H ed -s db/database.sql < mysql-postgresql-converter/move_drop_indexes.ed
```
LOAD DATABASE
FROM mysql://username:password@host/gitlabhq_production
INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production
# Compress database backup
# Warning: If you have Gitlab 7.12.0 or older skip this step and import the database.sql directly into the backup with:
# sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql
# The compressed databasedump is not supported at 7.12.0 and older.
sudo -u git -H gzip db/database.sql
WITH include no drop, truncate, disable triggers, create no tables,
create no indexes, preserve index names, no foreign keys,
data only
# Replace the MySQL dump in TIMESTAMP_gitlab_backup.tar.
ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'
# Warning: if you forget to replace TIMESTAMP below, tar will create a new file
# 'TIMESTAMP_gitlab_backup.tar' without giving an error.
;
```
sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql.gz
1. Start the migration:
# Done! TIMESTAMP_gitlab_backup.tar can now be restored into a Postgres GitLab
# installation.
# See https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md for more information about backups.
``` bash
sudo -u gitlab-psql pgloader commands.load
```
1. Once the migration finishes, start GitLab:
``` bash
sudo gitlab-ctl start
```
Now, you can verify that everything worked by visiting GitLab.
## Troubleshooting
### Experiencing 500 errors after the migration
If you experience 500 errors after the migration, try to clear the cache:
``` bash
sudo gitlab-rake cache:clear
```
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment