Registry Cleanup[edit | edit source]

The docker registry does not remove deleted images. Like a bad memory leak, the registry will grow as old images are still stored even when dereferenced and deleted.

Use the docker-distribution-pruner utility which can be obtained from their project CI / CD page:

On the GitLab instance, run the following:

## This will soft-delete the files to /var/opt/gitlab/gitlab-rails/shared/registry/docker-backup
# EXPERIMENTAL=true ./docker-distribution-pruner -config=/var/opt/gitlab/registry/config.yml -delete
INFO[0000] Walking REPOSITORIES...
INFO[0000] REPOSITORIES DIR: repositories
INFO[0000] Walking BLOBS...
INFO[0001] BLOBS INFO: Objects/Unused: 296 / 75 Data/Unused: 3.0 GB / 398 MB
WARN[0001] DELETEABLE INFO: 115 links, 75 blobs, 0 other, 398 MB

## Remove the files
# rm -rf /var/opt/gitlab/gitlab-rails/shared/registry/docker-backup

Docker Registry Runners[edit | edit source]

# docker-compose run --rm gitlab-runner register -n --url --registration-token z276ujRTK69qD8xb-aNH --executor docker --docker-image "docker:stable"
Runtime platform                                    arch=amd64 os=linux pid=7 revision=3afdaba6 version=11.5.0
Running in system-mode.

Registering runner... succeeded                     runner=z276ujRT
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

root@docker:/root/docker/running/gitlab# cat /var/volumes/gitlab/runner-config/config.toml
concurrent = 1
check_interval = 0

  session_timeout = 1800

  name = "3b70de49f826"
  url = ""
  token = "1dbe4535e5aaa49d6582d374a29c28"
  executor = "docker"
    tls_verify = false
    image = "docker:stable"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0

root@docker:/root/docker/running/gitlab# cat docker-compose.yml /root/docker/running/gitla
version: '3.3'


    #  context: ./gogs
    #  dockerfile: Dockerfile
    # letsencrypt['enable'] = false
    image: gitlab/gitlab-ce:latest
    container_name: gitlab
        external_url ''
        nginx['listen_port'] = '80'
        nginx['listen_https'] = false
        nginx['proxy_set_headers'] = { 'X-Forwarded-Proto' => 'https', 'X-Forwarded-Ssl' => 'on' }
        registry_external_url ''
        registry['enable'] = true
        gitlab_rails['registry_enabled'] = true
        registry_nginx['listen_port'] = 5001
        registry_nginx['listen_https'] = false
        registry_nginx['proxy_set_headers'] = { 'X-Forwarded-Proto' => 'https', 'X-Forwarded-Ssl' => 'on' }
      - /var/volumes/gitlab/config:/etc/gitlab
      - /var/volumes/gitlab/logs:/var/log/gitlab
      - /var/volumes/gitlab/data:/var/opt/gitlab
    restart: always
      - "80"
      - "5001"
      - ""
      - "traefik.enable=true"
      - ""
      - "traefik.web.port=80"
      - ""
      - "traefik.registry.port=5001"
      - ""
      - traefik
      disable: true

    image: gitlab/gitlab-runner:latest
    container_name: gitlab-runner
    restart: always
      - /var/volumes/gitlab/runner-config:/etc/gitlab-runner
      - /var/run/docker.sock:/run/docker.sock

      name: traefik

Kubernetes GitLab Runner[edit | edit source]

To install a GitLab runner with Helm, install the GitLab repo and then the helm chart. You might want to use an existing values.yaml file from as a reference.

# helm repo add gitlab
# helm install \
        --namespace gitlab \
        --name gitlab-runner \
        -f values.yaml \
        --set gitlabUrl=,runnerRegistrationToken=xxxxxxxxxxx \

Kubernetes GitLab[edit | edit source]

# helm repo add gitlab
# helm install gitlab/gitlab --name=gitlab --namespace=default --set global.operator.enabled=true --set global.operator.bootstrap=true -f gitlab-config.yml  --debug
# kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

Configuration files are set inside ConfigMaps. The unicorn/workhorse pod used to drive the GitLab web application gets its configs from the gitlab-unicorn ConfigMap.

Troubleshooting[edit | edit source]

Runner container crashes with a fatal error message:

FATAL: Failed to create listener for metrics server  builds=0 error=listen tcp [::]:9252: socket: address family not supported by protocol

To fix, set in values.yaml:

    value: :9252

Backups[edit | edit source]

GitLab using the Helm Chart can be backed up by running backup-utility inside the GitLab runner container. Backups that are generated will be placed at /srv/gitlab/tmp/backup as defined by the backup.path parameter in /srv/gitlab/config/gitlab.yml.

Omnibus GitLab can be backed up by scheduling a task that runs /opt/gitlab/bin/gitlab-rake gitlab:backup:create. The destination is configured based on the gitlab_rails['backup_path'] value in /etc/gitlab/gitlab.rb. Specific components can be skipped by passing in the SKIP value. For example:

# /opt/gitlab/bin/gitlab-rake gitlab:backup:create SKIP=registry,pages,artifacts,builds  DIRECTORY=daily

See Also:

To restore, run gitlab-rake gitlab:backup:restore BACKUP=1571687885_2019_10_21_12.3.5-ee, to target a backup file in the backup directory named 1571687885_2019_10_21_12.3.5-ee_gitlab_backup.tar.

git@gitlab-task-runner-6c64cd44f8-wvjb4:/srv/gitlab/tmp/backups$ gitlab-rake gitlab:backup:restore BACKUP=1571687885_2019_10_21_12.3.5-ee
WARNING: This version of GitLab depends on gitlab-shell 10.0.0, but you're running Unknown. Please update gitlab-shell.
Unpacking backup ...