In order properly build go software that depends on libraries in private gitlab repositories you’ll have to jump through some hoops. The following instructions are known to work as of 2019-12-14, I’ll try to keep this post updated in the future if needed.

Step 1. First of all we’ll want to create a new ssh keypair specifically for this usecase. I personally would startup a quick container for this using docker or podman, but this is also very easy to do on your normal host system. Simply execute ssh-keygen and follow the instructions (Make sure to NOT overwrite your usual ssh keys!!) and make sure to not set a password for this keypair.

Step 2. With this brand new ssh keypair we’ll now head over to gitlab. First of all we’ll want to go towards the private repository that we want to include and essentially retrieve in a CI pipeline through go get. Here we’ll have to go to Settings -> Repository -> Deploy Keys, here simply add the public key of the just created SSH keypair. It is probably wise to not enabled write access as read only access is more than enough for this usecase. I would however suggest giving this a proper title so you’ll know in the future what this key is for, name it something like “GitlabCI go get key”. In case you have multiple private libraries that you’re including you’ll have to do this with every repository. However Gitlab is nice to use and saved this key, meaning that you won’t have to copy paste it everytime and can just simply enabled it on other projects.

Step 3. We’ve finally come to the actual Gitlab CI stage. First of all we’ll have to add our private ssh key we generated in step 1 to our CI setup. In order to do this we’ll first head to the project we’re trying to build that has to include private repositories. Then simply head to Settings -> CI/CD -> Variables. Over here we’ll create a new variable. First of all select type ‘File’, name the key ‘SSH_PRIVATE_KEY_FILE’ and paste the contents of the private key file in the value field. Make sure this starts with -----BEGIN OPENSSH PRIVATE KEY----- and ends with -----END OPENSSH PRIVATE KEY-----. After this put one new line after -----END OPENSSH PRIVATE KEY----- otherwise the next step won’t work. Simply save these settings and continue with the next step, sadly there is no easy way to copy paste these settings to other projects like at step 2. Meaning you’ll have to repeat this step for every new project that needs to import go libraries from private repositories.

Step 4. Ok I lied, we only now actually got to the Gitlab CI stage. Add the following snippet to your .gitlab-ci.yml, I’ll explain afterwards what every command does.

.configure_private_repo: &configure_private_repo
    before_script:
        - eval $(ssh-agent -s)
        - chmod 700 $SSH_PRIVATE_KEY_FILE && ssh-add $SSH_PRIVATE_KEY_FILE
        - mkdir -p ~/.ssh && ssh-keyscan -p 22 gitlab.com | tee ~/.ssh/known_hosts
        - git config --global url."ssh://git@gitlab.com:22/".insteadOf https://gitlab.com

Basically we just created a yaml anchor that we can easily import in other steps of our CI. First of all we’ll start the ssh-agent. Next we’ll actually import the private key that we created in step 1 and attached to the SSH_PRIVATE_KEY_FILE variable in step 3. Afterwards we’ll scan and add the ssh host keys of gitlab.com so we won’t have to run with -o StrictHostKeyChecking=no. And lastly we’ll reconfigure git to replace all instances of https://gitlab.com with ssh://git@gitlab.com:22/ basically forcing every git clone to go over ssh even though an url contains https. In case of a selfhosted gitlab it should now be quite clear what you’ll have to replace with your own domain and/or ports. Now to finish it off we’ll have to slightly modify our existing jobs by merging the previously declared configure_private_repo anchor into it, an example of this below.

build:
    stage: build
    <<: *configure_private_repo
    script:
        - go build ./...

Because of the <<: *configure_private_repo it’ll merge the before_script previously declared into this block. Simply do this for every job that needs to do go get in the background and you should be good to go.