by Romano Roth and Patrick Steger
This video series will show you how to build up an enterprise-ready DevSecOps Pipeline with GitLab and GitHub and compare the two platforms.
Introduction#
GitLab
GitHub
| Topic | GitLab | GitHub |
|---|---|---|
| Security Tools | Provides integration points to include security tools. Offers GitLab maintained integrations for GitLab recommended default security tools. | Provides integration points to include security tools. There are no default GitHub recommended/maintained security tools. There are security tools provided by the community for open source and commercial tools. |
Creating a simple project#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Repository Creation | Can create new empty repository. Offers to start with GitLab provided default project/repo-structure (e.g. C#, SpringBoot, PHP, etc). They are easy to find. | Can create new empty repository. Can create a new repository based on a template project. Template projects are hard to find when there are none in your organization. |
| Import | Can import from other repositories like any Git repository (URL), GitHub, Jira, Bitbucket, etc. | Can import from existing Repositories. Including GitLab. But does not work when using Social-Login. |
| Pipeline Definition | .gitlab-ci.yml is the entry point defining the whole pipeline. Option to include additional files (their content is included automatically at runtime). | Pipelines are called Workflows. We can have a dedicated file for each workflow we want. Within the workflow file we define what triggers the workflow/pipeline. It is possible to “call” sub-workflows from a workflow file. |
| Pipeline Structure | Pipeline is broken down into stages and jobs, where stages are executed after each other and all jobs of a stage run in parallel (dependencies can be configured). This is a very neat approach but can become tricky to understand for complex pipelines. | Pipelines/Workflows are cleanly separated in the UI and make it easy to find the right workflow/pipeline. Workflows use actions to implement jobs/steps. Such Actions are available over a Marketplace and not always easy to be found. |
| Ease of Start | Very easy to create a simple first pipeline because GitLab provides defaults for most everything that is required to start. | Quite complex to get the first simple pipeline running because not much is automatically provided by GitHub. |
| Pipeline Runs | Pipeline runs are found in menu CI/CD -> pipelines. | Pipeline/Workflow runs are found cleanly listed by Workflow in tab “Actions” in the UI. |
| Artifacts | Artifacts can simply be declared and are then available in the pipeline results view for download, this is GitLab provided and well documented. Standard artifacts like test-results can be made visible in the GitLab UI using default GitLab functionality. | Artifacts are made available using an Action (for example using upload-artifact Action found in the Marketplace). Standard artifacts like test-results require other actions from the marketplace. UI quality varies. |
| Multiple Pipelines | Different pipelines must be configured with rules (e.g. based on what branch was committed). This may become complicated for complex scenarios. | Different pipelines are simply configured in extra workflow files. This becomes natural after some time and supports complex scenarios. |
| Visualization | Nice graphical view to visualize stages and jobs as well as their dependencies (but not the conditions). | Nice graphical view to visualize the workflow/pipeline run for executed runs. |
Software Composition Analysis (SCA)#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Default Tool | GitLab provided open source scanner available, documented and easily found. Simply add a GitLab provided template. | No GitHub provided default tool. Find one in the Marketplace. We chose CRDA of RedHat based on Snyk (which proved to be not enterprise ready). |
| Tool Quality | The default tool has adequate quality and findings and is enterprise ready. | Marketplace tools (Actions) are provided by third parties. Be aware of the risk and consider what providers you trust. |
| Integration | Findings are well integrated in GitLab UI (Pipeline view, Security&Compliance -> Vulnerability report). Vulnerabilities are also available as a download (json). | GitHub has built-in functionality called Dependabot that can be enabled in settings and allows regular, scheduled scans of unchanged code. |
| Format | GitLab uses a GitLab specific format to integrate vulnerabilities in the GitLab UI. | The GitHub format to display vulnerabilities is based on standards. This proves to be better for custom integration. |
| Scheduled Scans | No built-in scheduled re-scans, but can be defined explicitly. | Dependabot can alert of new findings and offer pull requests for known fixes. But it is not executed on commit and cannot block merges. |
| Special Feature | Comparable simple editor of files in the project. | Fully integrated development environment (similar to Visual Studio Code). This proved to be very powerful. |
License Compliance#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Tool | License compliance is provided as a managed GitLab tool (based on License-Finder open source project) that is easy and straightforward to add. | Requires community provided (Marketplace) or custom Action to implement this. It’s meaningful to define an extra sub-workflow for this. |
| Configuration | All used licenses of the project are visible in a default GitLab UI. Can configure acceptable licenses as part of the platform (project settings). | Can define what licenses are configurable, get error/failed pipeline when other licenses are found. |
| Workflow | Support of basic workflows to verify and accept new licenses (i.e. include a security team member as acceptor/reviewer). | Failed licenses are reported as “customized Test Results”. Results are visible in the pipeline run only. |
| Visibility | Failed (not marked as acceptable) licenses are reported. | There is no easy way to see “all licenses used by the project” (this is only in the log-output of the pipeline run). |
Static Application Security Testing (SAST)#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Default Tool | GitLab provided standard tool based on semgrep and other open source tools (e.g. spotbugs) available. Easy to integrate with a few lines of configuration. | GitHub provides a standard tool for static analysis (CodeQL). We added in addition the open source tools semgrep and spotbugs. |
| Quality | GitLab added extra rules to the open source tools, so that it finds way more security issues than the plain open source semgrep tool. Overall findings look good enough. | The combined results of all three tools still miss important findings. Custom work (i.e. defining custom rules) or extra licenses are required to improve. |
| Marketplace | N/A | Finding appropriate SAST tools (Actions) in the Marketplace is not straight forward. Many tools require licenses. |
| Integration | Findings of the tools are integrated in the GitLab user interface (vulnerability view). | Findings of the tools (they need to support sarif) are integrated in the GitHub user interface (security tab). |
Container Scanning#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Tool | Standard tool provided by GitLab (based on Trivy and Grype). Easy to add. | No GitHub standard. We selected Trivy as container scanner, you have to find it in the Marketplace. |
| Integration | Result is Integrated in the GitLab UI in two places: In the pipeline run, in the Vulnerability Report. | Results are integrated into the GitHub UI in Security -> Code Scanning (when they provide a sarif report). |
| Findings | Findings are reasonably good. | Findings are reasonably good (with our selected tool). Duplicates with SCA may happen. |
| Container Registry | Creating a container image and making it available in the default container registry requires a bit more effort but is documented. | Creating a container and making it available in the container registry is reasonably documented but not trivial. |
| Pipeline Break | Findings will not break the pipeline; they are just reported. We cannot configure GitLab to break a pipeline when vulnerabilities are found. | Findings will not break the pipeline; they are just reported. Making the pipeline break will require some custom work. |
Secret Detection#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Tool | GitLab provides a standard tool based on GitLeaks. Integration is easy. | GitHub has built-in functionality for secrets. Integration is easy. |
| Capabilities | Secrets found are limited; a good SCA tool is required as complement. GitLeaks focus is on security tokens (e.g. AWS access tokens), not regular keys or passwords. | Finds secrets in the repository. Secrets found are limited. The focus is on security tokens, not regular keys or passwords. You can define and test your own secret patterns. |
| Integration | Findings are reported in the GitLab UI; both in the pipeline and the Vulnerability Report. | At the time when we recorded the video “push protection” of secrets did not work. |
| Secret Storage | GitLab has no integrated secure way to store secrets. They support HashiCorp Vault, but the Vault must be hosted somewhere else and managed by you. | GitHub has a well-protected secrets area where secrets can be stored securely. AND they also support Azure KeyVault out-of-the-box. A huge plus point. |
Dynamic Application Security Testing (DAST)#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Tool | Provides default tooling based on OWASP ZAP. | No out-of-the-box solution. We chose OWASP ZAP based on the action zaproxy/action-full-scan. |
| Capabilities | Scan capabilities are very limited out-of-the-box, needs considerable configuration. Out-of-the-box findings are mostly irrelevant. | Scan capabilities are limited out-of-the-box, but results better than in GitLab. |
| Container | Having to create a container first that can be started to run DAST against. Container can be started in GitLab internal runtime environment. | Having to create a container first. Container can be started in GitHub using an Ubuntu image that has docker installed. |
| Integration | Results are nicely integrated in Pipeline run. Results can also be found in the vulnerability management report of the GitLab UI. | Results are not integrated in the GitHub UI. Just a standalone report is made available. The reason is missing sarif support in OWASP ZAP. |
Vulnerability Management#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Functionality | GitLab offers functionality to manage vulnerabilities in the platform UI. You can add Vulnerabilities manually. | GitHub offers limited functionality to manage vulnerabilities in the platform UI. Most critically missing: the capability to add custom vulnerabilities/findings. |
| Dismissing | Dismissing a finding is possible but without reasoning. We are missing most the capability to add a description why we dismissed a finding. | Dismissing findings as false positive, won’t fix (accepted risk), test code only is possible with a comment. |
| Issues | You can create an issue from a vulnerability -> “Developer Task”. | You can create an issue from a vulnerability/finding -> “Developer Task”. |
| Resolving | You can manually mark findings as resolved or accept the automatically detected resolving. | Resolving is only automatically possible. No marking as fixed in the vulnerability management. |
| Integration | While it is possible to integrate findings of additional tools they are required in a non-standard format, so hard to get. | You can integrate findings of additional tools when they deliver the report in sarif format. |
| Limitations | N/A | You cannot dismiss for a limited time. You can’t change the severity of a finding. |
Merge Request / Pull Request#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Branch Protection | You can protect branches, so that merge requests are mandatory. | You can enforce that pull requests are mandatory for a branch. |
| Security Findings | Lists security findings delta between the merged and the default branch. Deltas and findings are available for all the security tools integrated in GitLab. | You can present some results of the security tools, quite a bit of customization is required to get acceptable quality. |
| Presentation | Deltas/Findings are nicely presented and can be navigated to. | New findings are then available, to some degree, for the security tools (checks). Resolved findings are not visible. Findings are not aggregated between tools. |
| Approvals | You can have defined people eligible to approve if new vulnerabilities of a given criticality are found. | You can have defined people eligible to approve the pull request, but this is not based on new findings/vulnerabilities. |
| Other Info | You can see also non-security stuff such as: code changes, review comments, test results, pipeline runs. | You can see also non-security stuff such as: code changes, review comments, test results. |
Schedule Pipeline#
GitLab
GitHub
| Aspect | GitLab | GitHub |
|---|---|---|
| Trigger | Triggered by a dedicated scheduler found in GitLab. | Triggered directly from within a workflow definition file. |
| Branch Selection | You can freely choose the branch on which it should run. | Always executes on the default branch only, which is most likely not what you need. There are (complicated) workarounds to trigger execution on another branch. |
| Configuration | You need to enhance the pipeline definition file to understand when a scheduler triggered the run (to exclude unused jobs, adapt the pipeline run, etc). | Create a dedicated workflow file for this. This new workflow only includes jobs required to do security checks on the production branch/code. |
Our Recommendation#
GitLab
GitHub
| GitLab | GitHub |
|---|---|
| Define default branch: This will be the branch that is used by GitLab to display security vulnerabilities in the built-in vulnerability management tool. | Create top-level workflows (pipelines) orchestrating (simpler) workflows into complete DevSecOps pipelines. |
| Define a scheduled pipeline: This is to make sure you scan your production code for new vulnerabilities. | Define re-usable workflows on a per-job level. |
| Use Merge Requests: To visualize the impact of changes on the security posture. | Define on what branches to run pipelines. |
| Consider using a Vault to store credentials: GitLab has no real solution built in. You should consider a third-party tool. | Define a scheduled pipeline for the current production release branch. |
| Consider using out-of-the-box GitLab tooling: This greatly simplifies matters and gives you a head start applying potentially “just-good-enough” tools relatively cheap. | Use Pull Requests to find newly introduced security issues. Summary view is less ideal. |
| DAST: Remember to customize the scanner configuration: Without customization results are weak. | Use the capability to protect branches. At least the release branch should be protected. |
| (Security-) review the tools you source from the marketplace. | |
| Use the GitHub-provided default GitHub secrets to store secrets or use the professional Azure Key Vault. | |
| Evaluate security tools and use a set that works good enough for you. Sadly, there is no out-of-the-box standard tooling. | |
| DAST: Remember to customize the scanner configuration. | |
| Until GitHub provides the capability to add vulnerabilities manually, you might have to use an extra tool to track vulnerabilities. |
Code#
| GitLab | GitHub |
|---|---|
| https://gitlab.com/romano_roth/gitlabdevsecopspipeline | https://github.com/romanoroth/GitHubDevSecOps |
Summary#
Our epic journey comes to an end. In the past month, we have created 24 videos, 12 on GitLab and 12 on GitHub, in which we have built up a DevSecOps pipeline. Now in this 25th video, we are going to compare GitLab vs. GitHub.
📌 GitLab is faster to deliver results and has out-of-the-box tooling for everything but lacks proper secret management.
➡️ GitLab is our recommendation when you want to get there fast and are ok to stick to the defaults.
📌 GitHub offers more flexibility, supports great secret management and has a living community but comes with high supply chain risk, has no reasonable security tool defaults and is missing a critical vulnerability management feature (add external vulnerability).
➡️ GitHub is our recommendation when you have complex applications/pipelines, or you must integrate with a few external (security) tools.
💡 Whether you’re a developer, a project manager, or just someone interested in tech, our recommendations can help you make an informed decision about which platform is best for you.