Enterprise Server 3.11 release notes

Packages have been updated to the latest security versions.

The audit log cluster rebalance script incorrectly proceeded before all shards were ready. This caused the script to exit before the necessary data was available, potentially leading to issues with the audit log migration.

For instances hosted on Azure, if a pre-upgrade check failed due to insufficient user disk size, the Management Console displayed an internal server error.

Pull request synchronization—the process keeping pull requests up to date with the latest commits to a branch—sometimes failed to retry if the initial synchronization process failed.

When creating a pre-receive hook environment, attempts to include an image URL over 255 characters failed with a database error. The maximum length is still 255 characters, but the URL length is now validated before the process starts.

Performing a browser back navigation to a pull request now displays up-to-date status checks.

Subversion services were non-functional in some cases.

Pull request merges are handled more efficiently, allowing more Git objects to be created before timeout. Additionally, loose objects created by merges that time out are now discarded, limiting the accumulation of these objects.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation. When regenerating the certificates with management console, the subdomain reply.[hostname] is missing from the ssl certification.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Some customers upgrading from 3.11.x or 3.12.x may experience a bug with the feature "Automatic update checks", filling the root disk with logs causing a system degradation. To prevent this, you can turn off the feature Enable automatic update check in the management console.

Packages have been updated to the latest security versions.

Creating a new comment on a pull request could incorrectly emit a pull_request_review_comment.edited webhook event.

In some cases, ambiguous commit OIDs within a repository caused errors when selecting commits to view in a pull request.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation. When regenerating the certificates with management console, the subdomain reply.[hostname] is missing from the ssl certification.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Attempting to stop replications after stopping GitHub Actions on a GitHub Enterprise Server instance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl

Some customers upgrading from 3.11.x or 3.12.x may experience a bug with the feature "Automatic update checks", filling the root disk with logs causing a system degradation. To prevent this, you can turn off the feature Enable automatic update check in the management console.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This is a follow up fix for CVE-2024-9487 to further harden the encrypted assertions feature against this type of attack. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document to exploit this vulnerability.

HIGH: An attacker with Enterprise Administrator access to the GitHub Enterprise Server instance could escalate privileges to SSH root access. This is achieved by exploiting the pre-receive hook environment to bypass symlink checks in the ghe-firejail path and execute malicious scripts. GitHub has requested CVE ID CVE-2024-10007 for this vulnerability, which was reported via the GitHub Bug Bounty program. [Updated: 2024-11-07]

This error message mbind: Operation not permitted was repeatedly showing in the /var/log/mysql/mysql.err MySQL logs.

When saving settings in the Management Console, the configuration run would stop if the enterprise-manage process was restarted.

A missing configuration value prevented Dependabot from creating group update pull requests.

On an instance with GitHub Actions enabled, some maintenance tasks could fail due to incomplete upgrade steps during previous upgrades to new releases of GitHub Enterprise Server.

The initial setup certificate generation in AWS took longer than expected due to fallback to private IPs. The time for this fallback has been reduced.

If the primary instance was unreachable, running ghe-repl-stop --force on a replica would fail during the config apply run.

When restoring from a backup, repositories that had been deleted in the last 90 days were not completely restored.

Restoring Git repositories using backup-utils occasionally failed.

Some customers upgrading from 3.11 to 3.13 may experience issues with undecryptable records during the upgrade. This issue has now been resolved. We recommend you read Undecryptable records.

For instances deployed on AWS, the default settings for Chrony NTP synchronization have been aligned with AWS's suggested default configurations.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. See Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track GitHub Advanced Security contributions.

The reply.[HOSTNAME] subdomain is falsely always displaying as having no SSL and DNS record, when testing the domain settings via the Management Console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

MEDIUM: Malicious URLs for SVG assets provided information about a victim user who clicked the URL, allowing an attacker to retrieve metadata belonging to the user and use it to generate a convincing phishing page. This required the attacker to upload malicious SVGs and phish a victim user to click the URL for the uploaded asset. GitHub has requested CVE ID CVE-2024-9539. This vulnerability was reported via the GitHub Bug Bounty program.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This was a regression introduced as part of follow-up remediation from CVE-2024-4985, which resulted in a new variant of the vulnerability. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document. GitHub has requested CVE ID CVE-2024-9487. This vulnerability was reported via the GitHub Bug Bounty program.

HAProxy reloading was failure prone, which could lead to failed Git operations. This reloading process has been replaced with a more resilient Systemd process.

An unhandled nil value when configuring Actions storage with AWS S3 via OIDC configuration in the terminal could cause an error.

On an instance with secret scanning enabled, the custom pattern page would not load because dry run results were tied to a deleted repository.

The "List teams" API endpoint returned duplicate results when paginating.

A model with no URL could cause a ghe-migrator import to fail.

Restore could fail when restoring MySQL using backup-utils.

Pre-receive hook environments can use the clone3() system call.

The creation, deletion, or change in visibility of a gist has been added to the audit log.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no SSL and DNS record, when testing the domain settings via management console without subdomain isolation.

The admin stats REST API endpoints may time out on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

MEDIUM: An attacker could steal sensitive information by exploiting a Cross-Site Scripting vulnerability in the repository transfer feature. This exploitation would require social engineering. GitHub has requested CVE ID CVE-2024-8770 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could push a commit with changes to a workflow using a PAT or OAuth app that lacks the appropriate workflow scope by pushing a triple-nested tag pointing at the associated commit. GitHub has requested CVE ID CVE-2024-8263 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: A GitHub App installed in organizations could upgrade some permissions from read to write access without approval from an organization administrator. An attacker would require an account with administrator access to install a malicious GitHub App. GitHub has requested CVE ID CVE-2024-8810 for this vulnerability, which was reported via the GitHub Bug Bounty Program. [Updated: 2024-11-07]

For instances deployed on AWS with IMDSv2 enforced, fallback to private IPs was not successful.

A config apply run may not have been properly applied due to calls being made to Nomad before it was ready to accept connections. When this occurred, the Error querying agent info: failed querying self endpoint: Get "http://127.0.0.1:4646/v1/agent/self" error was written to the /data/user/common/ghe-config.log file.

ghe-storage-find was sometimes unable to identify a data disk.

After upgrading the relevant GHES version, the resolvconf service failed to start due to a missing directory.

When configuring a high availability replica and during the database seeding of a MySQL replica node, restarting the nomad service could time out. Consequently, when MySQL replication attempted to start an error was reported, and setting up replication failed.

Placing Nomad jobs would not allow retries in cases when Nomad wasn't available yet.

On an instance in a cluster configuration, the ghe-cluster-status command returned an error if a soft-deleted repository had a checksum mismatch.

Some repositories could miss spokes information after restoring in a clustering topology due to unrescued exceptions.

Fixes and improvements for the git core module.

The CommandPalette component no longer displays repository information on 404 pages, preventing the leakage of private repository information for users without access.

Custom links to other repositories displayed incorrect breadcrumbs.

When a GitHub App installation had all repositories installed individually, it was not possible to remove the repositories from the selection.

Some custom pattern matches were incorrectly filtered during post-scan filtering. You may want to edit and republish your custom patterns. You can manually republish custom patterns with the following command: ghe-secret-scanning jobs queue custom-patterns republish --custom-pattern-id=?. Outdated alerts caused by edits during custom pattern backfills have been fixed in version 3.13 and above.

For instances deployed on Amazon Web Services (AWS), site administrators can configure regional AWS STS endpoints for OIDC from the Management Console.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Users can view the app state of gists, networks, and wikis in the spokesctl info output, enhancing visibility into the status of these elements. Additionally, spokesctl check can diagnose and, in most cases, fix empty repository networks, improving network management.

CRITICAL: On GitHub Enterprise Server instances that use SAML single sign-on (SSO) authentication with specific IdPs utilizing publicly exposed signed federation metadata XML, an attacker could forge a SAML response to provision and/or gain access to a user account with site administrator privileges. GitHub has requested CVE ID CVE-2024-6800 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could update the title, assignees, and labels of any issue inside a public repository. This was only exploitable inside a public repository, and private/internal repositories were not affected. GitHub has requested CVE ID CVE-2024-7711 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the issue contents from a private repository using a GitHub App with only contents: read and pull requests: write permissions. This was only exploitable via user access token, and installation access tokens were not impacted. GitHub has requested CVE ID CVE-2024-6337 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

During hotpatching and sometimes when applying configuration changes, a configuration run to upgrade the GitHub Actions service was unnecessarily triggered. The GitHub Actions service will only be upgraded in GitHub Enterprise Server feature releases.

On an instance with GitHub Actions enabled, during a hotpatch upgrade, a race condition could block various upgrade activities.

The ghe-config-apply process made an unnecessary number of connections to Redis.

Restarting the resolvconf service would not correctly update the contents of /etc/resolv.conf.

Instances installed on Google Cloud Platform (GCP) could have their hostname overwritten by GCP when a hotpatch was applied.

The minimum password requirements for Management Console users and the root site administrator required an upper case character when providing a password with a minimum of 8 characters, contradicting the documentation and password hint.

The ghe-migrations utility for visualizing migrations did not work due to a regression. Administrators can now run ghe-migrations to view the progress and status of github migrations, or run ghe-migrations --all to view progress on all services.

On an instance with subdomain isolation enabled, configuration runs created subdomains for ChatOps services, such as slack.HOSTNAME and teams.HOSTNAME, regardless of whether the service was enabled.

On an instance with GitHub Actions enabled, due to an insufficient wait time, MS SQL and MySQL replication could fail with the error message Failed to start nomad service!.

Site administrators could not switch maintenance mode directly from "scheduled" to "on," or vice versa.

Some users were unable to delete project views.

When importing using ghe-migrator, team URLs containing dots were imported as-is, leading to 404s when attempting to view the imported teams. Dots in imported team URLs are now escaped to dashes.

Due to a regression introduced in a previous patch, for enterprises that use encrypted SAML assertions, SSO attempts failed with a digest mismatch error if the entire SAML response was signed, rather than just the assertions.

Running go get for a Golang repository with a directory structure that overlaps with GitHub UI routes failed

The github-stream-processor service could get into a state where it would continually fail to process messages with a TRILOGY_CLOSED_CONNECTION error.

The wrong help link was displayed when push protection blocked a secret from the CLI.

For repositories with issues disabled, issue links were redirected to pull requests.

In custom pre-receive hooks, the paths stored in environment variables that allow for newly pushed objects to be in a quarantine directory could be incorrectly interpreted as relative to a worktree instead of the Git directory, causing certain commands to fail to read from the repository. The variables now use absolute paths.

A corrupted entry in the Git audit log could cause out of memory errors.

Fixes and improvements for the git core module.

Actions KPI logs are disabled by default to reduce log size.

Users can set their styling preference for link underlines in the web interface, on their "Accessibility" settings page.

Audit log events related to audit log streaming are available in the enterprise audit log page, and via audit log streaming.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

The reply.HOSTNAME subdomain is falsely displayed as having no SSL and DNS record, when testing the domain settings via the Management Console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

On boot, the resolvconf service may fail to start because the /run/resolvconf directory does not exist when the service attempts to touch a file there, with the error:

/bin/touch: cannot touch '/run/resolvconf/postponed-update': No such file or directory

If this occurs, workaround this issue with the following commands — this change will persist on reboots, but not upgrades:

sudo sed -i.bak \
'/\[Service\]/a ExecStartPre\=\/bin\/mkdir \-p \/run\/resolvconf' \
/etc/systemd/system/resolvconf.service.d/local.conf

sudo systemctl daemon-reload
sudo systemctl start resolvconf 

[Updated: 2024-08-26]

These release notes previously indicated as a known issue that on GitHub Enterprise Server 3.11.14 when log forwarding is enabled, some forwarded log entries may be duplicated. The fix for this problem was already included in GitHub Enterprise Server 3.11.13. [Updated: 2024-09-16]

HIGH: An attacker could cause unbounded resource exhaustion on the instance by sending a large payload to the Git server. To mitigate this issue, GitHub has limited the count of "have" and "want" lines for Git read operations. GitHub has requested CVE ID CVE-2024-5795 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An improper privilege management vulnerability allowed users to migrate private repositories without having appropriate scopes defined on the related personal access token. GitHub has requested CVE ID CVE-2024-5566 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized access in a public repository using a suspended GitHub App via a scoped user access token. This was only exploitable in public repositories while private repositories were not impacted. GitHub has requested CVE ID CVE-2024-5816 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could execute a Cross Site Request Forgery (CSRF) attack to perform write operations on a victim-owned repository in GitHub Enterprise Server by exploiting incorrect request types. A mitigating factor is that the attacker has to be a trusted user and the victim has to visit a tag in the attacker's fork of their own repository. GitHub has requested CVE ID CVE-2024-5815 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the name of a private repository on the GitHub Enterprise Server appliance when the private repository has a deploy key associated to it. GitHub has requested CVE ID CVE-2024-6395 for this vulnerability, which was reported via the GitHub Bug Bounty program.

LOW: Instance administrators could see fine-grained personal access tokens in plaintext in the babeld and gitauth logs.

LOW: An attacker with read access to a project could use the REST API to view a list of all members in an organization, including members who had made their membership private. This vulnerability was reported via the GitHub Bug Bounty program.

LOW: An attacker could include MathJax syntax in Markdown to bypass GitHubs normal restrictions on CSS properties in Markdown. This vulnerability was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose sensitive information from a private repository exploiting organization ruleset features. This attack required an organization member to explicitly change the visibility of a dependent repository from private to public. GitHub has requested CVE ID CVE-2024-6336 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized read access to issue content inside an internal repository via GitHub projects. This attack required attacker access to the corresponding project board. GitHub has requested CVE ID CVE-2024-5817 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

When an instance hosted on Azure was upgraded with a hotpatch, the upgrade failed with an rsync error.

On an instance with GitHub Actions enabled, remote blob storage could fill up with large amounts of data because cleanup jobs were skipped on old hosts.

The threshold set by server_rejoin_age_max for single-node GHES deployments was too low.

In some cases, commands run in an administrative SSH shell were not written to the audit log.

When an administrator submitted support data to GitHub Support, spokesd keys were incorrectly sanitized.

When log forwarding was enabled, some specific service logs, including babeld, gitauth, unicorn, and resqued, were duplicated.

During the initial boot of an instance, a data disk attached as /dev/sdb may not have been recognized as an available disk.

In a high availablity configuration, running ghe-repl-node multiple times from a node that did not have replication running had the potential to overwrite the configuration on the primary node.

Configuration history is only generated for instances in a cluster, high availability (HA) cluster, or standalone HA configuration. The current node must be a primary or replica node with replication running.

In some cases, the HAProxy kill_timeout setting caused service outages during upgrades or large transactions.

The ssh-audit-log.sh script did not effectively log SSH commands, and the ghe-sanitize-log.psed script inadequately sanitized password-related logs.

The default MSSQL timeout of 8 seconds sometimes caused issues during administrator activities. The default timeout has been increased to 30 seconds.

For an instance running on Microsoft Azure, the user disk service failed to start because the attached volume could not be found.

Establishing a new GitHub Connect connection could fail with a 500 error.

When using ghe-migrator to migrate a repository, the links for pull requests merge commits were not imported.

When a user used the REST API endpoints that returned secret scanning alerts at the repository or organization level with non-cursor-based pagination (for example, without before or after query parameters), the REST API endpoints for secret scanning returned incorrect Link headers.

On certain branch names, the branch info bar was causing frozen string errors.

On instances with SAML authentication configured, users were unable to sign out and became stuck in an infinite SAML SSO loop.

On instances with SCIM enabled, the administrator was unable to view users without an external identity record (for example, because they were provisioned before SCIM was enabled on the instance) in stafftools.

On instances enrolled in the SCIM private beta, built-in authentication users can be added to organizations and teams. Organization owners will no longer see the misleading message that the organization membership is managed by the SAML identity provider when updating organization memberships.

Enterprise owners managed by an identity provider were asked to authenticate within GitHub when performing privileged actions.

On an instance that restricts emails to verified domains, secret scanning emails would sometimes be sent to an unverified domain.

In some cases, on the "Files" tab of a pull request, a comment on the first line did not render.

Some organizations were not recognized as part of an instance's enterprise account.

Some users would encounter an error when navigating to their personal security settings page at https://HOSTNAME/settings/security.

The SpokesSyncCacheReplicaJob could not initialize in some cases, resulting in an exception when handling the error.

On the "Code scanning" page of a repository, the branch filter did not correctly display all branches.

When including a .gitignore or README.md file on repository creation failed due to a ruleset or pre-receive hook, no error message displayed.

On an instance with a GitHub Advanced Security license, requests to the /enterprises/{enterprise}/settings/billing/advanced-security REST API endpoint could fail due to timeout.

Users viewing the alerts index page experienced inconsistencies in rendering the closed alert state.

Organizations named "C" were incorrectly routed to the GitHub Enterprise Server contact page instead of their organization page.

On an instance with a GitHub Advanced Security license, commits made by users who do not belong to an organization were not counted.

When servers responded with unsupported characters, webhook deliveries were not displayed in the UI.

Chat integrations required frequent reauthentication, as a result of new app installations overwriting previous ones.

On an instance in a cluster configuration, the ghe-spokesctl ssh command did not select the correct Nomad container when running a command within a Git repository.

On an instance with a GitHub Advanced Security license, disabling and re-enabling GitHub Advanced Security for an organization resulted in redundant scans of some repositories.

On an instance with a GitHub Advanced Security license, contributions were not tracked on public repositories.

On an instance with a GitHub Advanced Security license, the "adjust configuration" step failed when enabling code scanning with the default setup on self-hosted Windows runners.

Migration of the issue_edits table caused intermittent failures during the upgrade to GitHub Enterprise Server version 3.11, resulting in the error message ActiveRecord::ConcurrentMigrationError: Failed to release advisory lock. [Updated: 2024-08-14]

In a high availability configuration, users can only run ghe-config-apply or ghe-cluster-config-apply on a replica node if replication is already running (from ghe-repl-start). If replication isnt running on the node, the user will be instructed to start replication.

Configuration history has been extended. When ghe-config-apply, ghe-cluster-config-apply, or ghe-config-archive is run: secrets.conf is captured, a sha256sum for each of the current configuration files is included, the existing patch that is generated includes secrets.conf, and an additional sanitized patch that excludes secrets.conf is also generated.

The timeout for requests made to the REST API endpoints for secret scanning has been extended.

A more specific error message is shown when a non-provisioned user tried to sign in to an instance with SCIM enabled.

When a user changes a repository's visibility to public, the user is now warned that previous Actions history and logs will become public as well.

A more specific error message is shown when a deprovisioned user attempts signing into an instance with SCIM enabled.

In the audit logs, administrators can see more context for failed user authentication attempts using LDAP.

The system logs provide more context for authentication failures related to multi-factor authentication.

When using the ghe-webhook-logs utility, webhook delivery logs can be filtered by event and action. Users can use ghe-webhook-logs --event issues to filter by event, or ghe-webhook-logs --event issues.opened to filter by event and action.

To avoid excessive log volume and associated disk pressure, requests for GetCacheKey are no longer logged. Previously, the high frequency of these requests caused significant log accumulation.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

HIGH: An attacker with the site administrator role could gain arbitrary code execution capability on the GitHub Enterprise Server appliance when configuring audit log streaming. GitHub has requested CVE ID CVE-2024-5746 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

On an instance with GitHub Actions and External MySQL enabled, a validation step in the config apply could fail.

Users would see an error message from the server while pushing to a gist (the push would still complete).

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

ghe-migrations visualizer is not working due to a known regression. As a results, users will not be able to use ghe-migrations to view the status of migrations during an upgrade. Instead you can inspect the log files in /var/log/dbmigration to get the status/progress of migrations.

When enabling log forwarding, specific services logs (babeld and some more) are duplicated.

The reply.[hostname] subdomain is falsely always displaying as having no SSL and DNS record, when testing the domain settings via management console without subdomain isolation.

When log forwarding is enabled, some forwarded log entries may be duplicated.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

CRITICAL: On instances that use SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, an attacker could forge a SAML response to provision and/or gain access to a user with administrator privileges.

Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO or utilizing SAML SSO authentication without encrypted assertions are not impacted. Exploitation of this vulnerability would allow unauthorized access to the instance without requiring prior authentication. GitHub has requested CVE ID CVE-2024-4985 for this vulnerability, which was reported via the GitHub Bug Bounty program.

For more information, see Configuring SAML single sign-on for your enterprise and Enabling encrypted assertions.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Firewall port 9199, which linked to a static maintenance page used when enabling maintenance mode with an IP exception list, was opened unnecessarily.

As a result of a security vulnerability, the editor role for a Management Console user has been deprecated in the Manage GitHub Enterprise Server API.

Packages have been updated to the latest security versions.

Running ghe-repl-node -d did not validate value length in order to prevent values longer than 20 characters.

On an instance in a cluster configuration with high availability enabled, ghe-repl-setup did not successfully complete on a replica due to a missing key.

For an instance in a cluster configuration, during the migration phase of a configuration run, the process of copying configuration updates to all nodes would fail.

An LDAP-related error message was incorrectly displayed at the enterprise and organization levels.

On an instance with Dependabot enabled, a misconfiguration caused jobs to be added to the hydro_update_rule_override_for_repository_visibility_change_event queue but not processed.

An incorrect job queue mapping caused the hydro_advanced_security_archived_status_changed queue to constantly grow.

External collaborators with read-only access were able to run workflows on their pull requests from private forks without approval.

On an instance with a GitHub Advanced Security license, custom pattern matches were incorrectly filtered during post-scan filtering.

To aid in understanding the CPU/memory utilization of secret scanning processes, the binary names of nomad workers were updated to differentiate between the different types of secret scanning jobs.

A more specific error message is shown when the ghe-repl-node command is run on an instance not configured for high availability.

The SCIM private beta has resumed with support from GitHub engineering in GitHub Enterprise Server version 3.11 and later. Site administrators can provision users and groups on a GitHub Enterprise Server instance automatically with SCIM. SCIM for GitHub Enterprise Server is in private beta and subject to change. For more information, see Configuring user provisioning with SCIM on GitHub Enterprise Server and REST API endpoints for SCIM in the REST API documentation.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker with the editor role in the Management Console could gain administrative SSH access to the appliance by command injection when configuring the chat integration. GitHub has requested CVE ID CVE-2024-3646 for this vulnerability, which was reported via the GitHub Bug Bounty program. The editor role has been deprecated. For more information, see the "Changes" section of these release notes.

HIGH: An attacker with an editor role in the Management Console could gain SSH access to the instance by command injection when configuring Artifact & Logs and Migrations Storage. GitHub has requested CVE ID CVE-2024-3684 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker with a deploy key for an organization-owned repository could bypass a ruleset that specified organization administrators as bypass actors. Exploitation would require an attacker to already have access to a valid deploy key for a repository. GitHub has requested CVE ID CVE-2024-3470 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could maintain admin access to a detached repository in a race condition by making a GraphQL mutation to alter repository permissions while the repository is detached. GitHub has requested CVE ID CVE-2024-2440 for this vulnerability, which was reported via the GitHub Bug Bounty program.

A GraphQL endpoint was disabled as part of a previous security fix, causing projects auto-add workflow and inline issue creation errors. To resolve these errors, a security patch was applied that allows for the affected GraphQL endpoint to be re-enabled.

Packages have been updated to the latest security versions.

When configuring audit log streaming to Datadog or Splunk on an instance with custom CA certificates, the connection failed with the error There was an error trying to connect.

Disk usage, utilization, and latency for data devices could render incorrectly in Grafana.

On an instance in a cluster configuration with high availability replication enabled, Git operations for existing repositories would fail after failover to the replica cluster.

On an instance in a cluster configuration, former primary nodes were able to access the newly promoted nodes after failover. The ghe-cluster-failover command has been updated to block access from the old cluster, and four new command-line utilities have been introduced to manually block IP addresses: ghe-cluster-block-ips, ghe-cluster-block-ip, ghe-cluster-unblock-ips, and ghe-cluster-unblock-ip. For more information, see Command-line utilities. [Updated: 2024-05-01]

A Redis job had a memory limit that was too low in some cases, leading the process to run out of memory.

The ghe-update-check command did not clean up .tmp files in /var/lib/ghe-updates/, which could lead to full disk issues.

On an instance that failed a configuration run, when attempting to repeat the restore step of a backup, the audit log restore step returned error lines even though audit logs were being fully restored.

In some cases, Treelights timeouts caused pull requests to return a 500 error.

Some search indices in Stafftools failed to load.

The web UI presented inapplicable fine-grained permissions for assignment to custom repository roles. The permissions were also displayed as implicitly included in certain base roles.

Unauthenticated requests to the REST APIs /search/code endpoint returned erroneous rate-limit values.

The profile settings for organizations displayed a warning about profile images that does not apply to organizations on a GitHub Enterprise Server instance.

On some pages that listed users on an instance, the web UI erroneously rendered checkboxes to the right of users details.

Administrators could get a 500 error when trying to access the "File storage" section of the site admin dashboard.

Setting a maintenance message failed if the message contained a multibyte character.

On an instance where user avatars had been deleted directly from the database, an identicon avatar was not correctly displayed for affected users, and administrators may have observed a relatively high number of application exceptions.

On an instance with repository caching configured, adding new repositories to a cache node sometimes failed.

On an instance with a GitHub Advanced Security license, after enabling secret scanning for the first time for an organization or the instance, the historical backfills for alerts in existing repositories issues did not appear.

On an instance with a GitHub Advanced Security license, metrics for custom patterns alerts incorrectly included tokens in ignored locations.

On an instance with code scanning enabled, on the tool status page for code scanning, outdated upload errors were still displayed after a successful upload.

On an instance hosted on Azure, administrators can set and reset SSH keys and passwords via the Azure Agent.

As a result of a security vulnerability, the editor role for a Management Console user has been deprecated. For details, see the "Security fixes" section of these release notes. Existing users with the editor role will be unable to log in to the Management Console, and should contact their site administrator requesting that access be reinstated by updating the user to the operator role if appropriate.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker with an Administrator role in GitHub Enterprise Server could gain SSH root access via remote code execution. GitHub has requested CVE ID CVE-2024-2469 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain SSH access to the instance by command injection when configuring GeoJSON settings. GitHub has requested CVE ID CVE-2024-2443 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

In some cases, storage initialization on a new instance launch could cause EBS-backed data volumes to not be detected correctly.

Administrators could initiate an SSH audit that unknowingly unverified all SSH keys.

Attributes used to debug LDAP issues were not included in system logs.

On an instance in a high availability or cluster configuration, configuring fluent-bit on a primary node returned an empty primary_host value.

On an instance in a cluster configuration with many nodes, requests to the REST API for managing GitHub Enterprise Server would exceed the instances HTTP timeouts.

Redundant messages caused an increase in the volume of events logged in /var/log/syslog.

On an instance in a cluster configuration with high availability enabled, the ghe-spokesctl command failed when run on a replica node.

If an administrator lost SSH access to an instance, authentication from the hypervisor console using the password for the root site administrator would fail.

On an instance with GitHub Actions enabled, GitHub Actions workflows that deployed GitHub Pages sites failed with the following error: Error: Deployment failed, try again later.

On an instance in a cluster configuration, Jupyter notebooks did not render correctly.

Some API endpoints for projects did not properly filter target repositories based on the users access.

On an instance with a GitHub Advanced Security license, some searches for secret scanning alerts resulted in a 500 error.

When an administrator set a policy to require two-factor authentication (2FA) for an enterprise, a message incorrectly indicated that users without 2FA enabled on their account would be removed from the enterprise. These users will be removed from repositories and organizations in the enterprise, but not from the enterprise itself.

On an instance with a GitHub Advanced Security license, viewing a secret scanning alert as a user without the security manager role would return a 500 error if the alert was generated from a Git tag instead of a normal commit.

When using GitHub Enterprise Importer to import repositories, ghost users in archive metadata files would cause an error when generating a list of migration conflicts using ghe-migrator conflicts.

After an administrator ran ghe-saml-mapping-csv, the output did not include the corresponding SQL query.

On an instance with a GitHub Advanced Security license, the security overview did not display updated alert counts for code scanning immediately after the completion of analysis.

During a configuration run prompted by the delayed restart of the notebooks service, a container validation warning appeared in system logs.

On an instance in a cluster configuration, rebuilds of GitHub Pages sites failed if no replicas of the GitHub Pages data were available (for example, on a newly restored cluster).

In some cases, manual repository maintenance using ghe-spokesctl would fail with the following error: panic: runtime error: invalid memory address or nil pointer dereference.

On an instance with a GitHub Advanced Security license, the speed of migration for code scanning analyses is increased during an upgrade from GitHub Enterprise Server 3.10 or earlier.

On an instance with a GitHub Advanced Security license, in some cases, weekly scheduled runs for code scanning's default setup might not occur.

Gists can be deleted using the Purge Gist button on the Deleted Gists page in Staff Tools.

People deploying a GitHub Enterprise Server instance in AWS can now deploy in an environment that uses Instance Metadata Service Version 2 (IMDSv2).

On an instance with a GitHub Advanced Security license, in some cases, when a user deleted a custom pattern for secret scanning, GitHub Enterprise Server failed to close or delete the patterns alerts.

On an instance in a cluster configuration, MySQL replica nodes can be configured to skip database seeding. For more information, see Deferring database seeding.

The payload for the push webhook event is now limited to 2,048 commits. If there are more than 2,048 commits in a push, the payload for the push webhook will not contain serialized diff information for each commit. If you need to fetch commit information, you can use the Commits endpoints of the REST API. For more information, see Webhook events and payloads and REST API endpoints for commits.

Organizations using projects (classic) returned an error log about a soon-to-be deprecated MySQL feature when viewing a project.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: On an instance with GitHub Connect enabled and non-default settings for GitHub Connect configured, an attacker could use an enterprise GitHub Actions download token to fetch private repository data. This token is only accessible to users on the GitHub Enterprise Server instance. To fix this vulnerability, the Actions download token will now be a permissionless token. GitHub has requested CVE ID CVE-2024-1908 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

Redundant messages caused increased log volumes in /var/log/syslog.

For instances deployed on Google Cloud Platform, GitHubs public images include support for Google Virtual NIC (gVNIC) by default. Previously, to use gVNIC, an administrator needed to use the --guest-os-features=gvnic flag when creating the instance.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

On an instance with GitHub Actions enabled, Actions workflows that deploy GitHub Pages sites may fail with the following error:

Error: Deployment failed, try again later.

To fix this issue, connect to any of the instance's nodes using SSH, then run the following commands.

if [ -d "$CHROOT_PATH/data/pages-untar" ] ; then
  rm -rf "$CHROOT_PATH/data/pages-untar"
fi
pages_untar_image_tag="$(cat "$CHROOT_PATH/data/docker-image-tags/pages_untar_image_tag")"
id="$(docker create "pages-untar:$pages_untar_image_tag")"
sudo docker cp "$id:/data/pages-untar" "$BASE_PATH/$CHROOT_PATH/data/pages-untar/"
docker rm "$id"

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker could gain unauthorized read permission to files by deploying arbitrary symbolic links to a GitHub Pages site with a specially crafted artifact tarball. GitHub has requested CVE ID CVE-2024-1082 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection when configuring SAML settings. GitHub has requested CVE ID CVE-2024-1372 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection when setting an HTTP proxy. GitHub has requested CVE ID CVE-2024-1359 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection into nomad templates when configuring SMTP options. GitHub has requested CVE ID CVE-2024-1378 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection in the actions-console docker container while setting a service URL. GitHub has requested CVE ID CVE-2024-1355 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection in the syslog-ng configuration file. GitHub has requested CVE ID CVE-2024-1354 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection when setting the username and password for collectd configurations. GitHub has requested CVE ID CVE-2024-1369 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain admin SSH access to the appliance by command injection into nomad templates when configuring audit log forwarding. GitHub has requested CVE ID CVE-2024-1374 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker could create new branches in public repositories, and run arbitrary GitHub Actions workflows with permissions from the GITHUB_TOKEN. GitHub has requested CVE ID CVE-2024-1482 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could make changes to a user account by taking advantage of a Cross-site Scripting vulnerability in the tag name pattern field in the tag protections UI. Exploitation of this vulnerability required user interaction with malicious javascript on a website along with further social engineering. GitHub has requested CVE ID CVE-2024-1084 for this vulnerability, which was reported via the GitHub Bug Bounty program.

LOW: An attacker could decrypt the user section of the enterprise user license list JSON file by using an exposed private key. This vulnerability was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

On startup, Elasticsearch logged an innocuous JMX MBeans registration error.

Hunk headers in C# files did not correctly display changed functions.

On an instance with a GitHub Advanced Security license, the code scanning "Tools" status page incorrectly displayed an Add tool button when Actions was disabled.

When using ghe-migrator to import repositories, issue and pull request attachments imported but failed to render in the UI.

A change to the way GitHub handles pushes caused custom pre-receive hooks to fail when inspecting the newly-pushed content.

When restoring a deleted repository, some metadata associated with the repository, such as packages or project items, did not properly restore.

During Git data server maintenance, a process that was ran on unsupported GitHub Enterprise Server topologies created a significant amount of system logs but did not perform any repair work.

The default 30 second webhook delivery HTTP timeout can be configured.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

On an instance with GitHub Actions enabled, Actions workflows that deploy GitHub Pages sites may fail with the following error:

Error: Deployment failed, try again later.

To fix this issue, connect to any of the instance's nodes using SSH, then run the following commands.

if [ -d "$CHROOT_PATH/data/pages-untar" ] ; then
  rm -rf "$CHROOT_PATH/data/pages-untar"
fi
pages_untar_image_tag="$(cat "$CHROOT_PATH/data/docker-image-tags/pages_untar_image_tag")"
id="$(docker create "pages-untar:$pages_untar_image_tag")"
sudo docker cp "$id:/data/pages-untar" "$BASE_PATH/$CHROOT_PATH/data/pages-untar/"
docker rm "$id"
``` [Updated: 2024-03-07]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

The instance incorrectly wrote the output for multiple workloads to /var/log/syslog.log.

During periods of high traffic, interruptions in service occurred due to insufficient resource allocations for internal components.

When starting up an instance using NVME storage in a cloud other than AWS, the attached data disk was not properly detected.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

Restoring backups with ghe-restore on a GHES cluster will exit prematurely if redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Pre-receive hooks which utilize git rev-list fail with an fatal: Invalid revision range error message.

On an instance with GitHub Actions enabled, Actions workflows that deploy GitHub Pages sites may fail with the following error:

Error: Deployment failed, try again later.

To fix this issue, connect to any of the instance's nodes using SSH, then run the following commands.

if [ -d "$CHROOT_PATH/data/pages-untar" ] ; then
  rm -rf "$CHROOT_PATH/data/pages-untar"
fi
pages_untar_image_tag="$(cat "$CHROOT_PATH/data/docker-image-tags/pages_untar_image_tag")"
id="$(docker create "pages-untar:$pages_untar_image_tag")"
sudo docker cp "$id:/data/pages-untar" "$BASE_PATH/$CHROOT_PATH/data/pages-untar/"
docker rm "$id"
``` [Updated: 2024-03-07]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker with access to a Management Console user account with the editor role could escalate privileges through a command injection vulnerability in the Management Console. GitHub has requested CVE ID CVE-2024-0507 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker could leverage an unsafe reflection vulnerability in GitHub Enterprise Server (GHES) that could lead to reflection injection. This vulnerability could lead to the execution of user-controlled methods and remote code execution. To exploit this bug, an actor would need to be logged into an account on the GHES instance with the organization owner role. GitHub has requested CVE ID CVE-2024-0200 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

Support for authenticating to GitHub Enterprise Server using GitHub CLI OAuth App with a device code was unintentionally disabled.

During periods of high load, users would see intermittent interruptions to services when upstream services failed internal health checks.

On an instance in a high availability configuration, site administrators using the Manage GitHub Enterprise Server API may have seen a status of UNKNOWN for the MSSQL service.

Hotpatch upgrades from GitHub Enterprise Server version 3.11.0 to 3.11.1 resulted in the instance losing network connectivity after a reboot.

On an instance with GitHub Actions enabled, some maintenance tasks could fail due to incomplete upgrade steps during previous upgrades to new releases of GitHub Enterprise Server.

Deleting a repository would enqueue unnecessary background jobs that would never complete.

When creating a new custom pattern for secret scanning, the "More options" section of the custom pattern form automatically collapsed when a user entered an invalid regex in the post processing expressions (before/after secret match or additional secret requirements).

On an instance with a GitHub Advanced Security license and secret scanning enabled, users could experience a 500 error when viewing a secret scanning alert page in cases where the alerted commits belonged to the user and one or more commits could not be found.

Members of an enterprise were incorrectly allowed access to the REST API endpoints for Enterprise licensing.

On an instance that uses SAML for authentication, an upgrade from GitHub Enterprise Server 3.7 to 3.9 could result in user login failures due to an outdated gem dependency.

Under rare circumstances, a repository could become unavailable due to a temporary file being left behind after a Git process was unexpectedly interrupted (for example, due to a power outage).

On an instance with GitHub Advanced Security enabled, a suspended user would consume a license for GitHub Advanced Security.

To avoid leaking secrets, the logging of all parameters is disabled for Management Console events in enterprise audit logs.

The branch protection setting to require PR approval of the most recent reviewable push is included in exports from ghe-migrator or the Organization Migrations API.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

Restoring backups with ghe-restore on a GHES cluster will exit prematurely if redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0.

Pre-receive hooks which utilize git rev-list fail with an fatal: Invalid revision range error message.

During periods of high traffic, interruptions in service may occur due to insufficient resource allocations for internal components. [Updated 2024-01-23]

The settings for enabling scheduled reminders were added unintentionally to this release. Scheduled reminders are not officially supported. [Updated: 2024-02-22]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

When an instance is upgraded with a hotpatch, the instance may lose network connectivity after a reboot.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

Restoring backups with ghe-restore on a GHES cluster will exit prematurely if redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0. [Updated 2024-01-03]

During periods of high traffic, interruptions in service may occur due to insufficient resource allocations for internal components. [Updated 2024-01-23]

The settings for enabling scheduled reminders were added unintentionally to this release. Scheduled reminders are not officially supported. [Updated: 2024-02-22]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Known issues incorrectly indicated that an upgrade to GitHub Enterprise Server 3.11 may fail. This issue does not impact GitHub Enterprise Server instances when upgrading to version 3.11.1 or later. [Updated: 2024-01-26]

HIGH: An improper authentication vulnerability was identified in GitHub Enterprise Server that allowed a bypass of private mode by using a specially crafted API request. Private mode is the mechanism that enforces authentication for publicly-scoped resources. For more information, see Enabling private mode.

This vulnerability would allow unauthenticated attackers to gain access to various types of resources set as public on the instance. To exploit this vulnerability, an attacker would need network access to the GitHub Enterprise Server instance configured in private mode. This vulnerability was reported via the GitHub Bug Bounty program and assigned CVE-2023-6847.

HIGH: A path traversal vulnerability was identified in GitHub Enterprise Server that allowed arbitrary file reading when building a GitHub Pages site. To exploit this vulnerability, an attacker would need permission to create and build a GitHub Pages site on the GitHub Enterprise Server instance. This vulnerability was reported via the GitHub Bug Bounty program and assigned CVE-2023-46645.

MEDIUM: An attacker could maintain admin access via a race condition when an organization was converted from a user. GitHub has requested CVE ID CVE-2023-46649 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An insertion of sensitive information into log file in the audit log in GitHub Enterprise Server was identified that that could allow an attacker to gain access to the Management Console. To exploit this, an attacker would need access to the log files for the GitHub Enterprise Server instance, a backup archive created with GitHub Enterprise Server Backup Utilities, or a service which received streamed logs. GitHub has requested CVE ID CVE-2023-6802 for this vulnerability.

MEDIUM: A race condition in GitHub Enterprise Server allowed an outside collaborator to be added while a repository is being transferred. GitHub has requested CVE ID CVE-2023-6803 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: Due to an insufficient entropy vulnerability, an attacker could brute force a user invitation to the Management Console. To exploit this vulnerability, an attacker would have needed knowledge that a user invitation was pending. This vulnerability was reported via the GitHub Bug Bounty program and assigned CVE-2023-46648.

MEDIUM: An insertion of sensitive information into log file vulnerability was identified in the log files for a GitHub Enterprise Server backend service that could permit an adversary in the middle attack when combined with other phishing techniques. To exploit this, an attacker would need access to the log files for the GitHub Enterprise Server instance, a backup archive created with GitHub Enterprise Server Backup Utilities, or a service which received streamed logs. GitHub has requested CVE ID CVE-2023-6746 for this vulnerability.

MEDIUM: An attacker could maintain admin access to a transferred repository in a race condition by making a GraphQL mutation to alter repository permissions during the transfer. GitHub has requested CVE ID CVE-2023-6690 for this vulnerability, which reported via the GitHub Bug Bounty program.

MEDIUM: Improper privilege management allowed arbitrary workflows to be committed and run using an improperly scoped personal access token. To exploit this, a workflow must have already existed in the target repository. GitHub has requested CVE ID CVE-2023-6804 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An incorrect authorization vulnerability was identified that allowed issue comments to be updated with an improperly scoped token. This vulnerability did not allow unauthorized access to any repository content as it also required contents.write and issues.read permissions. This vulnerability was reported via the GitHub Bug Bounty Program and has been assigned CVE-2023-51379.

MEDIUM: An incorrect authorization vulnerability was identified that allowed issue comments to be read with an improperly scoped token. This vulnerability was reported via the GitHub Bug Bounty Program and has been assigned CVE-2023-51380.

LOW: To render interactive maps in an instance's web UI using Azure Maps, GitHub Enterprise Server has migrated from use of an unsecure Azure Maps API token to a more secure access token provided by role-based access control (RBAC) in Entra ID. After upgrading to this release, to re-enable interactive maps, an administrator must reconfigure authentication to Azure Maps in the Management Console. For more information, see Configuring interactive maps.

To address scenarios that could lead to denial of service, HAProxy has been upgraded to version 2.8.4.

Packages have been updated to the latest security versions.

In rare cases, on an instance with GitHub Actions enabled, a failed check on a deleted repository could cause upgrades to a new version of GitHub Enterprise Server to fail.

Error messages were not shown when ghe-config-apply encountered specific kinds of errors.

In some environments, stale .backup log files could accumulate in the system.

On an instance hosted on AWS, when configuring GitHub Packages, virtual-hosted-style AWS S3 URLs would default to path-style URLs if a region-code was included. For more information, see Virtual hosting of buckets in the AWS documentation.

In some cases, when an administrator uploaded a custom TLS certificate, the certificate was not correctly installed on the instance.

Because the | character was not permitted, administrators could not add an SMTP username to authenticate with the Azure Communication Service.

In some cases, upgrades to GitHub Enterprise Server 3.11 could fail due to the Consul server failing to start.

Endpoints for the REST API's Manage GitHub Enterprise Server operation returned internal service error if cluster.conf was not found on the instance.

On an instance with GitHub Actions enabled, an issue with GH_TOKEN sometimes prevented GitHub Pages sites from building successfully in workflows.

An administrator could enable GitHub Connect on an instance with a license that does not support GitHub Connect.

On an instance with GitHub Connect enabled, some system users were incorrectly counted as consuming a license following license sync.

A user in the process of being converted into an organization could be added as a collaborator on a repository. This resulted in the new organizations owners unexpectedly receiving access to the repository.

When using ghe-migrator to import repositories into GitHub Enterprise Server, the conflicts and audit subcommands produced an invalid CSV file due to an extra log line appended to the file.

On an instance with a GitHub Advanced Security license and secret scanning enabled, dry runs sometimes incorrectly reported no results for custom patterns.

On an instance with a GitHub Advanced Security license and secret scanning enabled, webhooks for alert locations did not contain information about push protection bypasses.

On an instance with Dependabot updates enabled, Dependabot relies on the node installation provided by the actions runner instead of dynamically downloading.

To avoid negative effects on disk utilization, babeld log files have a maximum size of 15 GB.

When using ghe-migrator prepare to import an archive, a missing schema.json file results in an UnsupportedArchive error rather than an UnsupportedSchemaVersion error.

The audit log now tracks all failed password attempts individually. Previously, duplicate failed password attempts in sequence within the same day would be grouped into one failed password attempt, with a count field.

On an instance in a cluster configuration, administrators can identify the repository networks or gists that are common across a specified set of storage nodes using the spokesctl find-on-replicas command.

Hotpatch upgrades from GitHub Enterprise Server version 3.11.0 to 3.11.1 will result in the appliance losing network connectivity after a reboot. If you have upgraded to GitHub Enterprise Server 3.11.1 from 3.11.0 using a hotpatch upgrade, please contact GitHub Support for assistance. If you have upgraded directly to GitHub Enterprise Server 3.11.1 from previous versions (for example, from any version of GitHub Enterprise Server in the 3.9 or 3.10 series), your instance is not affected by this issue. [Updated 2023-12-27]

Custom firewall rules are removed during the upgrade process.

The GitHub Packages npm registry no longer returns a time value in metadata responses. This was done to allow for substantial performance improvements. We continue to have all the data necessary to return a time value as part of the metadata response and will resume returning this value in the future once we have solved the existing performance issues.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

Restoring backups with ghe-restore on a GHES cluster will exit prematurely if redis has not restarted properly.

The HAProxy version has been updated in this release. You may encounter elevated error rates when HAProxy is upgraded as part of a hotpatch upgrade to a GitHub Enterprise Server instance. These elevated error rates should resolve within 5 minutes of the hotpatch being applied.

Please note, when performing a hotpatch upgrade to GitHub Enterprise Server version 3.11.1 or higher you will encounter this known issue only if you are hotpatching from GitHub Enterprise Server version 3.11.0. [Updated 2024-01-03]

During periods of high traffic, interruptions in service may occur due to insufficient resource allocations for internal components. [Updated 2024-01-23]

The settings for enabling scheduled reminders were added unintentionally to this release. Scheduled reminders are not officially supported. [Updated: 2024-02-22]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Interactive maps in the web UI no longer allow authentication using an Azure Maps API key

• To allow users to render interactive maps in an instance's web UI by writing GeoJSON or TopoJSON syntax, GitHub Enterprise Server previously required a potentially unsecure API key for authentication with Azure Maps. If an administrator previously enabled interactive maps on an instance, the feature is disabled upon upgrade to this release.

  To re-enable interactive maps for your instance, you must configure an application on an Entra ID tenant that has access to Azure Maps using role-based access control (RBAC). For more information, see Configuring interactive maps and the security fixes for this release.

Instance administration

• Instance administrators can perform administrative tasks using the gh es extension for the GitHub CLI. The extension communicates with your instance's management API, so you don't need to SSH into the instance or write a custom application. For more information, see Administering your instance using the GitHub CLI.

Authentication

• To help users discover the required permissions for calls to a REST API endpoint, GitHub Enterprise Server returns the X-Accepted-GitHub-Permissions header for requests to endpoints that use fine-grained permissions, including requests from GitHub Apps. For more information, see the following articles.

  • Troubleshooting the REST API
  • Permissions required for fine-grained personal access tokens
  • Permissions required for GitHub Apps

Audit logs

• The web interface for enterprise, organization, and user audit logs include an expandable view that displays the full audit log payload for each event. Administrators and users can see the same event metadata when searching the audit log in the web interface or via streaming. For more information, see the following articles.

  • About the audit log for your enterprise
  • Reviewing the audit log for your organization
  • Reviewing your security log

GitHub Advanced Security

• On an instance with GitHub Actions enabled, in repositories that use default setup for code scanning, the default setup configuration updates automatically if GitHub detects new languages. Users can view a repository's language configuration for default setup from the repository's "Code security and analysis" settings page. Additionally, users can view information about setup and debug failed languages from the tools status page. For more information, see the following articles.

  • Configuring default setup for code scanning at scale
  • Configuring default setup for code scanning
  • About the tool status page for code scanning

• On an instance with GitHub Actions enabled, default setup for code scanning at the organization level is now generally available. For more information, see Configuring default setup for code scanning at scale and REST API endpoints for organizations in the REST API documentation.

• On an instance with GitHub Actions enabled, during configuration of default setup for code scanning, users can select either the "Extended" or "Default" query suite for eligible repositories in an organization. For more information, see CodeQL query suites and Configuring default setup for code scanning.

• On an instance with GitHub Actions enabled, to better protect active and inactive repositories, GitHub Enterprise Server automatically analyzes repositories that use a default setup for code scanning. The analysis runs on a weekly schedule and uses the most recent version of CodeQL. When configuring code scanning, the fixed time for the weekly scan is randomly chosen. The scan will take place at the same time every week, and the schedule is displayed after the setup is completed, so users can easily see when the next scheduled analysis will occur. The scheduled analysis will be automatically disabled if a repository has no activity for six months. Creation of a pull request or pushes to the repository will re-enable scheduled analysis.

• Code scanning default setup is available for Swift analysis with CodeQL. For more information, see Configuring default setup for code scanning.

• CodeQL 2.14.6 and later supports analysis of code written in Go 1.21. For more information, see Supported languages and frameworks in the CodeQL documentation.

• With CodeQL model packs for Java, users can improve code scanning results by ensuring that any custom Java libraries and frameworks used by their codebase are recognized by CodeQL. For more information, see the following documentation.

  • Editing your configuration of default setup
  • Customizing your advanced setup for code scanning
  • Using the CodeQL model editor in the CodeQL documentation

• For instances with GitHub Connect configured, code scanning with CodeQL supports Java codebases that use Project Lombok. Previously, code scanning users were able to scan Java applications that contained Lombok code, but all the contents of files containing Lombok code were either skipped or users had to apply a workaround to prepare the applications for scanning. Lombok features will now be automatically scanned without requiring any workaround.

  For more information about syncing the required GitHub Actions workflow to scan Lombok code, see Configuring code scanning for your appliance.

• Push protection for secret scanning is now generally available. For more information, see About push protection.

• To prevent the leak of tokens where users work outside of code, secret scanning detects tokens in both new and historical issue titles, descriptions, and comments. When a new token type is added to secret scanning, GitHub Enterprise Server scans for matches automatically. This expanded coverage also detects and surfaces secrets that match any custom pattern defined at the repository, organization, or enterprise level. These secrets appear both in the web interface and in queries to the REST API. For more information, see About secret scanning and Defining custom patterns for secret scanning.

• Users can view metrics associated with push protection usage across an organization. The overview shows a summary of blocks and bypasses, as well as more granular metrics. For more information, see Assessing your code security risk.

• A new REST API endpoint is available for dataflow analysis using custom CodeQL queries. The new endpoint offers additional flexibility, includes improvements that prevent common pitfalls with the old API, and improves the performance of query evaluation by five percent. For more information, see New dataflow API for writing custom CodeQL queries in the GitHub Changelog.

Dependabot

• For developers who manage Node.js dependencies using the pnpm package manager, pnpm is fully supported by dependency graph, Dependabot alerts, and Dependabot security updates. For more information about securing your supply chain with Dependabot, see Keeping your supply chain secure with Dependabot.

• Developers can enforce policies related to vulnerabilities and licenses in pull requests for complex ecosystems with transitive dependencies like Gradle and Scala. Dependency review supports dependencies from the dependency submission API. For more information, see the following articles.

  • About dependency review
  • About dependency review
  • Using the dependency submission API

• To control how Dependabot structures pull requests and improve mergeability, users can implement flexible grouping options in dependabot.yml. You can also control Dependabot's behavior for groups using comment commands. For more information, see Dependabot options reference and Managing pull requests for dependency updates.

• Dependabot can open pull requests for Swift and Gradle dependencies.

  • Users can also configure scheduled updates for Swift dependencies using dependabot.yml.
  • If users have used the REST API for dependency submission to upload Gradle dependencies to the dependency graph and receive Dependabot alerts for those dependencies, Dependabot will try to open a pull request to resolve security updates enabled for the repository.

  For more information, see Configuring Dependabot security updates.

• Responses from REST API endpoints for repositories display whether Dependabot security updates are enabled or disabled. Users can also enable or disable security updates for a repository using the REST API. For more information, see REST API endpoints for repositories in the REST API documentation.

• When Dependabot is first enabled, GitHub will not send notifications for all vulnerable dependencies found in the repository, only for new vulnerable dependencies ifentified after Dependabot is enabled. For more information, see Configuring notifications for Dependabot alerts.

Code security

• To assess risks to code security and ensure adoption of features to improve code security, the "Security risk" and "Security coverage" pages for organizations and the entire instance are generally available. Additionally, the alert-centric pages for Dependabot, code scanning, and secret scanning are also now generally available. For more information, see Assessing your code security risk and Assessing adoption of code security features.

• Users can take advantage of the GitHub Advisory Database using the REST API. The Advisory Database is a free, open-source list of actionable security advisories and CVEs. API responses include machine-readable mappings to the ecosystem, package name, and affected versions of impacted software. For more information, see REST API endpoints for global security advisories in the REST API documentation.

GitHub Actions

• For self-hosted GitHub Actions runners on this GitHub Enterprise Server release, the minimum required version of the GitHub Actions Runner application is 2.309.0. See the release notes for this version in the actions/runner repository. If your instance uses ephemeral self-hosted runners and you've disabled automatic updates, you must upgrade your runners to this version of the Runner application before upgrading your instance to this GitHub Enterprise Server release. [Updated: 2024-04-25]

• To better navigate, trace, understand, and monitor deployments, users can view and track the full history of deployments in a repository or filter across environments. For more information, see Viewing deployment history.

• Users can improve the security of deployment environments by configuring a branch protection policy to only allow specific branches to deploy to an environment. Additionally, the following security improvements apply to environments.

  • GitHub Enterprise Server blocks runs triggered from forks with branch names that match the protected branch's name.
  • Tags with the same name as a protected branch cannot deploy to the environments with a branch protection configuration.

  For more information, see Managing environments for deployment.

• On an instance with GitHub Actions enabled and a configuration for deployment environments, administrators for environments can improve the security of deployments by enforcing a review by someone other than the person who triggered the run. This option prevents required reviewers from self-reviewing to trigger workflows. For more information, see Managing environments for deployment.

Organizations

• Organization owners can signal that an organization is no longer actively maintained by archiving the organization. For more information, see Archiving an organization.

Repositories

• Users can govern protections for branches and tags in a repository using repository rules. To govern the protections for all of an organization's repositories, users can also enable rulesets for an organization. Contributors to a repository can see which rules apply via the web interface, Git, or the GitHub CLI. For more information, see About rulesets.

• Users can create new repositories with predefined attributes using query parameters. For example, a user can create a URL that prepopulates information about the repository like the name, description, visibility, and more. For more information, see Creating a new repository.

• Users can more easily understand changes to a repository using the activity view. For more information, see Using the activity view to see changes to a repository.

Issues

• Users can automatically add a new issue to projects using a custom issue form by defining projects in the issue template. For more information, see Syntax for issue forms.

Projects

• Users can review items in a project view broken down by a certain field value. For more information, see Customizing the table layout.

• Users can create charts to visualize current project items, or visualize project items over time. For more information, see About insights for Projects.

Accessibility

• To improve the visibility of links with blocks of text in the web interface for GitHub Enterprise Server, users can apply underline styling. For more information, see Managing accessibility settings.

The speed of restoration operations with GitHub Enterprise Server Backup Utilities has increased.

Configuration runs now correspond with a unique ID. During the run, the log remains at /data/user/common/ghe-config.log. After the run, the instance rotates the log's contents into /data/user/config-apply/logs/YYYYMMDD/ghe-config.HOSTNAME.ID.log, where YYYYMMDD is the date of the run, HOSTNAME is the hostname of the node, and ID is the ID of the run. For more information, see About system logs.

Field names for some service logs on GitHub Enterprise Server have changed as part of GitHub's gradual migration to internal semantic conventions for OpenTelemetry. Additional field names were changed in GitHub Enterprise Server 3.9 and 3.10. If any tooling or processes in your environment rely on specific field names within logs, or log entries in specific files, the following changes may affect you.

• level is now SeverityText.
• log_message, msg, or message is now Body.
• now is now Timestamp.
• Custom field names such as gh.repo.id or graphql.operation.name use semantic names.
• Log statements that the instance would previously write to auth.log, ldap.log, or ldap-sync.log now appear in containerized logs for github-unicorn if the statement originated from a web request, or in logs for github-resqued if the statement originated from a background job. For more information about containerized logs, see About system logs.

For a full list of mappings, download the OpenTelemetry attribute mapping CSV for GitHub Enterprise Server 3.9, 3.10, and 3.11.

On an instance that uses built-in authentication or LDAP, if two-factor authentication (2FA) is configured for an organization, a user could use a TOTP code multiple times within the code's window of validity during authentication or when entering sudo mode for sensitive actions. To improve security, this reuse is no longer allowed. External systems with a scripted login flow across multiple parallel jobs may stop working as a result of this change.

For more information about 2FA, see the following articles.

• Requiring two-factor authentication for an organization
• Sudo mode
• Configuring two-factor authentication

On an instance with a GitHub Advanced Security license, during analysis of Python projects with code scanning using CodeQL and an advanced setup, GitHub Enterprise Server would automatically install dependencies for the project. Due to improvements to CodeQL, GitHub Enterprise Server no longer needs to fetch these dependencies to analyze a codebase. To improve scan times for Python projects, automatic dependency installation is disabled.

If you configured code scanning with CodeQL via advanced setup to disable dependency installation, GitHub recommends setting setup-python-dependencies to false for the configuration. For more information, see Customizing your advanced setup for code scanning.

On an instance with Dependabot enabled, due to misconfiguration or incompatible versions, Dependabot jobs for a repository can fail. After 30 failed runs, subsequent scheduled jobs will fail immediately until you trigger a check for updates from the dependency graph, or until you update a manifest file. Jobs for Dependabot security updates will still trigger normally.

On an instance with GitHub Advanced Security, to help users more efficiently review and filter code scanning alerts at scale using the REST API, the updated_at field in API responses is improved. The updated_at timestamp now represents an alert's most recent state change on the branch that you requested. State changes include an alert being introduced, fixed, dismissed, reopened, or reintroduced. Previously, the updated_at timestamp changed frequently, whenever an alert was found in an analysis or the alert state changed. For more information about using the REST API to retrieve code scanning alerts, see REST API endpoints for code scanning in the REST API documentation.

On an instance with Dependabot enabled, the following improvements apply to the repository view for dependency graph, available from the repository's "Insights" tab.

• Users can search by package name from a paginated list of all dependencies.
• Dependency licenses are displayed.
• Dependabot alerts appear for dependencies, sorted by severity, and link to the Dependabot alerts and the Dependabot update pull request where applicable.

For more information about the dependency graph, see About the dependency graph.

After first enabling Dependabot on an instance, GitHub Enterprise Server will no longer send web or email notifications for repositories that are initially populated with Dependabot alerts. This allows you to review the new Dependabot alerts for a repository, organization, or the entire instance without immediately notifying other users of existing alerts.

On an instance with GitHub Actions enabled, workflows that use Node.js 16 or earlier will log a warning. Node.js 16 has been end-of-life since September 2023.

• Workflow authors should update actions to run on Node.js 20. For more information, see Metadata syntax for GitHub Actions.
• Users with workflows that use Node.js should specify Node.js 20 or later in the workflows using versioned actions. For more information, see Workflow syntax for GitHub Actions.

[Updated: 2024-03-05]

On an instance with GitHub Actions enabled and runners using GitHub Actions Runner 2.309.0 or later, users can no longer use GITHUB_ENV to set the NODE_OPTIONS environment variable in workflows. Workflows that set NODE_OPTIONS as an environment variable will now log the following error. For more information, see Workflow commands for GitHub Actions and the v2.309.0 release in the actions/runner repository on GitHub.com.

Can't store NODE_OPTIONS output parameter using '$GITHUB_ENV' command.

Users can quickly take action on multiple items in a group, or the group itself, using the ••• button in a table, board, or roadmap.

Users can break out items in a project by workstreams, team members, priorities, or other groupings using a swimlane view. For more information, see Customizing the board layout.

Users can view view the template used to create a project from a project's settings.

When scrolling through a project, group headers are now sticky.

The colors for single-select fields in a project have been updated, so users see the same colors within the field picker and within project views.

Users create can create issues in a project view that's grouped by repository in the board layout by clicking "Create new issue", or by starting to type the issue's title.

An upgrade to GitHub Enterprise Server 3.11.0 may fail, hanging on the "Reloading system services" screen. The following error will appear in /var/log/syslog.

agent: Error starting agent: error="Failed to start Consul server: Failed to start Raft: failed to load any existing snapshots"

This issue is resolved in GitHub Enterprise Server 3.11.1. When upgrading to a release in the 3.11 series, upgrade to 3.11.1 or later. [Updated 2023-12-21]

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a high-availability configuration, passive replica nodes accept Git client requests and forward the requests to the primary node.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In GitHub Enterprise Server 3.10 and later, the requirements for TLS security levels have changed due to an upgrade to containers in the underlying OS. On an instance with GitHub Actions enabled and a custom TLS certificate, users may experience disruptions with workflow runs if the TLS certificate uses weak encryption. Workflow runs will not trigger, and the following error message will appear in system logs for babeld.

CA certificate key too weak

To resolve this issue, confirm that your certificate complies with level 2 of the OpenSSL security specification. For more information, see SSL_CTX_set_security_level in the OpenSSL docs. For more information about reviewing your instance's logs, see "About system logs."

If the error appears in babeld logs because your TLS certificate does not comply with level 2 of the specification, you must create and upload a new certificate with stronger security before you upgrade to GitHub Enterprise Server 3.10 or later. For more information, see "Configuring TLS."

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

During periods of high traffic, interruptions in service may occur due to insufficient resource allocations for internal components. [Updated 2024-01-23]

Redundant messages may cause an increase in the volume of events logged in /var/log/syslog. [Updated: 2024-03-08]

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Enterprise-level security overview is deprecated

• The enterprise-level "Security overview" page is deprecated in favor of the new "Security risk" and "Security coverage" pages. For more information, see Assessing your code security risk and Assessing adoption of code security features.

Dependabot updates no longer support Python 3.6 or 3.7

• Dependabot updates no longer support Python 3.6 or 3.7, which have reached end-of-life. If a user's code uses these versions, Dependabot will no longer be able to open pull requests in your repository and will log errors. Update to Python 3.8 or later to ensure your code is secure and Dependabot can still run.

  Users will continue to receive Dependabot alerts for dependencies with known vulnerabilities. To resolve these alerts, users can manually upgrade the affected package.

  For more information about Python releases, see Status of Python versions on the Python website. For more information about supported package managers for Dependabot, see About Dependabot version updates.

Upcoming deprecation of team discussions

• GitHub will deprecate team discussions for users in GitHub Enterprise Server 3.13. In GitHub Enterprise Server 3.11, a banner appears atop teams' discussions with information about the deprecation, including a link to tooling to migrate existing team discussions to GitHub Discussions. For more information, see About team discussions and About discussions. [Updated: 2024-03-04]

Elasticsearch index `repository-stack` is no longer in use

• The Elasticsearch index repository-stacks is no longer in use. [Updated: 2024-06-24]

The Changes section previously indicated that users should update GitHub Actions workflows and actions to run on Node.js 16. Node.js 16 has reached end of life, and users should instead update actions and workflows to run on Node.js 20 or later. [Updated: 2024-03-05]