Code to Cloud Attacks: From Github PAT to Cloud Control Plane

How attackers are leveraging compromised employee GitHub Personal Access Tokens to compromise cloud environments.

Threat actors are targeting GitHub Personal Access Tokens (PAT) as high-value entry points, leveraging compromised PATs to gain initial access to victim cloud environments. Historically, compromised PATs were used to identify and obtain secrets hard-coded in source code. However, the Wiz Customer Incident Response Team (Wiz CIRT) recently responded to multiple incidents where threat actors exploited leaked PATs (with standard developer permissions) to abuse trust between code repositories and cloud accounts.

Based on recent incidents investigated by the team, this blog post will detail unique methods attackers are currently using in the wild to leverage compromised PATs. It will describe a real attack flow in which a GitHub PAT was leveraged for cross-cloud lateral movement from GitHub to Cloud Service Provider (CSP) control plane. The examples are focused on the GitHub platform, though the underlying process is also applicable to GitLab.

Observed Malicious Techniques

Attackers leveraged compromised PATs to discover GitHub Action Secrets names in the codebase, and used them in newly created malicious workflows to execute code and obtain CSP secrets.

Secret Discovery

Wiz CIRT has observed incidents involving the misuse of GitHub Action Secrets, which are intended for the secure storage of credentials to be used within CI/CD workflows. While enumerating Actions Secrets is gated behind admin scopes in GitHub’s API, a threat actor with basic read permissions can still discover the secret names used within the codebase. 

The secret name is embedded directly in the workflow's yaml code, always accessed with the syntax: ${{ secrets.SECRET_NAME }}. Instead of cloning the entire repository, threat actors can leverage GitHub’s API code search functionality across the organization to look for this specific syntax and search for all the secrets used in the code repository. For example, an API call searching for all appearances of the string ${{ secrets. in the organization code will be:

curl -L -H "Authorization: Bearer $GH_TOKEN" -H 
"X-GitHub-Api-Version: 2022-11-28" 
"https://api.github.com/search/code?q=org:$GH_ORG_NAME+%24%7B%7B%20secrets."

Crucially, this type of active secret discovery can’t be monitored as these search API calls are not logged, despite contradictory claims in current GitHub Enterprise audit logs documentation. Wiz is currently working with GitHub support to resolve this discrepancy.

Wiz data shows that 73% of organizations that utilize GitHub Action Secrets store CSP credentials within them, and it is recommended to manage those secrets with caution and risk awareness, giving the associated principals the minimal permissions necessary.

Execution

Next, if the PAT has write permissions, attackers can leverage the GitHub Actions platform to execute malicious code. 

A possible process consists of the creation of custom workflows and pull requests designed to trigger the execution of an Action, which essentially allows the threat actor to run any arbitrary piece of code and retrieve the execution results by fetching the Action’s run log.

Possible attack flow for a threat actor leveraging a PAT with write permissions.

Malicious activity originating from the GitHub Actions ecosystem significantly obscures the true origin.
GitHub-hosted Actions run from GitHub-managed resources using legitimate, shared IP addresses that cannot be reliably marked as malicious. While GitHub provides the CIDR ranges for their Action exit nodes, these are often very broad and contain IPs not exclusively used for GitHub activity.
Self-hosted runners introduce potential additional risks because the Actions gain access to the data residing on the same resource hosting the runner. The specific severity of these risks depends on the runner's configuration.

Due to all of the above, executing malicious code via the GitHub Actions ecosystem provides several advantages for the attacker:

  • Secret abuse - It allows the threat actor to directly use the Actions Secrets whose names have been harvested during the Discovery phase.

  • Origin Impersonation - It enables malicious code to assume the true origin of a workflow by exploiting the trusted status of a legitimate runner.

  • Possible access to other resources - In case the action code is reused between workflows, or misconfigured on self-hosted runners.

Credential Access

Lastly, attackers can compromise the discovered Actions Secrets by utilizing GitHub Actions execution capabilities. 

GitHub automatically masks the content of secrets printed directly to the Action logs, and even the Base64 encoding of those secrets.

jobs:
 print-secret-content:
   steps:
   - name: Print
     run: |
       echo "Secret:"
       echo ${{ secrets.SECRET_NAME }}
       echo "====="
       echo "Base64 Secret:"
       echo -n "${{ secrets.SECRET_NAME }}" | base64
Example GitHub Action trying to print the secret content, and the resulting logs.

But, attackers have leveraged a simple but effective method to bypass this control. Any further manipulation of the secret, for example encoding it twice using Base64, allows attackers to successfully exfiltrate the raw secret value via the Action logs.

echo "Base64 * 2 Secret:"
echo -n "${{ secrets.SECRET_NAME }}" | base64 -w 0 | base64
Altered GitHub Action example, successfully printing the secret content to the Action logs. 

Threat actors have also been observed exfiltrating secrets to a webhook endpoint they control, completely bypassing Action logs.

A more sophisticated abuse pattern Wiz CIRT has documented is the direct use of these secrets to run code in target CSPs. For example, an attacker can use a CSP credential, to run a command that creates new CSP cloud credentials (e.g., an Access Key for the same user). Since these newly created credentials are not masked by GitHub, they can be printed to the Action logs and used for persistent access. While the logs are not publicly available if the repository is private, the PAT can be leveraged to read the logs, and obtain the printed credentials.
Theoretically, threat actors could run their entire planned activity from GitHub Actions, but Wiz CIRT has only observed this as a stepping-stone to achieve lateral movement into the organization's core cloud environment.

Defense Evasion

A final step Wiz CIRT observed in these attacks is the use of the compromised PAT for Defense Evasion.
A PAT with write scopes can be abused to perform critical deletion activities, including the removal of workflow logs, workflow runs, pull requests and created branches. As workflow logs are rarely streamed into a SIEM, by executing these deletion commands, the threat actor can evade detection and investigation, leaving security teams to speculate regarding the full scope of the attack.

Lateral Movement

After successfully obtaining valid cloud credentials, whether by scraping plain-text secrets, by bypassing GitHub's masking to retrieve the content of Action secrets or by creating new cloud keys, threat actors were observed using these keys for cross-cloud lateral movement, into the target's cloud infrastructure.
At this point, the initial compromise of the GitHub PAT is complete, and the attacker has transitioned from a source code compromise to a full-cloud incident that allows broader abuse opportunities.

Additional Techniques

Beyond the steps detailed above to allow cross-cloud lateral movement from GitHub to CSP control plane, there are obvious threats of Exfiltration from code repositories, Impact, direct Credential Access and Lateral Movement.

In the case of tj-actions supply-chain attack, threat actors compromised the broadly used GitHub Action tj-actions/changed-files with a malicious payload that affected any repository using this action and compromised all the public repositories using it by leaking their secrets in logs. This poses high security risks for Impact in a supply-chain scenario caused by a legitimate developer's PAT being abused to introduce malicious code into trusted libraries.

Additional high-profile cases are known as the Shai-Hulud NPM supply-chain attacks. As reported previously by Wiz, attackers used open-source tools like TruffleHog for Credential Access, and leaked them to GitHub using PATs found by the tool. 

A developer's PAT with access to a GitHub organization makes even private repositories (typically considered more secure for sensitive data) unsafe for storing secrets. Based on Wiz data, 45% of organizations have plain-text cloud keys stored directly in their private code repositories, compared to just 8% in public repositories. This stark difference indicates a false sense of security, where organizations trust plain-text secrets to the privacy of their code repositories, leaving them severely vulnerable to this specific attack vector.

The Investigation Challenge

Limitations in log availability, combined with threat actor evasion techniques, create a significant investigative challenge. Crucially, audit log gaps exist, as code-search API calls are not logged at all. 

To conduct a comprehensive incident investigation on GitHub and determine the scope of an incident, including the content of malicious files, real-time enterprise audit log streaming is essential. This is the only method to capture detailed API request information. Without this preservation, the only remaining evidence is limited to undeleted workflow files and Actions logs from threat actors who did not employ defense evasion.
In addition, GitHub's audit logs do not enable IP tracking for requests by default, this is an opt-in setting. Therefore, it is critical to capture essential artifacts by both enabling IP tracking and utilizing real-time audit log streaming.

Finally, a question that often remains unanswered is how a PAT was initially compromised. Wiz CIRT suggests that incidents like the recent Shai-Hulud attack may be a starting point, as it publicly exposed thousands of PATs. In this attack, secrets were frequently exfiltrated cross-victim, which left many organizations unaware of the compromise and consequently unremediated.

How Wiz Can Help

To perform effective post-incident forensics and understand the full scope of an incident, an organization needs to see exactly what API calls the attacker made. Wiz Defend enables this by supporting GitHub Enterprise log streaming and presenting these logs in the Cloud Events page. These logs are the only way to gain visibility into the API requests made against the enterprise. By correlating these logs with other security findings, Wiz helps contextualize the suspicious activity and establish the incident timeline.

Wiz also monitors abnormal PAT behavior and creates Detections alerting on high volume activity, unusual ASO, unusual Action usage and secrets exfiltrated through workflows. These Detections are created, maintained, and continuously updated by the Wiz Threat Research team, ensuring the rules reflect the latest cloud threat intelligence.

Additionally, the Wiz Secret Scanner detects cloud keys and other secrets across the entire codebase, including Git history and all branches. This detection helps identify and track secrets that should be rotated or revoked. Wiz customers can check the Secret Findings page to review secrets already in their code, alongside the cloud context and potential impact. Secrets can be prevented from being committed by using Wiz IDE extension and precommit hooks on developers’ workstations, or by using Wiz scanners that integrate directly into the CI/CD pipelines and VCS to catch them later in the process. 

When a secret is already exposed, Wiz Defend monitors for misuse, alerts on secret-scanning activity in the cloud environment, and helps teams investigate and remediate the incident with workflows built for cloud specific attacks.

The Wiz Customer Incident Response Team is available to anyone experiencing a cloud incident.

References

Continue reading

Get a personalized demo

Ready to see Wiz in action?

"Best User Experience I have ever seen, provides full visibility to cloud workloads."
David EstlickCISO
"Wiz provides a single pane of glass to see what is going on in our cloud environments."
Adam FletcherChief Security Officer
"We know that if Wiz identifies something as critical, it actually is."
Greg PoniatowskiHead of Threat and Vulnerability Management