Skip to content

Commit

Permalink
feat: Support self-hosted GitHub Enterprise servers
Browse files Browse the repository at this point in the history
  • Loading branch information
twpayne committed May 2, 2024
1 parent a1196a7 commit b29a454
Show file tree
Hide file tree
Showing 13 changed files with 245 additions and 101 deletions.
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# `gitHubLatestRelease` *owner-repo*
# `gitHubLatestRelease` *host-owner-repo*

`gitHubLatestRelease` calls the GitHub API to retrieve the latest release about
the given *owner-repo*, returning structured data as defined by the [GitHub Go
API
the given *host-owner-repo*, returning structured data as defined by the [GitHub
Go API
bindings](https://pkg.go.dev/github.com/google/go-github/v57/github#RepositoryRelease).

Calls to `gitHubLatestRelease` are cached so calling `gitHubLatestRelease` with
the same *owner-repo* will only result in one call to the GitHub API.
the same *host-owner-repo* will only result in one call to the GitHub API.

!!! example

Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# `gitHubLatestReleaseAssetURL` *owner-repo* *pattern*
# `gitHubLatestReleaseAssetURL` *host-owner-repo* *pattern*

`gitHubLatestReleaseAssetURL` calls the GitHub API to retrieve the latest
release about the given *owner-repo*, returning structured data as defined by
the [GitHub Go API
release about the given *host-owner-repo*, returning structured data as defined
by the [GitHub Go API
bindings](https://pkg.go.dev/github.com/google/go-github/v61/github#RepositoryRelease).
It then iterates through all the release's assets, returning the first one that
matches *pattern*. *pattern* is a shell pattern as [described in
`path.Match`](https://pkg.go.dev/path#Match).

Calls to `gitHubLatestReleaseAssetURL` are cached so calling
`gitHubLatestReleaseAssetURL` with the same *owner-repo* will only result in one
call to the GitHub API.
`gitHubLatestReleaseAssetURL` with the same *host-owner-repo* will only result
in one call to the GitHub API.

!!! example

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# `gitHubLatestTag` *owner-repo*
# `gitHubLatestTag` *host-owner-repo*

`gitHubLatestTag` calls the GitHub API to retrieve the latest tag for the given
*owner-repo*, returning structured data as defined by the [GitHub Go API
*host-owner-repo*, returning structured data as defined by the [GitHub Go API
bindings](https://pkg.go.dev/github.com/google/go-github/v57/github#RepositoryTag).

Calls to `gitHubLatestTag` are cached the same as [`githubTags`](gitHubTags.md),
so calling `gitHubLatestTag` with the same *owner-repo* will only result in one
call to the GitHub API.
so calling `gitHubLatestTag` with the same *host-owner-repo* will only result in
one call to the GitHub API.

!!! example

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# `gitHubReleases` *owner-repo*
# `gitHubReleases` *host-owner-repo*

`gitHubReleases` calls the GitHub API to retrieve the first page of releases for
the given *owner-repo*, returning structured data as defined by the [GitHub Go
API
the given *host-owner-repo*, returning structured data as defined by the [GitHub
Go API
bindings](https://pkg.go.dev/github.com/google/go-github/v57/github#RepositoryRelease).

Calls to `gitHubReleases` are cached so calling `gitHubReleases` with the same
*owner-repo* will only result in one call to the GitHub API.
*host-owner-repo* will only result in one call to the GitHub API.

!!! example

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# `gitHubTags` *owner-repo*
# `gitHubTags` *host-owner-repo*

`gitHubTags` calls the GitHub API to retrieve the first page of tags for
the given *owner-repo*, returning structured data as defined by the [GitHub Go
`gitHubTags` calls the GitHub API to retrieve the first page of tags for the
given *host-owner-repo*, returning structured data as defined by the [GitHub Go
API
bindings](https://pkg.go.dev/github.com/google/go-github/v57/github#RepositoryTag).

Calls to `gitHubTags` are cached so calling `gitHubTags` with the
same *owner-repo* will only result in one call to the GitHub API.
same *host-owner-repo* will only result in one call to the GitHub API.

!!! example

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# GitHub functions

The `gitHub*` template functions return data from the GitHub API.
The `gitHub*` template functions return data from GitHub or GitHub Enterprise
using the GitHub API.

All functions take a *host-owner-repo* argument of the form:

[host/]owner/repo

The optional `host` specifies the host and defaults to `github.com` if omitted.
`owner` and `repo` specify the repository owner and name respectively.

By default, chezmoi makes anonymous GitHub API requests, which are subject to
[GitHub's rate
Expand All @@ -9,13 +17,35 @@ limits](https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-
from identical GitHub API requests for the period defined in
`gitHub.refreshPeriod` (default one minute).

If any of the environment variables `$CHEZMOI_GITHUB_ACCESS_TOKEN`,
`$GITHUB_ACCESS_TOKEN`, or `$GITHUB_TOKEN` are found, then the first one found
will be used to authenticate the GitHub API requests which have a higher rate
limit (currently 5,000 requests per hour per user).
For `github.com` repos, if any of the environment variables

* `$CHEZMOI_GITHUB_ACCESS_TOKEN`
* `$CHEZMOI_GITHUB_TOKEN`
* `$GITHUB_ACCESS_TOKEN`
* `$GITHUB_TOKEN`

are found, then the first one found will be used to
authenticate the GitHub API requests which have a higher rate limit (currently
5,000 requests per hour per user).

In practice, GitHub API rate limits are high enough chezmoi's caching of results
mean that you should rarely need to set a token, unless you are sharing a source
IP address with many other GitHub users. If needed, the GitHub documentation
describes how to [create a personal access
IP address with many other GitHub users or accessing a private repo. If needed,
the GitHub documentation describes how to [create a personal access
token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token).

For non-`github.com` repos, e.g. self-hosted GitHub Enterprise repos, if any of
the environment variables

* `$CHEZMOI_HOST_ACCESS_TOKEN`
* `$HOST_ACCESS_TOKEN`

are found then the first one will be used to authenticate requests, where `HOST`
is the host converted to uppercase and with all non-letter characters replaced
with underscores.

!!! example

Given the host `git.example.com`, chezmoi will look for the
`$CHEZMOI_GIT_EXAMPLE_COM_ACCESS_TOKEN` and `$GIT_EXAMPLE_COM_ACCESS_TOKEN`
environment variables.
49 changes: 41 additions & 8 deletions internal/chezmoi/github.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,8 @@ import (

// NewGitHubClient returns a new github.Client configured with an access token
// and a http client, if available.
func NewGitHubClient(ctx context.Context, httpClient *http.Client) *github.Client {
for _, key := range []string{
"CHEZMOI_GITHUB_ACCESS_TOKEN",
"CHEZMOI_GITHUB_TOKEN",
"GITHUB_ACCESS_TOKEN",
"GITHUB_TOKEN",
} {
func NewGitHubClient(ctx context.Context, httpClient *http.Client, host string) (*github.Client, error) {
for _, key := range accessTokenEnvKeys(host) {
if accessToken := os.Getenv(key); accessToken != "" {
httpClient = oauth2.NewClient(
context.WithValue(ctx, oauth2.HTTPClient, httpClient),
Expand All @@ -27,5 +22,43 @@ func NewGitHubClient(ctx context.Context, httpClient *http.Client) *github.Clien
break
}
}
return github.NewClient(httpClient)
gitHubClient := github.NewClient(httpClient)
if host == "github.com" {
return gitHubClient, nil
}
return gitHubClient.WithEnterpriseURLs(
"https://"+host+"/api/v3/",
"https://"+host+"/api/uploads/",
)
}

func accessTokenEnvKeys(host string) []string {
if host == "github.com" {
return []string{
"CHEZMOI_GITHUB_ACCESS_TOKEN",
"CHEZMOI_GITHUB_TOKEN",
"GITHUB_ACCESS_TOKEN",
"GITHUB_TOKEN",
}
}
hostKey := makeHostKey(host)
return []string{
"CHEZMOI_" + hostKey + "_ACCESS_TOKEN",
hostKey + "_ACCESS_TOKEN",
}
}

func makeHostKey(host string) string {
hostKey := make([]byte, 0, len(host))
for _, b := range []byte(host) {
switch {
case 'A' <= b && b <= 'Z':
hostKey = append(hostKey, b)
case 'a' <= b && b <= 'z':
hostKey = append(hostKey, b-'a'+'A')
default:
hostKey = append(hostKey, '_')
}
}
return string(hostKey)
}
35 changes: 35 additions & 0 deletions internal/chezmoi/github_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package chezmoi

import (
"testing"

"github.com/alecthomas/assert/v2"
)

func TestAccessTokenEnvKeys(t *testing.T) {
for _, tc := range []struct {
host string
expected []string
}{
{
host: "github.com",
expected: []string{
"CHEZMOI_GITHUB_ACCESS_TOKEN",
"CHEZMOI_GITHUB_TOKEN",
"GITHUB_ACCESS_TOKEN",
"GITHUB_TOKEN",
},
},
{
host: "git.example.com",
expected: []string{
"CHEZMOI_GIT_EXAMPLE_COM_ACCESS_TOKEN",
"GIT_EXAMPLE_COM_ACCESS_TOKEN",
},
},
} {
t.Run(tc.host, func(t *testing.T) {
assert.Equal(t, tc.expected, accessTokenEnvKeys(tc.host))
})
}
}
5 changes: 5 additions & 0 deletions internal/cmd/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,11 @@ func newConfig(options ...configOption) (*Config, error) {
homeDir: userHomeDir,
templateFuncs: sprig.TxtFuncMap(),

// Password manager data.
gitHub: gitHubData{
clientsByHost: make(map[string]gitHubClientResult),
},

// Command configurations.
apply: applyCmdConfig{
filter: chezmoi.NewEntryTypeFilter(chezmoi.EntryTypesAll, chezmoi.EntryTypesNone),
Expand Down
5 changes: 4 additions & 1 deletion internal/cmd/doctorcmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,10 @@ func (c *latestVersionCheck) Run(system chezmoi.System, homeDirAbsPath chezmoi.A

ctx := context.Background()

gitHubClient := chezmoi.NewGitHubClient(ctx, c.httpClient)
gitHubClient, err := chezmoi.NewGitHubClient(ctx, c.httpClient, "github.com")
if err != nil {
return checkResultFailed, err.Error()
}
rr, _, err := gitHubClient.Repositories.GetLatestRelease(ctx, "twpayne", "chezmoi")
var rateLimitErr *github.RateLimitError
var abuseRateLimitErr *github.AbuseRateLimitError
Expand Down
Loading

0 comments on commit b29a454

Please sign in to comment.