Add support for Forgejo/Codeberg

FAQS.md

@@ -5,7 +5,7 @@
 `gitbackup` currently has two operation modes.
 
 The first and original operating mode is to create clones of only your git repository. 
-This is supported for Bitbucket, GitHub and Gitlab. It runs `git clone` (the first time) and then
+This is supported for Bitbucket, GitHub, Gitlab, and Forgejo. It runs `git clone` (the first time) and then
 subsequently `git pull`. You can also configure it to perform a bare clone instead. This means that
 if you mess around your original repo, `gitbackup` will introduce the mess into your backup as well.
 

README.md

@@ -1,8 +1,9 @@
-# gitbackup - Backup your GitHub, GitLab, and Bitbucket repositories
+# gitbackup - Backup your GitHub, GitLab, Bitbucket, and Forgejo repositories
+
 Code Quality [![Go Report Card](https://goreportcard.com/badge/github.com/amitsaha/gitbackup)](https://goreportcard.com/report/github.com/amitsaha/gitbackup)
 [![.github/workflows/ci.yml](https://github.com/amitsaha/gitbackup/actions/workflows/ci.yml/badge.svg)](https://github.com/amitsaha/gitbackup/actions/workflows/ci.yml)
 
-- [gitbackup - Backup your GitHub, GitLab, and Bitbucket repositories](#gitbackup---backup-your-github-gitlab-and-bitbucket-repositories)
+- [gitbackup - Backup your GitHub, GitLab, Bitbucket, and Forgejo repositories](#gitbackup---backup-your-github-gitlab-and-bitbucket-repositories)
   - [Introduction](#introduction)
   - [Installing `gitbackup`](#installing-gitbackup)
   - [Using `gitbackup`](#using-gitbackup)
@@ -11,12 +12,14 @@ Code Quality [![Go Report Card](https://goreportcard.com/badge/github.com/amitsa
       - [Bitbucket](#bitbucket)
       - [GitHub](#github)
       - [GitLab](#gitlab)
+      - [Forgejo](#forgejo)
     - [Security and credentials](#security-and-credentials)
     - [Examples](#examples)
       - [Backing up your GitHub repositories](#backing-up-your-github-repositories)
       - [Backing up your GitLab repositories](#backing-up-your-gitlab-repositories)
       - [GitHub Enterprise or custom GitLab installation](#github-enterprise-or-custom-gitlab-installation)
       - [Backing up your Bitbucket repositories](#backing-up-your-bitbucket-repositories)
+      - [Backing up your Forgejo repositories](#backing-up-your-forgejo-repositories)
       - [Specifying a backup location](#specifying-a-backup-location)
       - [Cloning bare repositories](#cloning-bare-repositories)
       - [GitHub Migrations](#github-migrations)
@@ -25,11 +28,11 @@ Code Quality [![Go Report Card](https://goreportcard.com/badge/github.com/amitsa
 ## Introduction
 
 ``gitbackup`` is a tool to backup your git repositories from GitHub (including GitHub enterprise),
-GitLab (including custom GitLab installations), or Bitbucket.
+GitLab (including custom GitLab installations), Bitbucket, or Forgejo.
 
 ``gitbackup`` currently has two operation modes:
 
-- The first and original operating mode is to create clones of only your git repository. This is supported for Bitbucket, GitHub and Gitlab.
+- The first and original operating mode is to create clones of only your git repository. This is supported for GitHub, Gitlab, Bitbucket, and Forgejo.
 - The second operating mode is only available for GitHub where you can create a user migration (including orgs) which you get back as a .tar.gz
   file containing all the artefacts that GitHub supports via their Migration API.
 
@@ -47,11 +50,11 @@ If you are on MacOS, a community member has created a [Homebrew formula](https:/
 
 ``gitbackup`` requires a [GitHub API access token](https://github.com/blog/1509-personal-api-tokens) for
 backing up GitHub repositories, a [GitLab personal access token](https://gitlab.com/profile/personal_access_tokens)
-for GitLab repositories, and a username and [app password](https://bitbucket.org/account/settings/app-passwords/) for
-Bitbucket repositories.
+for GitLab repositories, a username and [app password](https://bitbucket.org/account/settings/app-passwords/) for
+Bitbucket repositories, or a [Forgejo access token][https://docs.codeberg.org/advanced/access-token/] for Forgejo.
 
-You can supply the tokens to ``gitbackup`` using ``GITHUB_TOKEN`` and ``GITLAB_TOKEN`` environment variables
-respectively, and the Bitbucket credentials with ``BITBUCKET_USERNAME`` and ``BITBUCKET_PASSWORD``.
+You can supply the tokens to ``gitbackup`` using ``GITHUB_TOKEN``, ``GITLAB_TOKEN``, or ``FORGEJO_TOKEN`` environment
+variables respectively, and the Bitbucket credentials with ``BITBUCKET_USERNAME`` and ``BITBUCKET_PASSWORD``.
 
 ### GitHub Specific oAuth App Flow
 
@@ -88,6 +91,13 @@ For the App password, the following permissions are required:
 - `api`: Grants complete read/write access to the API, including all groups and projects.
 For some reason, `read_user` and `read_repository` is not sufficient.
 
+#### Forgejo
+
+The following permissions are required:
+
+- `read:repository`
+- `read:user`
+
 ### Security and credentials
 
 When you provide the tokens via environment variables, they remain accessible in your shell history 
@@ -107,6 +117,8 @@ Usage of ./gitbackup:
         Backup directory
   -bare
         Clone bare repositories
+  -forgejo.repoType string
+        Repo types to backup (user, starred) (default "user")
   -githost.url string
         DNS of the custom Git host
   -github.createUserMigration
@@ -132,7 +144,7 @@ Usage of ./gitbackup:
   -ignore-private
         Ignore private repositories/projects
   -service string
-        Git Hosted Service Name (github/gitlab/bitbucket)
+        Git Hosted Service Name (github/gitlab/bitbucket/forgejo)
   -use-https-clone
         Use HTTPS for cloning instead of SSH
 ```
@@ -233,6 +245,27 @@ To backup all your Bitbucket repositories to the default backup directory (``$HO
 $ BITBUCKET_USERNAME=username BITBUCKET_PASSWORD=password gitbackup -service bitbucket
 ```
 
+#### Backing up your Forgejo repositories
+
+The `forgejo` service backs up `codeberg.org` by default.
+To back up all your Codeberg repositories to the default directory (``$HOME/.gitbackup/``):
+
+```lang=bash
+$ FORGEJO_TOKEN=access_token gitbackup -service forgejo
+```
+
+To back up a different Forgejo instance, specify `githost.url`:
+
+```lang=bash
+$ FORGEJO_TOKEN=access_token gitbackup -service forgejo -githost.url https://git.yourhost.com
+```
+
+To back up starred repositories instead of those you have access to, specify `forgejo.repoType`:
+
+```lang=bash
+$ FORGEJO_TOKEN=access_token gitbackup -service forgejo -forgejo.repoType starred
+```
+
 #### Specifying a backup location
 
 To specify a custom backup directory, we can use the ``backupdir`` flag:
@@ -242,8 +275,10 @@ $ GITHUB_TOKEN=secret$token gitbackup -service github -backupdir /data/
 ```
 
 This will create a ``github.com`` directory in ``/data`` and backup all your repositories there instead.
-Similarly, it will create a ``gitlab.com`` directory, if you are backing up repositories from ``gitlab``, and a
-``bitbucket.com`` directory if you are backing up from Bitbucket.
+Similarly, it will create a ``gitlab.com`` directory, if you are backing up repositories from ``gitlab``, a
+``bitbucket.com`` directory if you are backing up from Bitbucket, and a ``codeberg.org`` (or your custom host)
+directory if you are backing up from Forgejo.
+
 If you have specified a Git Host URL, it will create a directory structure ``data/host-url/``.
 
 

backup_test.go

@@ -149,6 +149,7 @@ func TestSetupBackupDir(t *testing.T) {
 
 	serviceGithubCustomUrl := "https://company.github.com"
 	serviceGitlabCustomUrl := "https://company.gitlab.com"
+	serviceForgejoCustomUrl := "https://company.forgejo.com"
 
 	var testConfigs = []struct {
 		backupRootDir  string
@@ -205,6 +206,30 @@ func TestSetupBackupDir(t *testing.T) {
 			"",
 			"/home/fakeuser/.gitbackup/bitbucket.org",
 		},
+		{
+			backupRoot,
+			"forgejo",
+			"",
+			"/my/backup/root/codeberg.org",
+		},
+		{
+			"",
+			"forgejo",
+			"",
+			"/home/fakeuser/.gitbackup/codeberg.org",
+		},
+		{
+			backupRoot,
+			"forgejo",
+			serviceForgejoCustomUrl,
+			"/my/backup/root/company.forgejo.com",
+		},
+		{
+			"",
+			"forgejo",
+			serviceForgejoCustomUrl,
+			"/home/fakeuser/.gitbackup/company.forgejo.com",
+		},
 	}
 
 	for _, tc := range testConfigs {

bitbucket.go

@@ -11,7 +11,7 @@ func getBitbucketRepositories(
 	client interface{},
 	service string, githubRepoType string, githubNamespaceWhitelist []string,
 	gitlabProjectVisibility string, gitlabProjectMembershipType string,
-	ignoreFork bool,
+	ignoreFork bool, forgejoRepoType string,
 ) ([]*Repository, error) {
 
 	if client == nil {

client.go

@@ -10,6 +10,7 @@ import (
 
 	"golang.org/x/oauth2"
 
+	forgejo "codeberg.org/mvdkleijn/forgejo-sdk/forgejo/v2"
 	"github.com/google/go-github/v34/github"
 	bitbucket "github.com/ktrysmt/go-bitbucket"
 	gitlab "github.com/xanzy/go-gitlab"
@@ -81,6 +82,8 @@ func newClient(service string, gitHostURL string) interface{} {
 		return newGitLabClient(gitHostURLParsed)
 	case "bitbucket":
 		return newBitbucketClient(gitHostURLParsed)
+	case "forgejo":
+		return newForgejoClient(gitHostURLParsed)
 	default:
 		return nil
 	}
@@ -181,3 +184,23 @@ func newBitbucketClient(gitHostURLParsed *url.URL) *bitbucket.Client {
 	}
 	return client
 }
+
+// newForgejoClient creates a new Forgejo client.
+func newForgejoClient(gitHostURLParsed *url.URL) *forgejo.Client {
+	forgejoToken := os.Getenv("FORGEJO_TOKEN")
+	if forgejoToken == "" {
+		log.Fatal("FORGEJO_TOKEN environment variable not set")
+	}
+
+	url := "https://" + knownServices["forgejo"]
+	if gitHostURLParsed != nil {
+		url = gitHostURLParsed.String()
+	}
+
+	client, err := forgejo.NewClient(url, forgejo.SetToken(forgejoToken), forgejo.SetForgejoVersion(""))
+	if err != nil {
+		log.Fatalf("Error creating forgejo client: %v", err)
+	}
+
+	return client
+}

client_test.go

@@ -4,6 +4,7 @@ import (
 	"net/url"
 	"testing"
 
+	forgejo "codeberg.org/mvdkleijn/forgejo-sdk/forgejo/v2"
 	"github.com/google/go-github/v34/github"
 	"github.com/ktrysmt/go-bitbucket"
 	gitlab "github.com/xanzy/go-gitlab"
@@ -44,6 +45,14 @@ func TestNewClient(t *testing.T) {
 	client = newClient("bitbucket", "")
 	client = client.(*bitbucket.Client)
 
+	// Client for codeberg
+	client = newClient("forgejo", "")
+	client = client.(*forgejo.Client)
+
+	// Client for forgejo
+	client = newClient("forgejo", customGitHost.String())
+	client = client.(*forgejo.Client)
+
 	// Not yet supported
 	client = newClient("notyetsupported", "")
 	if client != nil {

config.go

@@ -22,4 +22,7 @@ type appConfig struct {
 	// GitLab specific configuration
 	gitlabProjectVisibility     string
 	gitlabProjectMembershipType string
+
+	// Forgejo specific configuration
+	forgejoRepoType string
 }

forgejo.go

@@ -0,0 +1,85 @@
+package main
+
+import (
+	"fmt"
+	"log"
+
+	forgejo "codeberg.org/mvdkleijn/forgejo-sdk/forgejo/v2"
+)
+
+func getForgejoRepositories(
+	client interface{},
+	service string, githubRepoType string, githubNamespaceWhitelist []string,
+	gitlabProjectVisibility string, gitlabProjectMembershipType string,
+	ignoreFork bool, forgejoRepoType string,
+) ([]*Repository, error) {
+	if client == nil {
+		log.Fatalf("Couldn't acquire a client to talk to %s", service)
+	}
+
+	forgejoClient := client.(*forgejo.Client)
+
+	switch forgejoRepoType {
+	case "starred":
+		user, _, err := forgejoClient.GetMyUserInfo()
+		if err != nil {
+			log.Fatalf("Error fetching user info from %s: %v", service, err)
+		}
+
+		log.Printf("Found user %s with ID %d", user.UserName, user.ID)
+
+		repos, err := paginateForgejoRepositories(func(page int) ([]*forgejo.Repository, *forgejo.Response, error) {
+			return forgejoClient.SearchRepos(forgejo.SearchRepoOptions{
+				ListOptions:     forgejo.ListOptions{Page: page},
+				StarredByUserID: user.ID,
+			})
+		})
+		if err != nil {
+			return nil, fmt.Errorf("fetching starred repositories from %s: %v", service, err)
+		}
+
+		return repos, nil
+	case "user", "":
+		repos, err := paginateForgejoRepositories(func(page int) ([]*forgejo.Repository, *forgejo.Response, error) {
+			return forgejoClient.ListMyRepos(forgejo.ListReposOptions{
+				ListOptions: forgejo.ListOptions{Page: page},
+			})
+		})
+		if err != nil {
+			return nil, fmt.Errorf("fetching user repositories from %s: %v", service, err)
+		}
+
+		return repos, nil
+	default:
+		return nil, fmt.Errorf("unknown repo type: %s", forgejoRepoType)
+	}
+}
+
+func paginateForgejoRepositories(fetch func(page int) ([]*forgejo.Repository, *forgejo.Response, error)) ([]*Repository, error) {
+	var repositories []*Repository
+	page := 1
+
+	for {
+		results, resp, err := fetch(page)
+		if err != nil {
+			return nil, err
+		}
+
+		for _, repo := range results {
+			repositories = append(repositories, &Repository{
+				CloneURL:  repo.CloneURL,
+				Name:      repo.Name,
+				Namespace: repo.Owner.UserName,
+				Private:   repo.Private,
+			})
+		}
+
+		if resp == nil || resp.NextPage == 0 {
+			break
+		}
+
+		page = resp.NextPage
+	}
+
+	return repositories, nil
+}

git_repository_clone.go

@@ -32,6 +32,7 @@ func handleGitRepositoryClone(client interface{}, c *appConfig) error {
 		c.gitlabProjectVisibility,
 		c.gitlabProjectMembershipType,
 		c.ignoreFork,
+		c.forgejoRepoType,
 	)
 	if err != nil {
 		return err

github.go

@@ -12,7 +12,7 @@ func getGithubRepositories(
 	client interface{},
 	service string, githubRepoType string, githubNamespaceWhitelist []string,
 	gitlabProjectVisibility string, gitlabProjectMembershipType string,
-	ignoreFork bool,
+	ignoreFork bool, forgejoRepoType string,
 ) ([]*Repository, error) {
 
 	if client == nil {
@@ -44,7 +44,15 @@ func getGithubRepositories(
 				continue
 			}
 
-			cloneURL := getCloneURL(*repo.CloneURL, *repo.SSHURL)
+			var httpsCloneURL, sshCloneURL string
+			if repo.CloneURL != nil {
+				httpsCloneURL = *repo.CloneURL
+			}
+			if repo.SSHURL != nil {
+				sshCloneURL = *repo.SSHURL
+			}
+
+			cloneURL := getCloneURL(httpsCloneURL, sshCloneURL)
 			repositories = append(repositories, &Repository{
 				CloneURL:  cloneURL,
 				Name:      *repo.Name,
@@ -74,7 +82,16 @@ func getGithubStarredRepositories(ctx context.Context, client *github.Client, ig
 				continue
 			}
 			namespace := strings.Split(*star.Repository.FullName, "/")[0]
-			cloneURL := getCloneURL(*star.Repository.CloneURL, *star.Repository.SSHURL)
+
+			var httpsCloneURL, sshCloneURL string
+			if star.Repository.CloneURL != nil {
+				httpsCloneURL = *star.Repository.CloneURL
+			}
+			if star.Repository.SSHURL != nil {
+				sshCloneURL = *star.Repository.SSHURL
+			}
+
+			cloneURL := getCloneURL(httpsCloneURL, sshCloneURL)
 			repositories = append(repositories, &Repository{
 				CloneURL:  cloneURL,
 				Name:      *star.Repository.Name,