Add Git source support to load TypeScript types from remote repositories #1
Description
Goal:
- Allow ts-mock-proxy to clone (or fetch) a GitHub/git repository and use TypeScript type files from a specified path/branch as additional sources for generating mocks and Swagger docs.
Scope
- In-scope: Public GitHub (and generic Git) repos, specifying repo URL, branch/tag/commit, and target path inside the repo; integrate with existing --external-dir behavior so cloned files are parsed as external sources.
- Out-of-scope (initial): Full multi-repo sync scheduling, GUI for credentials, or complex monorepo workspace resolution.
User-facing API
- CLI options:
--git-repo : repository URL to clone (can be repeated)
--git-branch : branch, tag, or commit (optional; default: main/master)
--git-path : relative path inside the repo to scan for types (optional; default: repo root)
--git-depth : shallow clone depth (optional; default: 1)
--git-token : (optional) token for private repos (env var preferred) - Config file support: add git source entries to ts-mock-proxy.config.json with same fields.
Design & Implementation Notes
- Clone strategy: use simple-git or shell git invocation to clone into a temporary folder under .tmp/git-sources/; prefer shallow clones (--depth) and single-branch fetch.
- Credential handling: recommend supporting GIT_TOKEN env var and git-username/git-token CLI options; avoid storing tokens on disk. Use URL-embedding or Authorization for API clones when needed.
- Integration: after clone, pass the resolved path to existing parser/file-watcher as an additional source (same priority/order as --external-dir).
- Hot-reload / sync: provide optional --git-poll-interval to periodically git fetch and auto-pull changes, invalidating cache when files change. Default: disabled.
- Caching & cleanup: reuse local clone directories across runs (by repo+branch hash) to speed up; provide --git-clean to remove clones on shutdown.
- Error handling: surface clear 4xx/5xx style errors for clone/fetch failures in logs; fall back to existing behavior (start with local sources only) unless --fail-on-git-error is set.
Acceptance Criteria
- Parsing: Cloned repo files at --git-path are parsed and their exported interfaces are available as endpoints (same naming rules).
- CLI: The listed CLI flags work as documented; multiple --git-repo entries are supported.
- Hot-reload: When --git-poll-interval is set, changes pushed to the remote and pulled locally reflect in the server without manual restart.
- Auth: Private repo cloning is possible using an env var token (documented).
- Tests: Unit tests for clone logic (mocked), integration test that uses a small public test repo and verifies endpoints appear.
- Docs: README updated with examples of CLI usage and config file.
Testing
- Unit: mock simple-git or child_process to test success/failure flows, shallow clone, path resolution, and cleanup.
- Integration: small public test repo with a types/ folder used in CI; test server boots, routes are generated, and hot-reload picks up a simulated change.
- Manual: example commands to verify locally:
npx tsx index.ts --git-repo https://github.com/org/test-contracts --git-path src/types --port 8080
GIT_TOKEN=xxx npx tsx index.ts --git-repo https://github.com/org/private-contracts --git-path src/dto
Security & Edge Cases
- Secrets: recommend env var only for tokens; never log tokens or persist them in repo clones.
- Malicious repos: only read TypeScript files; do not execute code from cloned repos.
- Large repos: support --git-depth and --git-path to limit scope.
- Network failures: fail gracefully and optionally continue with local sources.
Docs
- README: add a Git Support section with CLI examples, env var instructions, config file snippet, and security notes.
- EXTERNAL_TYPES.md: update to show git source example.