Add --exclude option to filter files during uploads
#269
+393
−21
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| Feature: Exclude option tests | ||
|
|
||
| Scenario: Exclude files with wildcard extension pattern | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=*.map --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| Then I should receive no sourcemaps | ||
|
|
||
| Scenario: Exclude files matching specific filename pattern | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=main.js.map --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| And I wait to receive 1 sourcemap | ||
| Then the sourcemaps are valid for the API | ||
| And the sourcemap payload field "minifiedUrl" equals "example.com/other.js" | ||
|
|
||
| Scenario: Exclude files with multiple patterns | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=main.js.map --exclude=other.js.map --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| Then I should receive no sourcemaps | ||
|
|
||
| Scenario: Exclude with path pattern | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=features/js/fixtures/js-multiple-maps/dist/** --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| Then I should receive no sourcemaps | ||
|
|
||
| Scenario: Exclude with absolute path | ||
| Given I get the current working directory | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=$ABS_PATH/features/js/fixtures/js-multiple-maps/dist/other.js.map --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| And I wait to receive 1 sourcemap | ||
| Then the sourcemaps are valid for the API | ||
| And the sourcemap payload field "minifiedUrl" equals "example.com/main.js" | ||
|
|
||
| Scenario: Exclude with path glob pattern | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=**/dist/** --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| Then I should receive no sourcemaps | ||
|
|
||
| Scenario: Upload succeeds when exclude pattern doesn't match | ||
| When I run bugsnag-cli with upload js --upload-api-root-url=http://localhost:$MAZE_RUNNER_PORT --api-key=1234567890ABCDEF1234567890ABCDEF --overwrite --base-url=example.com --exclude=*.log --project-root=features/js/fixtures/js-multiple-maps features/js/fixtures/js-multiple-maps/dist/ | ||
| And I wait to receive 2 sourcemaps | ||
| Then the sourcemaps are valid for the API |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -92,7 +92,7 @@ func UploadAndroidNdk( | |
| "/ndk-symbol", | ||
| params, | ||
| fileField, | ||
| originalFile, | ||
| opts, | ||
| logger, | ||
| ) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,7 +3,6 @@ package server | |
| import ( | ||
| "bytes" | ||
| "fmt" | ||
| "io" | ||
| "mime/multipart" | ||
| "net/http" | ||
|
|
@@ -14,6 +13,7 @@ import ( | |
|
|
||
| "github.com/pkg/errors" | ||
|
|
||
| "github.com/bugsnag/bugsnag-cli/pkg/endpoints" | ||
| "github.com/bugsnag/bugsnag-cli/pkg/log" | ||
| "github.com/bugsnag/bugsnag-cli/pkg/options" | ||
| "github.com/bugsnag/bugsnag-cli/pkg/utils" | ||
|
|
@@ -121,6 +121,14 @@ func buildFileRequest(url string, fieldData map[string]string, fileFieldData map | |
| // - error: An error if any step of the file processing fails. Nil if the process is successful. | ||
| func ProcessFileRequest(apiKey string, endpointPath string, uploadOptions map[string]string, fileFieldData map[string]FileField, fileName string, options options.CLI, logger log.Logger) error { | ||
|
|
||
| // Check if the fileName itself should be excluded based on exclude patterns | ||
| if len(options.Upload.Exclude) > 0 { | ||
| if utils.IsFileExcluded(fileName, options.Upload.Exclude) { | ||
| logger.Info(fmt.Sprintf("Skipping the upload of: %s (matches exclude pattern)", fileName)) | ||
| return nil | ||
| } | ||
| } | ||
|
|
||
| if apiKey != "" { | ||
| uploadOptions["apiKey"] = apiKey | ||
| } else { | ||
|
|
@@ -249,8 +257,18 @@ func processRequest(request *http.Request, timeout int, retryCount int, logger l | |
| // Returns: | ||
| // - error: An error if any step of the request processing fails. Nil if the process is successful. | ||
| func sendRequest(request *http.Request, timeout int, logger log.Logger) error { | ||
| // Configure transport to use HTTP/1.1 only | ||
| var protocols http.Protocols | ||
| protocols.SetHTTP1(true) | ||
| protocols.SetHTTP2(false) | ||
|
|
||
| transport := &http.Transport{ | ||
| Protocols: &protocols, | ||
| } | ||
|
|
||
| client := &http.Client{ | ||
| Timeout: time.Duration(timeout) * time.Second, | ||
| Transport: transport, | ||
| } | ||
|
|
||
| response, err := client.Do(request) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,7 +48,15 @@ func All(options options.CLI, logger log.Logger) error { | |
| fileFieldData["file"] = server.LocalFile(file) | ||
| } | ||
|
|
||
| err := server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| file, | ||
| options, | ||
| logger, | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -168,12 +168,28 @@ func ProcessAndroidProguard(options options.CLI, logger log.Logger) error { | |
| fileFieldData["proguard"] = server.LocalFile(outputFile) | ||
|
|
||
| // Attempt upload to Bugsnag API | ||
| err = server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "/proguard", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| outputFile, | ||
| options, | ||
| logger, | ||
| ) | ||
|
|
||
| // Retry at base endpoint if 404 received | ||
| if err != nil && strings.Contains(err.Error(), "404 Not Found") { | ||
| logger.Debug("Retrying upload for proguard at base endpoint") | ||
| err = server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| outputFile, | ||
| options, | ||
| logger, | ||
| ) | ||
| } | ||
|
|
||
| if err != nil { | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -71,7 +71,15 @@ func ProcessBreakpad(globalOptions options.CLI, logger log.Logger) error { | |
| ) | ||
|
|
||
| // Send the file upload request to the Breakpad symbol endpoint | ||
| err = server.ProcessFileRequest( | ||
| apiKey, | ||
| "/breakpad-symbol"+queryParams, | ||
| formFields, | ||
| fileFieldData, | ||
| file, | ||
| globalOptions, | ||
| logger, | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,7 +48,15 @@ func Dart(options options.CLI, logger log.Logger) error { | |
| fileFieldData := make(map[string]server.FileField) | ||
| fileFieldData["symbolFile"] = server.LocalFile(file) | ||
|
|
||
| err := server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "/dart-symbol", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| file, | ||
| options, | ||
| logger, | ||
| ) | ||
|
|
||
| if err != nil { | ||
|
|
||
|
|
@@ -91,7 +99,15 @@ func Dart(options options.CLI, logger log.Logger) error { | |
| if options.DryRun { | ||
| err = nil | ||
| } else { | ||
| err = server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "/dart-symbol", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| file, | ||
| options, | ||
| logger, | ||
| ) | ||
| } | ||
|
|
||
| if err != nil { | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -365,7 +365,15 @@ func uploadSingleSourceMap(sourceMapPath string, bundlePath string, bundleUrl st | |
| fileFieldData["sourceMap"] = sourceMapFile | ||
| fileFieldData["minifiedFile"] = server.LocalFile(bundlePath) | ||
|
|
||
| err = server.ProcessFileRequest( | ||
| options.ApiKey, | ||
| "/sourcemap", | ||
| uploadOptions, | ||
| fileFieldData, | ||
| sourceMapPath, | ||
| options, | ||
| logger, | ||
| ) | ||
|
|
||
| if err != nil { | ||
| return fmt.Errorf("encountered error when uploading js sourcemap: %s", err.Error()) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -49,7 +49,7 @@ func uploadSymbolFile(symbolFile string, linuxOpts options.LinuxOptions, opts op | |
| "/linux", | ||
| uploadOpts, | ||
| fileField, | ||
| symbolFile, | ||
| opts, | ||
| logger, | ||
| ); err != nil { | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,8 @@ import ( | |
| "path/filepath" | ||
| "strings" | ||
| "time" | ||
|
|
||
| "github.com/bmatcuk/doublestar/v4" | ||
| ) | ||
|
|
||
| const ( | ||
|
|
@@ -51,6 +53,41 @@ func IsDir(path string) bool { | |
| return err == nil && pathInfo.IsDir() | ||
| } | ||
|
|
||
| // IsFileExcluded checks if a file path matches any of the exclude patterns. | ||
| // It supports wildcards including *.map, path/to/*, and recursive patterns like node_modules/**. | ||
| // | ||
| // Parameters: | ||
| // - filePath: The file path to check. | ||
| // - excludePatterns: A list of patterns to match against (supports ** for recursive matching). | ||
| // | ||
| // Returns: | ||
| // - bool: True if the file matches any exclude pattern, false otherwise. | ||
| func IsFileExcluded(filePath string, excludePatterns []string) bool { | ||
| for _, pattern := range excludePatterns { | ||
| // Try matching the pattern against the full path using doublestar (supports **) | ||
| matched, err := doublestar.Match(pattern, filePath) | ||
| if err == nil && matched { | ||
| return true | ||
| } | ||
|
|
||
| // If pattern doesn't start with **, also try matching with **/ prepended | ||
| // This allows patterns like "node_modules/**" to match anywhere in the path | ||
| if !strings.HasPrefix(pattern, "**/") { | ||
| matched, err = doublestar.Match("**/"+pattern, filePath) | ||
| if err == nil && matched { | ||
| return true | ||
| } | ||
| } | ||
|
|
||
| // Try matching against the base name | ||
| matched, err = doublestar.Match(pattern, filepath.Base(filePath)) | ||
| if err == nil && matched { | ||
| return true | ||
| } | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| // BuildFileList compiles a list of files from the provided paths. | ||
| // | ||
| // Parameters: | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this change not have a functional effect? Or does it fix an existing limitation for nested files?