Skip to content

Conversation

@vgarcia13
Copy link
Contributor

@vgarcia13 vgarcia13 commented Aug 14, 2025

Relevant Link

https://allenai.atlassian.net/browse/GUNDI-4618


This pull request introduces a new Django management command in gundi_v1_migration_script.py to automate the migration of Gundi v1 inbound integrations to the Gundi v2 system. The script provides options for selecting specific inbounds, integration types, and actions, and handles the creation of necessary related objects (integrations, actions, organizations, routes, and configurations) during migration. It also includes custom JSON schema definitions for authentication and field mappings, and provides detailed output and error handling throughout the migration process.

Key additions and features:

Migration Command Implementation

  • Added a new management command gundi_v1_migration_script.py that migrates inbound integrations from Gundi v1 to Gundi v2, with arguments for inbound type, integration type, action, specific inbounds, and migration limits.
  • The command handles the creation or retrieval of related objects (organizations, integration types, integrations, actions, routes, configurations) as needed during the migration.
  • Supports filtering and limiting which inbounds are migrated, and provides summary output for created, skipped, and errored migrations.

Schema and Field Mapping Definitions

  • Introduced ER_DESTINATION_JSON_SCHEMA and ER_DESTINATION_UI_SCHEMA for defining authentication configuration structure for migrated integrations.
  • Added a default field

@vgarcia13 vgarcia13 changed the title AWT v1 migration script GUNDI-4618: AWT v1 migration script Aug 14, 2025
)

def handle(self, *args, **options):
if inbounds_to_migrate := self._get_awt_inbounds(options=options):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea: If we accept an integration-type param and use it for querying the inbounds, then this command could be made more generic and be used in the future to migrate inbounds of any type to v2 (it may need some other tweaks too). This could be left for a second iteration of course, but what do you think @vgarcia13 @victorlujanearthranger @chrisdoehring?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I would need to add support for defining the action to populate and the required field mappings

@marianobrc
Copy link
Contributor

Great progress @vgarcia13!
So I think the last steps now are: creating the field mappings for provider keys, and unit tests

@vgarcia13 vgarcia13 marked this pull request as ready for review August 25, 2025 16:09
@vgarcia13 vgarcia13 changed the title GUNDI-4618: AWT v1 migration script GUNDI-4618: Gundi v1 migration script MK1 Sep 24, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a Django management command to automate the migration of Gundi v1 inbound integrations to the Gundi v2 system. The script provides comprehensive migration functionality with options for filtering and limiting the inbounds to migrate.

Key changes:

  • Added a new management command with CLI arguments for inbound type, integration type, action, specific inbounds, and migration limits
  • Implemented migration logic that creates or retrieves related objects (organizations, integration types, integrations, actions, routes, configurations)
  • Defined JSON schemas for Earth Ranger authentication configuration and field mapping defaults

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

self.stdout.write(f" -- ERROR: {inbound_slug.capitalize()} Integrations with IDs {options['inbounds']} not found -- \n")
return []

if options['max']:
Copy link

Copilot AI Sep 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition if options['max']: will fail when --max is set to 0, as 0 is falsy in Python. This should be if options['max'] is not None: or check for non-negative values explicitly.

Suggested change
if options['max']:
if options['max'] is not None:

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants