-
Notifications
You must be signed in to change notification settings - Fork 17
feat: enhance image generation api #133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
esafwan
wants to merge
5
commits into
develop
Choose a base branch
from
feature/enhance-image-generation-api
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add support for advanced image configuration parameters to enable slide-optimized image generation with better text readability. Changes: - Add aspect_ratio parameter (e.g., "16:9", "9:16", "1:1") - Add image_size parameter (e.g., "2K", "4K", "1K") - Parameters are optional and maintain backward compatibility - Automatically applied for Google/Gemini providers - Enhanced return value with message_id, conversation_id, and metadata Benefits: - Enables slide-optimized generation (16:9 @ 2K resolution) - Improves text readability in generated images - Provides comprehensive metadata for image tracking - Follows AISlideAppGooGAPI pattern for advanced configuration Technical Details: - Parameters passed as provider-specific kwargs to LiteLLM - Only applied when provider is Google/Gemini/Vertex AI - Return value now includes full generation context
Register new parameters and get_conversation_images tool in the Agent Tool Function system. Changes: - Add aspect_ratio and image_size parameters to generate_image tool - Create new get_conversation_images tool for image retrieval - Update after_install() and after_migrate() hooks New Tool: get_conversation_images - Retrieves images from any conversation by conversation_id - Returns image URLs, message IDs, and metadata - Solves run_agent limitation (parent can access child images) - Enables parent-child agent image workflows Tool Parameters: - conversation_id (optional): Target conversation - limit (optional): Max images to return (default: 10) Use Case: Parent agent can now retrieve images generated by child agents: 1. Parent calls run_agent(agent_name="image_gen", prompt="...") 2. Parent calls get_conversation_images(conversation_id=result["conversation_id"]) 3. Parent accesses child's generated images
Add database migration patch to update existing installations with new image generation parameters. Changes: - Create patch: add_image_tool_advanced_params.py - Register patch in patches.txt - Patch adds aspect_ratio and image_size to existing generate_image tool Patch Behavior: - Checks if generate_image tool exists - Adds aspect_ratio parameter if not present - Adds image_size parameter if not present - Safe to run multiple times (idempotent) Deployment: - Runs automatically on 'bench migrate' - Updates existing installations without breaking changes - New installations get parameters from create_image_generation_tool() Migration Path: [post_model_sync] huf.patches.add_tool_types huf.patches.v1.update_image_tool huf.patches.v1.update_agent_background_color huf.patches.v1.add_image_tool_advanced_params <- NEW
…_ratio/image_size Instead of conditionally applying aspect_ratio and image_size only for Google/Gemini providers, pass them unconditionally to LiteLLM and let it handle provider-specific forwarding. Benefits: - Follows LiteLLM's design: "Any non-openai params will be treated as provider-specific params and sent as kwargs to the provider" - More future-proof: If other providers add support, they'll work automatically - Cleaner code: No provider-specific conditionals needed - Providers that don't support these params will ignore them gracefully Technical Details: - Removed provider check before adding aspect_ratio/image_size - LiteLLM automatically forwards these to providers that support them - Updated documentation to reflect universal parameter support
…ad of patch Instead of using a separate migration patch, update create_image_generation_tool() to handle both creation and updates. This is cleaner and eliminates the need for a separate patch file. Changes: - Function now checks if tool exists and updates it if needed - Adds missing aspect_ratio and image_size parameters to existing tools - Removed add_image_tool_advanced_params.py patch (no longer needed) - Removed patch registration from patches.txt Benefits: - Simpler: One function handles both creation and updates - No separate patch needed - function runs in after_migrate() hook - Idempotent: Safe to run multiple times - Cleaner codebase: Less files to maintain The function is called in after_install() and after_migrate() hooks, so existing installations will automatically get updated during migration.
5245810 to
de7d2f4
Compare
de7d2f4 to
9617a3e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR enhances the HUF image generation API with advanced configuration parameters and a new image retrieval tool, addressing key gaps identified in comparison with slide generation workflows.
Motivation
The current image generation API lacks:
run_agentThese limitations prevent effective slide-optimized image generation and parent-child agent workflows.
Changes
1. Advanced Image Configuration Parameters ✅
Added Parameters:
aspect_ratio: Control image aspect ratio (e.g., "16:9", "9:16", "1:1", "4:3", "3:4")image_size: Control resolution quality (e.g., "1K", "2K", "4K")Benefits:
Usage Example:
2. Enhanced Return Value with Metadata ✅
New Fields:
message_id: Reference to Agent Message documentconversation_id: For querying images laterprompt,size,quality,aspect_ratio,image_sizeBenefits:
3. New Tool: get_conversation_images ✅
Purpose: Retrieve images from any conversation, solving the
run_agentlimitation.Problem Solved:
The
run_agenttool cannot return images to parent agents because images are saved as separate Agent Message documents, not included in the text response.Solution:
New tool retrieves images from any conversation by conversation_id.
Usage Example:
Technical Details
Files Modified
huf/ai/sdk_tools.pyhandle_generate_image()with new parametershandle_get_conversation_images()functionhuf/install.pycreate_get_conversation_images_tool()functionhuf/patches/v1/add_image_tool_advanced_params.py(NEW)huf/patches.txtCommit Structure
feat: add aspect_ratio and image_size parameters to image generation
feat: add tool registration for enhanced image generation
feat: add migration patch for image generation parameters
Testing
Test Cases
aspect_ratio="16:9"andimage_size="2K"get_conversation_images()retrieves images correctlyManual Testing Steps
Test Advanced Parameters:
Test Image Retrieval:
Test Migration:
bench migrate # Verify new parameters in Agent Tool FunctionDeployment
Migration Path
Rollback Plan
If issues arise:
bench migrateto apply reverse patchesBreaking Changes
None. All changes are backward compatible:
Documentation
/workspace/development/Docs/HUF_Imagen.mdRelated Issues
Closes gaps identified in image generation comparison:
Checklist
Screenshots/Examples
Before:
After:
Additional Notes
Reviewers: Please test with Google/Gemini models for full functionality.