Tokens update dev #673
Open
Tokens update dev #673
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat: implement Redis to MongoDB data migration with daily cron job (#557)
Co-authored-by: Prayanshrajput <prayansh75@gmail.com>
* refactor: rename createdAt field to created_at in thread schema for consistency
* bug-remove default case for thread model
* Revert "feat: implement Redis to MongoDB data migration with daily cron job (…" (#563)
This reverts commit 40068ef30198d9c25de83e2e2cfcaf58ca581efb.
* proxy migrate done (#537)
Co-authored-by: Harsh Sahu <harksahu@Harshs-Mac-mini.local>
* fix: handle null responses in user data pagination
- Added null checks and array validation when processing paginated user data responses
- Fixed getUsers to return complete response object instead of just data property
- Added fallback to stop pagination if response format is invalid
- Protected against potential undefined access in user data aggregation
* local login validation and remove condition for local now works on all server
* feat: implement Redis to MongoDB data migration with cron job for usage metrics
* feat: implement soft delete and restore for bridges
- Added soft delete functionality for bridges with 30-day retention period
- Created new restore endpoint to recover soft-deleted bridges and their versions
- Added deletedAt field and TTL indexes to configuration and bridge_version models
- Added versions array field to track associated versions for bridges
- Removed duplicate deleteBridges route definition
- Enhanced delete API response with detailed status messages and version counts
- Adde
* pauthkey to cauthkey migrate
* refactor: simplify bridge deletion response to only return success and status message
* fix: correct bridge deletion query with proper ObjectId casting
- Updated bridge lookup query to properly cast bridge_id to ObjectId type
- Ensures MongoDB can correctly match the _id field when deleting bridges
- Prevents potential query failures when bridge_id is passed as a string
* feat: add migration to clean up unused MongoDB bridges and versions
- Created new migration to safely remove unused configuration bridges and versions from MongoDB
- Added logic to identify bridges not referenced in PostgreSQL conversations table
- Implemented version cleanup that only removes versions if they are unused and their parent bridge is also unused
- Added transaction support and detailed logging for migration progress and results
- Included documentation about backup requirements in
* add update and add the api and embed
* proxy fix
* poc for us
* feat: add Grok API integration support
- Implemented callGrokApi function to validate Grok API keys using X.AI's chat completions endpoint
- Added 'grok' as a valid service option in API key validation schemas
- Integrated Grok validation into the API key save workflow
* refactor: enhance migration for cleaning up unused MongoDB bridges and versions
- Updated MongoDB connection to use environment variable for connection URI
- Improved document retrieval to include additional fields for better decision-making
- Enhanced logic for identifying unused bridges and versions, ensuring only those with status 0 are considered for deletion
- Added detailed logging for skipped bridges and versions to aid in debugging
* fix localtoken api
* expire logic when switch org
* fix proxy embed user issue
* feat: add agent lookup endpoint for model-based queries
- Created /data/by-model endpoint to retrieve agents and versions by model name
- Implemented data enrichment by combining configuration, version, and MSG91 org details
- Added deduplication logic to ensure distinct results per organization
* refactor: update migration to utilize Mongoose for MongoDB operations
- Replaced direct MongoDB client usage with Mongoose for improved connection management
- Enhanced document retrieval and processing logic to include unarchiving of bridges and versions with history
- Added detailed logging for unarchiving actions and migration summary
- Ensured proper transaction handling and connection closure at the end of the migration
* refactor: optimize migration to filter bridges and versions by status
- Updated migration logic to only retrieve bridges with status 1, improving efficiency
- Enhanced version retrieval to include only those associated with active parent bridges
- Improved overall data handling and logging for better clarity during migration process
* refactor: streamline migration logic for unarchiving bridges
- Removed unnecessary unarchiving of versions, focusing on unarchiving parent bridges instead
- Updated logging to clarify the unarchiving process based on bridge and version history
- Enhanced overall clarity and efficiency of the migration process
* refactor: enhance migration to clean up unused apikeys and apicalls
- Added logic to update ApikeyCredentials by removing deleted version IDs and deleting apikeys with empty version_ids
- Implemented cleanup for apicalls by removing references to deleted bridge and version IDs, deleting apicalls with empty arrays
- Improved logging to summarize updates and deletions of apikeys and apicalls during migration process
* refactor: enhance migration logic for deleting unused bridges and versions
- Updated migration to include connected_agents data when retrieving bridges and versions
- Implemented a recursive function to find all connected agents, improving dependency tracking
- Adjusted deletion criteria to ensure only non-protected bridges and versions are removed
- Enhanced logging to provide clearer insights into the migration process and decisions made
* fix: correct user data caching and API response parsing
- Fixed JSON parsing of cached user data with proper error handling
- Corrected API response structure to access nested data.data and data.totalEntityCount
- Simplified control flow by removing IIFE pattern for better readability
* limit cron and api and folder cost optimization
* bug fixed corn db update
* lastused updated by cron and get all apikey with lastuse
* add validation when user save and update apiKey
* refactor: remove apikey deletion logic from migration and fix syntax error in test
- Removed deletion of apikeys with empty version_ids, keeping only the update logic
- Eliminated deletedApikeys counter and related logging
- Updated console logs to reflect simplified apikey update process
- Fixed syntax error in test.js (removed stray 'l' character in conditional check)
* refactor: separate active and archived bridges for improved deletion logic
- Split bridge retrieval into activeBridges (status 1) and allBridges (all statuses) for more accurate processing
- Use activeBridges for building connected agents dependency map to prevent false dependencies from archived bridges
- Use allBridges for deletion logic to properly identify and remove archived bridges with status 0
- Updated logging to show both total and active bridge counts for better visibility during migration
* refactor: enhance migration with comprehensive debugging and improved version deletion logic
- Added extensive debugging logs to track conversations table data, query results, and processing steps
- Separated active bridge versions from all versions for more accurate dependency mapping
- Fixed version deletion logic to handle both deleted parent bridges and archived unprotected parent bridges
- Corrected unarchiving logic to properly set status to 1 (active) instead of 0
- Added detailed logging for bridge
* Create new table for orchrestator andconversation (#583)
* feat: add migration for conversation_logs table with indexing
- Created a new migration to establish the conversation_logs table.
- Included various fields to capture conversation details, user interactions, and metadata.
- Added indexes for commonly queried fields to enhance performance.
* feat: add bridge_id field to orchestrator_andconversation migration
- Introduced a new field 'bridge_id' to the orchestrator_andconversation table to enhance data relationships.
- The field is of type STRING and allows null values.
* feat: add conversation logs retrieval API with pagination support
- Created conversation_logs model with comprehensive fields for tracking chatbot interactions, user feedback, and metadata
- Implemented GET endpoint /history/:bridge_id/:thread_id/:sub_thread_id for retrieving conversation logs with pagination
- Added validation for required parameters (bridge_id, thread_id, sub_thread_id) and pagination limits (1-100)
- Included database service layer with filtering by org_id, bridge_id, thread_id, and sub_thread_
* feat: add recent threads retrieval API with pagination support
- Implemented GET endpoint /threads/:bridge_id for retrieving recent threads grouped by thread_id
- Added getRecentThreads database service to fetch distinct threads ordered by most recent updated_at
- Included validation for required bridge_id parameter and pagination parameters (page must be positive integer)
- Used Sequelize aggregation to get MAX(id) and MAX(updated_at) for each thread_id group
- Added controller with error handling an
* feat: add conversation logs search API with flexible filtering
- Implemented POST endpoint /search/:bridge_id for searching conversation logs with multiple filter options
- Added searchConversationLogs database service to query logs by message_id, keywords, thread_id, sub_thread_id, and time_range
- Included validation for required bridge_id parameter and at least one search filter
- Used Sequelize operators for keyword search across llm_message, user, chatbot_message, and updated_chatbot_message fields
- Added time
* feat: update conversation_logs model and controller with enhanced validation and new fields
- Renamed 'updated_chatbot_message' to 'updated_llm_message' for clarity.
- Added 'child_id' field to the conversation_logs model.
- Removed unnecessary allowNull constraints for several fields to streamline the model.
- Introduced Joi validation schemas for URL parameters and request bodies in conversation logs controller.
- Improved error handling for validation errors in the controller methods.
- Updated search functionality to use a single 'keyword' filter instead of multiple optional filters.
* refactor: update conversation logs routes and remove default display name
- Changed authentication middleware from 'middleware' to 'combinedAuthWithChatBotAndPublicChatbot' for GET /:bridge_id/:thread_id/:sub_thread_id endpoint to support public chatbot access
- Removed default "Creating a brief summary." display_name from sub_thread grouping in searchConversationLogs
- Cleaned up inline comment in configController routes
* feat: add prompt field to conversation_logs model and migration
- Introduced a new 'prompt' field to both the conversation_logs model and the corresponding migration.
- The 'prompt' field is of type TEXT and allows null values, enhancing the model's capability to store additional conversation context.
---------
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* feat: add filtering support for conversation logs by user feedback and error status
- Added user_feedback and error query parameters to getRecentThreadsController with default values ('all' and 'false')
- Updated getRecentThreads database service to filter by user_feedback and error fields when provided
- Added validation schemas for user_feedback and error parameters in both getConversationLogsParamsSchema and paginationQuerySchema
- Changed authentication middleware from combinedAuthWithChatBotAn
* refactor: remove unused query parameter from getRecentThreadsController
- Removed unused 'query' variable declaration and parameter passing to getRecentThreads service
- Cleaned up redundant code as individual query parameters (user_feedback, error) are already being extracted and passed separately
* refactor: remove unused query parameter from getRecentThreads database service
- Removed unused 'query' parameter from getRecentThreads function signature
- Parameter was not being used in the function implementation
* feat: reverse conversation logs before returning in getConversationLogs service
- Updated the getConversationLogs function to reverse the order of conversation logs before returning them to ensure the most recent logs are displayed first.
* add removed limit funtionalities (#589)
* feat: add formatted thread history service with user/assistant message separation (#590)
- Created getThreadHistoryFormatted function to retrieve conversation logs with messages split into separate user and assistant entries
- Added pagination support with default values (page: 1, limit: 30) for formatted thread history
- Implemented message formatting logic to create distinct entries for user messages and assistant responses (llm_message/updated_llm_message/chatbot_message)
- Added comprehensive metadata
Co-authored-by: Natwar589 <natwarrathor961@gmail.com>
* feat: add migration script to transfer data from conversations and raw_data to conversation_logs (#587)
- Implemented a new migration to consolidate data from the conversations and raw_data tables into the conversation_logs table.
- The migration processes unique message_ids in batches for efficiency, handling related conversation entries and raw_data.
- Includes error handling and logging for successful migration and rollback procedures.
* refactor: clean up error field handling in getThreadHistoryFormatted service (#591)
- Removed unused fallback_model and firstAttemptError fields from user message entries
- Moved error field to user messages only, removed from assistant messages
- Simplified error handling logic by removing redundant firstAttemptError field
* migration fix
* image_urls -> user urls , urls -> llm_urls
* fix: correct image_urls and urls field mappings in getThreadHistoryFormatted (#592)
- Set image_urls to empty array for user messages instead of null
- Map urls to user_urls for user messages
- Map image_urls to llm_urls for assistant messages
* fix: prevent deletion of bridges with active connections (#577)
- Added aggregation pipeline to check for connected bridges in both versions and configurations
- Implemented validation to return descriptive error messages listing all connected bridge names
- Removed console.log statement for cleaner error handling
Co-authored-by: Yogesh Patel <ygurjar932@gmail.com>
* feat: add migration to set default bridge_status and normalize apicall bridge_ids (#593)
* fix: change default apikey_usage to -1 and remove unused cache cleanup in deleteApikey (#594)
* orcastral model history change
* proxy api change
* remove unused test HTML file
* refactor: Remove Flow, POC, Orchestrator, Template, and Showcase features while refining document loaders and Pinecone service.
* reverts templates routes
* revert: exclude exp and iat from token payload in userOrgLocal controllers (#598)
* feat: Introduce bridge versioning, agent management, prebuilt prompts, and various new utility services and routes, while refactoring existing configurations and controllers.
* feat: Introduce AI-powered prompt optimization, summary, function argument generation, and test case generation, and enable actual agent cloning.
* apiCallDbservice import to apicallDbService.js
* apiCallDbservice import to apicallDbService.js
* Refactor: Consolidate and reorganize controllers, routes, and validation schemas; remove deprecated files and introduce new services for improved functionality.
* refactor: Remove orgId parameter from GET /chatbots route path.
* feat: Update history endpoints to use `agent_id` instead of `bridge_id` and add `user_feedback` and `error` query parameters to conversation logs. (#603)
* fix: Update AI middleware authentication key from AI_MIDDLEWARE_PAUTH_KEY to GTWY_PAUTH_KEY
* fix: Update route parameter from `bridgeId` to `version_id` in config.routes.js
* feat: Integrate external API for generating authentication tokens in userOrgLocal controller
* ```
feat: Rename bridge-related controllers to agent-related and implement role-based access control with agent-specific permissions
- Rename controllers: createBridgesController -> createAgentController, updateBridgeController -> updateAgentController, getBridgeController -> getAgentController, getAllBridgesController -> getAllAgentController, deleteBridges -> deleteAgentController, getBridgesAndVersionsByModelController -> getAgentsAndVersionsByModelController
- Update route parameter from bri
* bug add the missed key in bridge and version schema
* refactor: Update Joi validation schemas to allow unknown properties across multiple files
* feat: Add token generation for various services in agentConfig.controller.js and implement generate_token method in helper.utils.js
* fix: Change API request method from POST to GET in userOrgLocal controller for token generation
* fix: Update API request method from POST to GET in userOrgLocal controller for token generation
* feat: Add user details to agent list and restrict member role permissions for agent updates
- Fetch and populate user details in getAllBridgesInOrg using conversationService.getUserUpdates
- Extend getUserUpdates to support filtering by user IDs when version_id is not provided
- Restrict member role permissions to only get_agent and create_agent, removing delete_agent and update_agent
- Add requireOwnerRole middleware to version update route to enforce owner-only access
* chore: Update AI service configurations and enhance error handling in getDefaultValuesController
* feat: Implement authorization checks for agent updates based on bridge user permissions
- Add a new function to fetch users associated with a bridge in the configuration service.
- Enhance the updateAgentController to verify if the current user is authorized to update the agent based on the users array from the bridge configuration.
- Deny access with a 403 status if the user is not authorized.
* Revert "Role and access management"
* fix: Replace environment variable with hardcoded secret key in JWT verification
* fix: Use environment variable for JWT secret key in middleware
* fix: Update bridge ID parameter in updateBridgeController
- Change the parameter from version_id to bridgeId in the updateBridgeController route and corresponding service call to ensure correct bridge data retrieval.
* fix: Remove orgId parameter and validation from getAllChatBots route
- Change getAllChatBots to retrieve org_id from req.profile.org.id instead of req.params
- Remove userOrgAccessCheck middleware and validation from the GET / route as org_id is now obtained from the authenticated user's profile
* feat: Add chatbot configuration update and user login functionality
- Implemented updateChatBotConfig to allow updating chatbot configurations.
- Added loginUser function to handle user login and token generation for chatbots.
- Introduced validation for updateChatBotConfig route to ensure proper request structure.
- Updated routes to include new endpoints for updating chatbot config and user login.
* chore: Add whitespace and deprecated public API route for chatbot history
- Add trailing whitespace in index.js metrics section
- Import combinedAuthWithChatBotAndPublicChatbot middleware in conversation routes
- Add deprecated public API endpoint for retrieving chatbot conversation history by thread_id and bridge_slugName
* fix: Update cache key handling for bridge data with tools
* fix: Add UI bridge data cache invalidation in updateBridge
* refactor: Rename bridge-related controller functions and update routes
- Renamed controller functions from `createBridgesController`, `updateBridgeController`, `getBridgeController`, and `getAllBridgesController` to `createAgentController`, `updateAgentController`, `getAgentController`, and `getAllAgentController` respectively for clarity.
- Updated corresponding routes to reflect the new function names.
- Removed deprecated chatbot configuration update and login user functionalities from the chatbot controller.
- Enhanced user access control by implementing role-based middleware for agent routes.
* refactor: Update role management and access control for agent operations
- Changed role names from 'owner' and 'member' to 'admin' and 'editor' respectively in the role permissions system.
- Updated middleware functions to enforce 'admin' role for write operations, replacing previous 'owner' checks.
- Adjusted routes to utilize the new 'requireAdminRole' middleware for agent-related operations, ensuring proper access control.
* refactor: Rename bridge references to agent in agent configuration controller
- Updated variable names and function parameters from 'bridge' to 'agent' for clarity and consistency.
- Adjusted logic in create, update, and delete operations to reflect the change from bridge to agent.
- Enhanced error messages and success responses to accurately represent agent operations.
* refactor: Update references from bridge to agent across multiple services
- Renamed functions and variables from 'bridge' to 'agent' in various controllers and services for consistency.
- Adjusted logic in data retrieval and manipulation methods to reflect the change from bridge to agent.
- Enhanced error handling and response messages to accurately represent agent operations.
- Updated cache handling and database queries to align with the new agent terminology.
* refactor: Update agent-related references in middleware, routes, and validation
- Changed parameter names from 'bridgeId' and 'bridge_id' to 'agent_id' across middleware, routes, and validation schemas for consistency.
- Adjusted logic in middleware to ensure proper handling of agent-specific permissions.
- Updated validation schemas to require 'agent_id' instead of 'bridgeId', enhancing clarity in API requests.
* feat: Integrate external API for auth token generation in embed login
- Added axios to call an external API for generating an authentication token during the embed login process.
- Updated the embedLogin function to utilize the new token instead of the previous token generation method.
- Refactored the response structure to include the new token and maintain existing embed details.
* refactor: Restore chatbot configuration and login user functionality
- Re-added `updateChatBotConfig` and `loginUser` controller functions to chatBot.controller.js for managing chatbot configurations and user authentication.
- Implemented `getChatBotConfig` and `updateChatbotConfig` methods in chatBot.service.js to handle database operations for chatbot configurations.
- Updated `getAllChatBots` to retrieve org_id from req.profile instead of req.params for consistency.
- Modified cache invalidation strategy
* refactor: Rename bridge version references to agent version across the application
- Updated import statements and route handlers to replace 'bridgeVersion' with 'agentVersion'.
- Refactored controller methods to utilize the new agent version service for creating, updating, and managing agent versions.
- Introduced a new agentVersion.routes.js file to handle agent version-specific routes.
- Enhanced error handling and response messages to reflect the changes in terminology from bridge to agent.
* fix: Correct response object key from 'bridge' to 'agent' in getVersion function
- Updated the response object in the getVersion function to use 'agent' instead of 'bridge' for consistency with recent terminology changes.
* refactor: Implement token reissuance in embed and user organization controllers
- Added reissueToken function to handle JWT re-signing with a new secret key.
- Updated embedLogin and userOrgLocalToken functions to utilize reissueToken for generating tokens.
- Modified middleware to verify tokens using the new SecretKey instead of TEMP_SECRET.
- Introduced token.utils.js for token management utilities.
* refactor: Add IsstarterQuestionEnable field to BridgeVersion schema
* refactor: Rename callAi function to callGtwy and update corresponding route
- Renamed the `callAi` function to `callGtwy` in the utils controller for clarity.
- Updated the route from `/call-ai` to `/call-gtwy` to reflect the new function name.
* refactor: Update default Groq model from llama3-70b-8192 to openai/gpt-oss-120b
- Changed the default Groq model in new_agent_service configuration to use openai/gpt-oss-120b instead of llama3-70b-8192.
* refactor: Transform service configuration structure in getAllServiceModelsController
- Updated the configuration format in the getAllServiceModelsController to include a new structure for transformed configurations.
- Moved additional configuration fields into an `additional_parameters` object for better organization and clarity.
* refactor: Update route for adding pre-tool to use a more descriptive path
- Changed the route for adding a pre-tool from `/:agent_id` to `pre_tool/:agent_id` for improved clarity and organization.
* chore: Add whitespace and deprecated public API route for chatbot history
- Add trailing whitespace in index.js metrics section
- Import combinedAuthWithChatBotAndPublicChatbot middleware in conversation routes
- Add deprecated public API endpoint for retrieving chatbot conversation history by thread_id and bridge_slugName
* feat: Implement token generation functionality in utils controller
- Added `generateToken` function to handle different types of token generation requests.
- Introduced `createOrgToken` function in the chatBot controller for organization token creation.
- Updated routes to include the new token generation endpoint.
- Removed deprecated token retrieval routes from embed and rag controllers for cleaner API structure.
* refactor: Update token generation validation and route handling
- Removed the requirement for the `type` parameter in the `generateToken` function to streamline error handling.
- Updated the error message for invalid token types to reflect the correct valid types: rag, org, embed.
- Added validation for the `generateToken` route to ensure the `type` parameter is provided and valid.
* feat: Integrate GPT tokenizer and enhance prompt token management
- Added `gpt-tokenizer` dependency for token calculation.
- Implemented `calculateTokens` function to determine token count for prompts and tools.
- Created `calculateAndSavePromptTokens` function to update total token count in the database.
- Introduced `getPromptEnhancerPercentage` function to fetch and store prompt enhancement results.
- Updated the `publish` function to include token calculations and prompt enhancement percentage retrieval.
* feat: Broaden JSON schema and example JSON validation, adapt `generate_json` message for objects, simplify RAG auth, and update test case parsing to use 'ai' matching.
* feat: Add bridge metrics retrieval functionality
- Implemented `getBridgeMetrics` function to fetch total token counts for bridges within a specified date range or the last 24 hours.
- Updated metrics routes to include a new endpoint for retrieving bridge metrics.
* refactor: Update agent controller fields, fix pre_tool route, and adjust RAG validation
- Added 'status' field to simple_fields array in updateAgentController for proper status updates.
- Fixed pre_tool route path by adding leading slash for consistency.
- Changed response key from 'bridge' to 'agent' in addPreTool controller.
- Made docType and fileFormat optional in RAG validation schema.
- Removed unnecessary whitespace in updateAgentController.
* refactor: Update default OpenAI model from gpt-5-mini to gpt-5-nano in new agent service configuration
* refactor: Simplify authorization checks in agent configuration
- Removed inline authorization logic from updateAgentController.
- Integrated requireAdminRole middleware to handle user access based on roles.
- Updated middleware to differentiate access permissions for 'admin', 'editor', and 'viewer' roles, enhancing security and maintainability.
* feat: Add endpoint to retrieve current organization users
- Implemented `getCurrentOrgUsers` function in the utils controller to fetch and return user details (name, email, user_id) for the current organization.
- Updated routes to include a new endpoint `/users-details` for accessing organization user information.
* refactor: Change org_id validation type from objectId to string
- Updated the validation schema for `org_id` in multiple endpoints to use Joi.string() instead of Joi.objectId(), ensuring consistency in data type handling across the application.
* feat: Add endpoint to create or remove actions in agents
- Implemented `createOrRemoveAction` function in chatBot controller to handle adding and removing actions from agents.
- Added new route `/agent/:agentId/action` with validation for action management.
- Integrated `configurationService` methods for adding and removing actions in agents.
- Created validation schema for action operations supporting 'add' and 'remove' types.
- Removed user details fetching logic from `getAllAgentsInOrg` function
* refactor: Update callAiMiddleware parameters and enhance error logging
- Modified the `callAiMiddleware` function to accept parameters in a more streamlined manner.
- Improved error handling by adding a console error log for better debugging.
- Updated validation schema for `org_id` across multiple files to ensure it is consistently treated as a string.
* feat: Enhance agent configuration with timestamps and update model service
- Updated agent configuration to include `createdAt` and `updatedAt` timestamps for better tracking of agent records.
- Changed the default AI service from `ai_ml` to `openai` and updated the model to `gpt-5-nano`.
- Modified the `updateAgentController` to ensure parent agent's `updatedAt` timestamp is also updated when a version is modified.
- Adjusted database queries to include new timestamp fields in agent data retrieval.
* no need to store bridgeType and summary in vesion
* feat: Add createConversationLog function to manage conversation log entries
- Implemented `createConversationLog` function to create new conversation log entries with a structured payload.
- Updated `configServices.js` to utilize the new `createConversationLog` function for creating thread history.
- Refactored the `updateMessageStatus` function to use the updated `updateStatus` method from the history service.
* fixed_tool_id to function_id
* fix: Update user ID handling in getAllAgentController
- Modified the logic for generating `viasocket_embed_user_id` to include `folder_id` when both `user_id` and `folder_id` are present, enhancing the uniqueness of the token.
- Cleaned up unnecessary whitespace in the code for better readability.
* feat: Add user_id field to user_bridge_config_history model
- Introduced a new `user_id` field to the `user_bridge_config_history` model to ensure user association is captured.
- Set `allowNull` to false for `user_id` to enforce mandatory user linkage in history records.
* feat: Enhance agent model data construction with configuration support
- Updated the `createAgentController` to construct model data based on available model configurations, allowing for dynamic updates of various model parameters.
- Introduced a new import for `modelConfigDocument` to facilitate the retrieval of model-specific configurations.
- Refactored the model data construction logic to utilize configurations for keys such as `creativity_level`, `max_tokens`, and others, improving flexibility and maintainability.
* feat: Add testcase update endpoint and enhance agent listing with last publisher info
- Implemented `updateTestcases` controller to handle testcase updates by ID
- Added `testcaseUpdateSchema` validation for testcase update requests
- Changed `expected` field validation from string to object in testcase schemas
- Enhanced `getAllAgentsInOrg` to include `last_publisher_id` by querying PostgreSQL history
- Added `getAllAgentsWithLastPublishers` function to fetch last publishers for all agents in a
* and .
- Added new validation schemas in for better structure and reusability.
* feat: Enhance bridge metrics query to include total cost and last used time
- Updated the SQL query in `getBridgeMetrics` to aggregate total cost and capture the last used time for each bridge, improving the metrics data returned for better insights.
* fix: Update user ID generation and prevent redundant version updates (#624)
- Changed `viasocket_embed_user_id` initialization to use `org_id` instead of `user_id` as the base identifier for better organization-level tracking
- Added condition to skip updating previous published version when it's the same as the current version being published, preventing unnecessary database operations
* feat: Add new fields to ApiCall model for enhanced metadata storage
- Added `fields` field with Mixed type to store flexible field configurations
- Added `required_params` array to track required parameters
- Added `description` and `title` fields with string type and empty string defaults
- All new fields include appropriate default values for backward compatibility
* embed login logic revert
* embed login logic revert
* feat: Update role determination logic to include embed user handling
- Modified `determineRoleFromPermissions` function to accept an `isEmbed` parameter, allowing embed users to be assigned the 'editor' role directly.
- Updated middleware to check for embed user status and pass it to the role determination function, ensuring accurate role assignment based on user permissions.
* refactor: Simplify embed user check in middleware
- Updated comments for clarity regarding embed user logic.
- Streamlined the embed user determination logic in the middleware to enhance readability and maintainability.
* feat: Add new configuration fields for prompt settings
- Introduced `prompt_total_tokens` and `prompt_enhancer_percentage` fields in the configuration schema to support enhanced prompt management and tracking.
* fix: Update bridge_id to agent_id in message status update
- Changed the parameter name from `bridge_id` to `agent_id` in the `updateMessageStatus` function to align with the updated request body structure.
- Updated validation schema to reflect the change from `bridge_id` to `agent_id`, ensuring consistency across the service and validation layers.
* fix: Update agent_id validation in message status update
- Modified the validation schema for `agent_id` in the `updateMessageStatus` function to remove the required constraint, allowing for more flexible input handling.
* refactor: Simplify chatbot controller and service error handling
- Removed try-catch blocks and success/error object wrappers from chatbot service methods, allowing errors to propagate naturally
- Simplified `getAllChatBots` by removing nested result object and cleaning up variable assignments
- Added `getOneChatBot` controller and route for fetching individual chatbot by ID
- Added `addorRemoveBridgeInChatBot` controller and service methods (`addBridge`, `removeBridge`, `findById`) for managing bridge associations
- Remove
* refactor: Streamline model feature handling in suggestModel function
- Updated the logic in the `suggestModel` function to simplify the retrieval of model features, ensuring that both available and unavailable models are populated with the correct specifications from the configuration document. This enhances code readability and maintainability.
* ```
refactor: Restructure action validation schema and improve error messages
- Split `createOrRemoveAction` validation into separate query and body schemas
- Moved action type validation from body to query parameters ('add'/'remove')
- Changed body type field to validate action kinds ('sendDataToFrontend'/'reply')
- Made `actionId` optional in body schema
- Added explicit error messages for all validation fields
- Removed redundant `createOrRemoveActionValidationSchema` and its exports
```
* refactor: Update userOrgLocalToken to return original JWT instead of reissued token
- Modified the `userOrgLocalToken` function to return the original JWT token directly instead of reissuing it, simplifying the response structure.
* refactor: Update role_id assignment in proxy.utils.js to use environment variable
- Changed the `role_id` assignment in the `createOrGetUser` function to utilize the `PROXY_USER_ROLE_ID` environment variable instead of a hardcoded value, enhancing configurability and maintainability.
* refactor: Update userOrgLocal functions to return token instead of jwtToken
- Modified the `userOrgLocalToken` and `switchUserOrgLocal` functions to return a `token` variable instead of the original `jwtToken`, improving consistency in response structure.
* refactor: Rename jwtToken to token in userOrgLocal functions for consistency
- Updated the `userOrgLocalToken` and `switchUserOrgLocal` functions to use the variable name `token` instead of `jwtToken`, enhancing clarity and uniformity in the codebase.
* refactor: Merge search functionality into getRecentThreads endpoint and add version_id filtering
- Removed separate `searchConversationLogs` controller, route, and validation schema
- Enhanced `getRecentThreads` to handle keyword search and time range filters via query parameters
- Added `version_id` optional filtering across `findConversationLogsByIds`, `findRecentThreadsByBridgeId`, and `findThreadHistoryFormatted`
- Implemented keyword search across multiple fields (message_id, thread_id, sub_thread_id, ll
* refactor: Update getAllConnectedAgents to use version_id parameter and improve error handling
- Changed parameter name from 'id' to 'version_id' in getConnectedAgents controller for clarity
- Added try-catch block in fetchDocument function to handle potential database query errors
- Added 'document_type' field to agent map entries to track whether agent is a bridge or version
- Improved code formatting with explicit return statements and consistent spacing
- Changed loop variable from 'key' to unuse
* refactor: Enhance route security by requiring admin role or agent access for critical operations
- Updated agentVersion.routes.js to require admin role for creating, publishing, deleting, and discarding versions.
- Modified apiCall.routes.js to enforce agent access for updating, deleting, and creating API calls.
- Adjusted apikey.routes.js to require agent access for managing API keys.
- Enhanced rag.routes.js to enforce agent access for document deletion and updates.
* feat: Add folder_id to request object for embedded profiles
- Introduced folder_id assignment in middleware for requests with embedded profiles, enhancing data accessibility for downstream processing.
* refactor: Simplify agent access validation in middleware
- Removed the check for required agent_id in the checkAgentAccessMiddleware, streamlining the middleware logic while maintaining functionality.
* feat: Add viewer access restriction in agent access middleware
- Implemented a check in the checkAgentAccessMiddleware to return a 403 status with an access denial message for users with 'viewer' role, enhancing security and access control.
* refactor: Remove checkAgentAccessMiddleware from agent retrieval route
- Simplified the agent retrieval route by removing the checkAgentAccessMiddleware, streamlining access validation while maintaining necessary middleware for other routes.
* refactor: Add read_only flag to Docstar token based on viewer role
- Updated `getAllAgentController` to include `read_only` parameter in Docstar token generation, setting it to `true` when `role_name` is 'viewer'
* refactor: Integrate checkAgentAccessMiddleware into embed routes
- Added checkAgentAccessMiddleware to the createEmbed and updateEmbed routes to enforce access control based on user roles, enhancing security for embed operations.
* fix: Handle missing folder_id in agent retrieval query
- Updated the getAllAgentsInOrg function to set query.folder_id to null when folder_id is not provided, ensuring that agents without a folder_id are correctly retrieved.
* feat: Dynamic table selection for bridge metrics query
- Updated the getBridgeMetrics function to dynamically select between 'daily_data' and 'fifteen_minute_data' tables based on the provided date range, improving data retrieval accuracy.
* feat: Add pre_tools array to configuration schema (#631)
- Introduced a new field `pre_tools` of type Array in the configuration schema, with a default value of an empty array, to enhance configuration flexibility.
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* Migration for api calls (#633)
* fix: Rename endpoint_name to title in saveApi function
- Updated the saveApi function to replace the endpoint_name property with title, improving clarity in the API data structure.
* feat: Implement migration to standardize API call schema to v2 format
- Added migration scripts to convert existing API call documents to a unified v2 format, consolidating `endpoint_name` into `title` and removing deprecated fields.
- Updated model schema to reflect new structure, including `bridge_ids`, `old_fields`, `status`, and `version`.
- Enhanced documentation to outline migration behavior, examples, and rollback procedures.
- Simplified service logic by eliminating runtime version handling, ensuring all API calls are now in the standardized format.
* refactor: Remove API call v2 migration infrastructure and revert schema changes
- Deleted MIGRATION_GUIDE.md and MIGRATION_SUMMARY.md documentation files that detailed the v2 format migration process
- Removed `version` field from ApiCall model schema, reverting to previous structure
- Updated test.js MongoDB connection URI to point to test database instead of production database
* chore: Remove deprecated migration documentation and examples
- Deleted MIGRATION_FINAL_SUMMARY.md and TITLE_MIGRATION_EXAMPLES.md files as part of the cleanup process following the removal of the `endpoint_name` field from the API schema.
- This change reflects the finalization of the migration to standardize on the `title` field for API endpoint display names.
* chore: Remove rollback migration script
- Deleted rollback_migration.js file as part of cleanup following the finalization of the v2 migration
- This script was used to revert API call schema changes by restoring old_fields backups, but is no longer needed after successful migration completion
* feat: Enhance API call migration to standardize field names and clean titles
- Introduced a new function `makeFunctionName` to sanitize titles by removing non-alphanumeric characters.
- Updated migration logic to prioritize setting the `title` field from `endpoint_name` and `function_name`, ensuring all API calls have a valid title.
- Renamed `function_name` to `script_id` across various controllers and services for consistency.
- Adjusted database queries and validation schemas to reflect the new field names, improving clarity and maintainability.
---------
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* refactor: Update API call schema by removing deprecated fields (#634)
- Removed unnecessary fields from the ApiCall model schema, including `required_fields`, `axios`, `optional_fields`, `endpoint`, and `api_description`, to streamline the data structure.
- Adjusted the updateApiCalls function to eliminate the `version` field, aligning with the new schema and improving clarity in API call updates.
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* Refactor conversation service to utilize conversation_logs table, enhancing query efficiency and structure. Updated SQL queries to reflect new field mappings and improved error handling for message retrieval and updates. (#635)
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* Refactor SQL query in findThreadMessage function to improve field mapping for image URLs, changing from a string to an array format for better clarity and consistency. (#636)
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* Enhance saveApi function to conditionally update fields based on emptiness checks. Introduced a helper function to determine if values are empty, ensuring that existing non-empty fields are preserved during updates. This improves data integrity when saving API records.
* feat: Add `chatbot_auto_answers` to the list of simple fields that ca… (#641)
* feat: Add `chatbot_auto_answers` to the list of simple fields that can be updated for an agent.
* feat: add chatbot_auto_answers boolean field to Configuration and BridgeVersion models
---------
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* feat: Implement searchKnowledge endpoint to query Hippocampus API
- Added a new `searchKnowledge` function in `rag.controller.js` to handle search requests.
- Integrated Axios for making POST requests to the Hippocampus API, including necessary headers and payload.
- Updated routes in `rag.routes.js` to include a new POST endpoint for search functionality.
- Introduced `searchSchema` validation in `rag.validation.js` to ensure required parameters are provided.
* fix: Update search route to include 'chatbot' prefix for clarity
- Changed the POST endpoint from '/search' to '/chatbot/search' in rag.routes.js to better reflect its purpose within the chatbot context.
- Ensured that the middleware and validation remain intact for the updated route.
* feat: Add gtwy_web_search_filters field to agent configuration and fixed for web_search_filters (#643)
Co-authored-by: Prayanshrajput <prayanshrajput15@gmail.com>
* feat: Introduce RAG collection management and Hippocampus integration
- Added new `ragCollection.service.js` for managing RAG collections, including create, read, update, and delete operations.
- Implemented `RagCollection.model.js` to define the schema for RAG collections in MongoDB.
- Updated `rag.controller.js` to remove the `searchKnowledge` function, which is now handled by the Hippocampus service.
- Created `hippocampus.service.js` for interacting with the Hippocampus API, including methods for creating and managing collections and resources.
- Enhanced `rag.routes.js` to include new routes for collection and resource management.
- Expanded validation schemas in `rag.validation.js` to support new collection and resource operations.
* feat: Enhance createCollection to integrate Hippocampus API and improve error handling
- Added functionality to create a collection via the Hippocampus API within the createCollection method in rag.controller.js.
- Updated MongoDB collection data preparation to include Hippocampus API response.
- Improved error handling to provide more detailed error responses based on the Hippocampus API's feedback.
* feat: Refactor RAG resource management to use direct Hippocampus API calls, add `url` and `rerankerModel` fields, and update API key header.
* feat: Enhance AI service API calls to accept dynamic model parameters (#644)
- Updated `apikey.controller.js` to import `new_agent_service` for dynamic model selection.
- Modified AI service functions (`callAnthropicApi`, `callGroqApi`, `callMistralApi`, `callGeminiApi`, `callAiMlApi`) to accept a model parameter, allowing for more flexible API interactions.
- Default models are retained for backward compatibility.
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
* feat: Use Hippocampus-generated collection ID and refine resource validation
- Changed collection_id assignment from locally generated identifier to Hippocampus API response ID in createCollection.
- Added `chunkingType` as optional field in collection settings validation schema.
- Made `content` optional in createResourceSchema and added `.or('content', 'url')` validation to require at least one.
- Simplified updateResourceSchema by removing `description` and `url` fields, keeping only `title` and `content`
* refactor: Clean up authentication logic in interfaceMiddlewares and fix org_id reference in configServices
- Commented out the public chatbot authentication logic in `interfaceMiddlewares.js` for clarity and potential future use.
- Updated the `org_id` reference to `org_id` in `configServices.js` for consistency with the profile structure.
* feat: Include user_id in agent version creation process
- Added user_id to the agentVersionData object in the createVersion function to associate the created version with the user who initiated the request.
- This change enhances tracking and accountability for version management.
* fix: Update Gemini API call to use static model URL
- Changed the fetch URL in the `callGeminiApi` function to use the static model identifier `gemini-2.5-flash` for consistency and clarity in API interactions.
* feat: Add getOrCreateDefaultCollections function to manage default collections
- Implemented a new endpoint to fetch or create default collections for an organization.
- Integrated Hippocampus API to create collections if they do not already exist.
- Enhanced error handling and response structure for better clarity on collection creation status.
- Updated routes to include the new resource management functionality.
* fix: Preserve chatbot_auto_answers during version update
- Added logic to retain the chatbot_auto_answers value from the parent configuration when updating the agent version, ensuring consistency in chatbot responses across versions.
* fix: Update org_id reference in configServices to align with profile structure
- Changed the org_id assignment in getThreads and updateMessageStatus functions to use req.profile.org.id instead of req.profile.org_id for consistency and clarity.
* feat: Refactor resource creation to use collection presets and include resources in default collections response
- Modified createResourceInCollection to accept `collection_details` parameter ('high_accuracy', 'moderate', 'fastest') instead of direct collectionId.
- Implemented automatic collection lookup and settings merging based on collection preset name.
- Added helper function `filterUndefined` to clean up settings objects before merging.
- Enhanced getOrCreateDefaultCollections to fetch an
* fix: Update folder_id and user_id handling in API calls
- Changed folder_id retrieval in getAllApiCalls to use extraDetails instead of user.
- Updated user_id conversion to string in getAllApiCallsByOrgId for consistency.
- Added folder_id and user_id fields to the ApiCall model with default values.
* fix: Update agent configuration fields and enhance resource ownerId logic
- Added an empty string to the list of simple fields in updateAgentController for consistency.
- Refactored ownerId assignment logic in createResourceInCollection to accommodate different user scenarios, including embedded users and folder IDs.
- Updated API call in getAllResourcesByCollectionId to include ownerId in the request for better resource management.
* Recursive history api added
* fix: Add 'doc_ids' to simple fields in updateAgentController for enhanced configuration
- Included 'doc_ids' in the list of simple fields to improve agent configuration handling.
- Ensured consistency in the field definitions for better data management.
* Fixed GET testcase score bug
* feat: Add required 'description' field to resource creation in RAG collections
- Added 'description' parameter to createResourceInCollection request body extraction.
- Included 'description' in resource creation payload sent to Hippocampus API.
- Updated createResourceSchema validation to require 'description' field with custom error message.
* refactor: Remove ragConsumer and related services to streamline consumer management
- Deleted ragConsumer.js and its associated service files to simplify the consumer architecture.
- Updated index.js to remove references to ragConsumer, ensuring a cleaner consumer setup.
- Removed unused database services and document models to enhance code maintainability and reduce complexity.
* fix: Update org_id retrieval logic in configServices.js for improved robustness
- Modified org_id assignment to use optional chaining and fallback to org_id for better handling of undefined profile structures.
- Ensured consistent org_id retrieval across multiple service functions to enhance stability and prevent potential errors.
* feat: Add optional 'type' parameter to embed creation for supporting RAG embed folders (#666)
- Added 'type' field extraction from request body in createEmbed controller with default fallback to embed.
- Updated FolderModel.create to use dynamic folder_type instead of hardcoded embed value.
- Enhanced createEmbed validation schema to accept 'type' parameter with allowed values embed and rag_embed, defaulting to embed.
* feat: Add ownerId parameter to RAG collection resource retrieval for user-specific filtering
- Added ownerId construction logic in middleware combining org_id, folder_id, and user_id when available.
- Attached ownerId to request object for downstream use in RAG operations.
- Updated getOrCreateDefaultCollections to include ownerId query parameter in Hippocampus API resource fetch calls.
* feat: Enhance embed details with dynamic folder_id extraction
- Added folder_id extraction from the request to the extraDetails object in ragEmbedUserLogin controller.
- Updated EmbeddecodeToken middleware to include folder_id in the profile object for improved context handling.
* refactor: Remove searchKnowledge API and related validation schema
- Deleted the searchKnowledge function from rag.controller.js to streamline the controller.
- Removed the searchSchema validation from rag.validation.js as it is no longer needed.
* feat: Add 'gpt_memory_context' field to BridgeVersion and Configuration models (#670)
- Introduced a new optional 'gpt_memory_context' field in both BridgeVersion and Configuration schemas, allowing for a string value with a default of null.
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
---------
Co-authored-by: Viasocket Interns <developers@whozzat.com>
Co-authored-by: Prayanshrajput <prayansh75@gmail.com>
Co-authored-by: Husain Baghwala <89205542+Husainbw786@users.noreply.github.com>
Co-authored-by: Harsh Sahu <harksahu@Harshs-Mac-mini.local>
Co-authored-by: Husain Baghwala <husainbw123@gmail.com>
Co-authored-by: Husain Baghwala <husainhackerrank@gmail.com>
Co-authored-by: Yogesh Patel <ygurjar932@gmail.com>
Co-authored-by: Harsh Sahu <harksahu@Harshs-Mac-mini.localdomain>
Co-authored-by: Natwar589 <natwarrathor961@gmail.com>
Co-authored-by: Natwar Singh Rathor <87537893+Natwar589@users.noreply.github.com>
Co-authored-by: Yogesh Patel <yogeshpatel@gmail.com>
Co-authored-by: Prayanshrajput <104488357+Prayanshrajput@users.noreply.github.com>
Co-authored-by: Prayanshrajput <prayanshrajput15@gmail.com>
Co-authored-by: adityajunwal <adityakjunwal@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.