[BUG] Uncaught exception occurred [op:error_handling] #451
Description
Title
After SFTP fail and hit refresh got an Uncaught exception occurred [op:error_handling] who crash the backend and make him restart
Platform
App - Linux
Server Installation Method
Docker
Version
1.11.0
Troubleshooting
- I have examined logs and tried to find the issue
- I have reviewed opened and closed issues
- I have tried restarting the application
- I have read Known Issues (Known Issues (Read Before Posting) #296)
The Problem
If file manager timeout and i hit refresh the backend crash.
This lead to backend reinitialisation and session crash cause JWT isn't verified (so i need to reauth)
How to Reproduce
- Create a Credential (ssh keys + passkey)
- Create an host with that credential
- Open console (in my case it work without problem)
- Open file manager (in my case he timeout)
- Hit the Refresh button (the arrow in circle)
Additional Context
Specificity on my computer:
I have a Netbird VPN on with trafic forwarded to exit node. The docker server run on the SAME computer.
The docker logs:
docker logs termixConfiguring web UI to run on port: 2280
SSL enabled - using HTTPS configuration with redirect
Checking SSL certificate configuration...
SSL certificates found, checking validity...
SSL certificates are valid and will be reused for domain: localhost
Starting nginx...
Starting backend services...
[2:38:10 PM] [INFO] [📦] Termix Backend starting - Version: 1.11.0 [op:startup]
[2:38:10 PM] [INFO] [🚀] SSL not enabled - skipping certificate generation [op:ssl_disabled_default]
[2:38:10 PM] [INFO] [🗄] Initializing SQLite database [op:db_init]
[2:38:10 PM] [SUCCESS] [🗄] Schema migration completed [op:schema_migration]
[2:38:12 PM] [INFO] [📡] Found 0 autostart hosts and 1 total hosts for endpointHost resolution
[2:38:15 PM] [WARN] [🗄] Session not found during JWT verification [op:jwt_verify_session_not_found,user:BTHpdH3dEnqKragdzJGoo,session:DCqMqf4TALz2Vc_mlEaUB]
[2:38:15 PM] [WARN] [🗄] Session not found during JWT verification [op:jwt_verify_session_not_found,user:BTHpdH3dEnqKragdzJGoo,session:DCqMqf4TALz2Vc_mlEaUB]
[2:38:23 PM] [SUCCESS] [🔐] User logged in successfully: mathys [op:user_login_success,user:BTHpdH3dEnqKragdzJGoo]
[2:38:33 PM] [INFO] [🖥] Creating shell [op:ssh_shell_start,host:1]
[2:39:46 PM] [ERROR] [📊] Failed to collect metrics for host
{
operation: 'metrics_poll_failed',
hostId: 1,
hostName: 'Zabbix',
error: 'Request timeout after 60000ms for host 1',
backoff: 'Retry in 30s (attempt 1/5)'
}
[2:40:37 PM] [WARN] [📁] Deferring session cleanup for 1 - 3 active operations [op:cleanup_deferred,session:1]
[2:40:37 PM] [WARN] [📁] SFTP failed for listFiles, trying fallback: No response from server
[2:40:37 PM] [ERROR] [🚀] Uncaught exception occurred [op:error_handling]
Error: Not connected
at Client.exec (/app/node_modules/ssh2/lib/client.js:1194:13)
at tryFallbackMethod (file:///app/dist/backend/backend/ssh/file-manager.js:1604:24)
at file:///app/dist/backend/backend/ssh/file-manager.js:1539:21
at /app/node_modules/ssh2/lib/client.js:1566:9
at wrapper (/app/node_modules/ssh2/lib/client.js:1655:5)
at onChannelOpenFailure (/app/node_modules/ssh2/lib/utils.js:25:3)
at onCHANNEL_CLOSE (/app/node_modules/ssh2/lib/utils.js:32:5)
at ChannelManager.cleanup (/app/node_modules/ssh2/lib/utils.js:200:7)
at Socket.<anonymous> (/app/node_modules/ssh2/lib/client.js:829:21)
at Socket.emit (node:events:519:28)
Configuring web UI to run on port: 2280
SSL enabled - using HTTPS configuration with redirect
Checking SSL certificate configuration...
SSL certificates found, checking validity...
SSL certificates are valid and will be reused for domain: localhost
Starting nginx...
Starting backend services...
[2:40:38 PM] [INFO] [📦] Termix Backend starting - Version: 1.11.0 [op:startup]
[2:40:38 PM] [INFO] [🚀] SSL not enabled - skipping certificate generation [op:ssl_disabled_default]
[2:40:38 PM] [INFO] [🗄] Initializing SQLite database [op:db_init]
[2:40:38 PM] [SUCCESS] [🗄] Schema migration completed [op:schema_migration]
[2:40:41 PM] [INFO] [📡] Found 0 autostart hosts and 1 total hosts for endpointHost resolution
[2:40:55 PM] [WARN] [🗄] Session not found during JWT verification [op:jwt_verify_session_not_found,user:BTHpdH3dEnqKragdzJGoo,session:TKbXTnml6t3C4KfIGhy8D]
[2:45:12 PM] [SUCCESS] [🔐] User logged in successfully: mathys [op:user_login_success,user:BTHpdH3dEnqKragdzJGoo]
[2:46:14 PM] [ERROR] [📊] Failed to collect metrics for host
{
operation: 'metrics_poll_failed',
hostId: 1,
hostName: 'Zabbix',
error: 'Request timeout after 60000ms for host 1',
backoff: 'Retry in 30s (attempt 1/5)'
}
[2:47:44 PM] [WARN] [📊] Removing unhealthy connection from pool [op:remove_dead_connection]
[2:47:45 PM] [ERROR] [📊] Failed to collect metrics for host
{
operation: 'metrics_poll_failed',
hostId: 1,
hostName: 'Zabbix',
error: 'Request timeout after 60000ms for host 1',
backoff: 'Retry in 60s (attempt 2/5)'
}
[2:48:45 PM] [ERROR] [📊] Failed to collect metrics for host
{
operation: 'metrics_poll_failed',
hostId: 1,
hostName: 'Zabbix',
error: 'Request timeout after 60000ms for host 1',
backoff: 'Retry in 120s (attempt 3/5)'
}
[2:50:31 PM] [WARN] [📁] Deferring session cleanup for 1 - 3 active operations [op:cleanup_deferred,session:1]
[2:50:31 PM] [WARN] [📁] SFTP failed for listFiles, trying fallback: No response from server
[2:50:31 PM] [ERROR] [🚀] Uncaught exception occurred [op:error_handling]
Error: Not connected
at Client.exec (/app/node_modules/ssh2/lib/client.js:1194:13)
at tryFallbackMethod (file:///app/dist/backend/backend/ssh/file-manager.js:1604:24)
at file:///app/dist/backend/backend/ssh/file-manager.js:1539:21
at /app/node_modules/ssh2/lib/client.js:1566:9
at wrapper (/app/node_modules/ssh2/lib/client.js:1655:5)
at onChannelOpenFailure (/app/node_modules/ssh2/lib/utils.js:25:3)
at onCHANNEL_CLOSE (/app/node_modules/ssh2/lib/utils.js:32:5)
at ChannelManager.cleanup (/app/node_modules/ssh2/lib/utils.js:200:7)
at Socket.<anonymous> (/app/node_modules/ssh2/lib/client.js:829:21)
at Socket.emit (node:events:519:28)
Configuring web UI to run on port: 2280
SSL enabled - using HTTPS configuration with redirect
Checking SSL certificate configuration...
SSL certificates found, checking validity...
SSL certificates are valid and will be reused for domain: localhost
Starting nginx...
Starting backend services...
[2:50:32 PM] [INFO] [📦] Termix Backend starting - Version: 1.11.0 [op:startup]
[2:50:32 PM] [INFO] [🚀] SSL not enabled - skipping certificate generation [op:ssl_disabled_default]
[2:50:32 PM] [INFO] [🗄] Initializing SQLite database [op:db_init]
[2:50:32 PM] [SUCCESS] [🗄] Schema migration completed [op:schema_migration]
[2:50:34 PM] [INFO] [📡] Found 0 autostart hosts and 1 total hosts for endpointHost resolution
[2:50:36 PM] [WARN] [🗄] Session not found during JWT verification [op:jwt_verify_session_not_found,user:BTHpdH3dEnqKragdzJGoo,session:gFZXJGQM5xGO05Nftbb7x]
(note: the crash happend 2 time here with the same method, one via app (downloaded via aur) and one via firefox)
This also happend without SSL.
Another strange thing is that the metric pool timeout but like i said previously SSH works.