Migrate to Private Docker Image & Add Federated Queries Test Suite by mkottakota1 · Pull Request #21 · vertica/ODBC-Loader

mkottakota1 · 2026-02-02T07:20:45Z

Description

This PR adds comprehensive test coverage for the ODBCLoader's federated query capabilities while migrating the CI/CD pipeline from the public Vertica Docker image to a private image for enhanced security, control, and customization.

Changes

Test Suite

New test file: tests/federated_queries.sql - Contains four test scenarios demonstrating federated query functionality:
1. Predicate Pushdown - Validates that WHERE conditions are pushed down to the external database
2. Column Pruning - Verifies that only required columns are fetched from the external source
3. Federated Joins - Tests joining external tables with Vertica-managed tables
4. Complex Predicates - Ensures multiple WHERE conditions are properly optimized
Expected output file: tests/expected/federated_queries.out - Contains baseline expected results for test validation
Updated Makefile:
- Modified test target to run federated queries test suite
- Added test_debug target for easier troubleshooting
- Enhanced test output with dynamic progress indicators and validation status

Infrastructure

Docker Image Migration:
- Migrated from public vertica/vertica-ce image to private custom Docker image
- Updated GitHub Actions workflow to use private image registry
- Added Docker authentication for private image access
- Improved build consistency and security posture

Testing

All tests pass successfully with live output display
Test validation compares actual results against expected output
ODBC driver compatibility ensured across different MySQL ODBC versions
Tests run against private Vertica image with enhanced configuration

Motivation

Federated Queries: Essential feature for joining Vertica tables with external databases while optimizing data movement through predicate pushdown and column pruning
Private Image: Provides better control over dependencies, security patches, and custom configurations for CI/CD pipeline stability

Related Issues

Closes #[issue-number]

Checklist

Federated queries tests added/updated
Expected output files created
Makefile updated with new test targets
All tests passing with private image
GitHub Actions workflow updated for private image registry
Docker authentication configured
CI/CD pipeline validated

sivaalamp · 2026-02-06T09:06:03Z

.github/workflows/ci.yml

+        docker exec -u dbadmin vertica_docker admintools -t create_db \
+          -s localhost \
+          -d mydb \
+          -p "MySecretPass"


Hardcoded Database Password (Security Risk)

I have removed the hardcoded password and followed the previous DB setup

sivaalamp · 2026-02-06T09:06:33Z

.github/workflows/ci.yml

+    - name: Login to Docker Hub
+      uses: docker/login-action@v3
+      with:
+        username: mkottakota419


why are the personal credentials used in pipelines

To access the private docker image, I have uploaded in my dockerhub and used it in the pipeline

I have removed the hardcoded Username and added in the Github secrets

sivaalamp · 2026-02-06T09:12:49Z

.github/workflows/ci.yml

+          -s localhost \
+          -d mydb \
+          -p "MySecretPass"
+


No error handling if the create fails

Added error handling at necessary stages

sivaalamp · 2026-02-06T09:19:56Z

tests/federated_queries.sql

+-- Test 2: Query with column pruning
+-- When only specific columns are selected, ODBCLoader optimizes the external query
+-- to fetch only the required columns. Columns not in the Vertica query are replaced
+-- with NULL values to match the External Table schema


can you please check if this comment is correct?

I have corrected the comment now

sivaalamp

Review done

mkottakota1 · 2026-02-06T11:45:36Z

Review done

I have addressed your comment, kindly review and approve

sivaalamp

The changes now look good to me, hence approving

mkottakota1 added 13 commits January 13, 2026 08:59

Using customized private repo

b4cf660

Rebuild DB

2bc6aee

Create Fresh DB

107d3f1

Installing Iproute package

ab79fc6

ensure we are running make in the exact folder where the Makefile sits

c126833

Passing DB Password

96f20db

Add wget installable

7198700

Add VSQL ENV Var for running tests

7e00911

Coping odbc.ini files to etc folder

5abdc12

Finding correct so file name

17a56e9

Introducing federted queries test and skipping copy_test

9a8d1f8

Updated federated test out file

30c5fd9

Adding more info to test run

d313c51

mkottakota1 requested review from sivaalamp and ssharieff21 February 2, 2026 08:22

sivaalamp reviewed Feb 6, 2026

View reviewed changes

sivaalamp requested changes Feb 6, 2026

View reviewed changes

mkottakota1 added 2 commits February 6, 2026 11:27

Added error handling and removed hardcoded DB password

c14c3f4

corrected the commane for column pruning test

4af4453

mkottakota1 requested a review from sivaalamp February 6, 2026 11:45

Removing hardcoded USERNAME

acc4011

sivaalamp approved these changes Feb 9, 2026

View reviewed changes

mkottakota1 merged commit 3f24583 into main Feb 9, 2026
2 checks passed

Conversation

mkottakota1 commented Feb 2, 2026

Description

Changes

Test Suite

Infrastructure

Testing

Motivation

Related Issues

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sivaalamp left a comment

Choose a reason for hiding this comment

Uh oh!

mkottakota1 commented Feb 6, 2026

Uh oh!

sivaalamp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments