Skip to content

catpineapple/metabase-doris-driver

Repository files navigation

Metabase Apache Doris Driver

This is a Metabase driver for Apache Doris, an open-source MPP analytical database.

Features

  • Full compatibility with Doris SQL syntax (MySQL-compatible)
  • Catalog support for multi-catalog queries (internal, Hive, Iceberg, etc.)
  • Support for all Doris data types including LARGEINT, DECIMALV3, DATEV2, etc.
  • Date/time functions and timezone support
  • Aggregation functions including percentile
  • Window functions support
  • Full/Left/Right/Inner JOIN support

Installation

Method 1: Pre-built JAR

  1. Download the latest doris.metabase-driver.jar from the releases page
  2. Copy the JAR file to your Metabase plugins directory:
    cp doris.metabase-driver.jar /path/to/metabase/plugins/
  3. Restart Metabase

Method 2: Build from Source

环境要求

  • Java: JDK 11 或更高版本
  • Clojure CLI: 1.11.0 或更高版本

安装 Clojure CLI

macOS (使用 Homebrew):

brew install clojure/tools/clojure

Linux:

curl -L -O https://github.com/clojure/brew-install/releases/latest/download/linux-install.sh
chmod +x linux-install.sh
sudo ./linux-install.sh

Windows: 参考官方文档: https://clojure.org/guides/install_clojure#_windows

准备 MySQL JDBC 驱动

MySQL JDBC 驱动会自动从 Maven 仓库下载,无需手动准备。

构建 JAR

# 在 doris/ 目录下执行
clojure -T:build uber

构建成功后,JAR 文件位于:target/doris-driver.metabase-driver.jar

使用 Metabase 官方构建工具(可选)

如果你有完整的 Metabase 源码,可以使用官方构建工具:

  1. 修改 deps.edn:metabase-build 的路径指向你的 Metabase 源码
  2. 运行:
    clojure -X:metabase-build

验证构建结果

# 查看 JAR 文件大小(应该约 9MB)
ls -lh target/doris-driver.metabase-driver.jar

# 验证 JAR 内容
unzip -l target/doris-driver.metabase-driver.jar | grep -E "doris|metabase-plugin|mysql"

安装到 Metabase

cp target/doris-driver.metabase-driver.jar /path/to/metabase/plugins/

# Docker 环境示例
docker cp target/doris-driver.metabase-driver.jar metabase:/plugins/

然后重启 Metabase

Configuration

When adding a new Apache Doris database in Metabase, you'll need to provide:

Field Description Default
Host Doris FE host address -
Port Doris FE MySQL protocol port 9030
Catalog Doris catalog name internal
Database name Database to connect to -
Username Database username -
Password Database password -

Catalog Support

Apache Doris supports multi-catalog to access external data sources. You can specify the catalog in the connection settings:

  • internal - The default internal catalog for Doris native tables
  • hive - Hive catalog for accessing Hive tables
  • iceberg - Iceberg catalog for accessing Iceberg tables
  • Other custom catalogs configured in your Doris cluster

Example: To connect to a Hive catalog:

  1. Set Catalog to your Hive catalog name (e.g., hive_catalog)
  2. Set Database name to the Hive database you want to query

Data Type Mapping

Doris Type Metabase Type
BOOLEAN Boolean
TINYINT, SMALLINT, INT Integer
BIGINT, LARGEINT Big Integer
FLOAT, DOUBLE Float
DECIMAL, DECIMALV2, DECIMALV3 Decimal
DATE, DATEV2 Date
DATETIME, DATETIMEV2 DateTime
CHAR, VARCHAR, STRING, TEXT Text
JSON, JSONB JSON
ARRAY, MAP, STRUCT Other
HLL, BITMAP Other

Limitations

  • Foreign key detection is not supported (Doris doesn't enforce foreign keys)
  • Data uploads are not supported
  • JSON nested field unfolding is not supported
  • Model persistence is not supported
  • Table privileges querying is not supported (Doris doesn't support SHOW GRANTS FOR CURRENT_USER())

Doris vs MySQL Compatibility Notes

This driver is based on MySQL protocol but includes specific fixes for Doris incompatibilities:

Issue 1: Boolean Type Handling

  • Problem: MySQL returns boolean values as integers (0/1), but Doris returns actual boolean values (true/false)
  • Solution: The driver's to-boolean function handles both formats correctly

Issue 2: SHOW GRANTS Syntax

  • Problem: MySQL driver uses SHOW GRANTS FOR CURRENT_USER() to query table privileges, which Doris doesn't support
  • Solution: The :table-privileges feature is disabled, and current-user-table-privileges returns nil

Issue 3: Column Metadata Retrieval

  • Problem: Using SHOW COLUMNS or information_schema.columns may not work correctly in Doris
  • Solution: The driver uses DESCRIBE table_name syntax to retrieve column information

Troubleshooting

Connection Issues

  1. Cannot connect: Ensure the Doris FE is accessible and the MySQL protocol port (default 9030) is open
  2. Authentication failed: Verify username and password
  3. Catalog not found: Ensure the catalog is properly configured in your Doris cluster

Query Issues

  1. Timeout errors: For large datasets, consider using Doris's query optimization features
  2. Memory errors: Check Doris FE memory settings

目录结构

doris/
├── deps.edn                    # 依赖配置和构建配置
├── build/
│   └── build.clj               # 构建脚本
├── lib/
│   └── mysql-connector-j-8.3.0.jar  # MySQL JDBC 驱动(需手动放置)
├── resources/
│   └── metabase-plugin.yaml    # 插件配置文件
├── src/
│   └── metabase/
│       └── driver/
│           └── doris.clj       # 驱动实现
├── test/
│   └── metabase/
│       └── driver/
│           └── doris_test.clj  # 测试文件
└── target/                     # 构建输出目录(gitignore)
    └── doris-driver.metabase-driver.jar

Development

Running Tests

clojure -M:test

Building

clojure -T:build uber

清理构建产物

rm -rf target/ classes/ .cpcache/

License

This driver is released under the same license as Metabase.

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

About

Apache Doris driver for Metabase.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors