Skip to content

feat: add Neo4j backup support #448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
IronAlloy wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add Neo4j backup support #448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ ENV INFLUX1_CLIENT_VERSION=1.8.0 \
MSSQL_VERSION=18.4.1.1-1 \
MYSQL_VERSION=mysql-8.4.4 \
MYSQL_REPO_URL=https://github.com/mysql/mysql-server \
NEO4J_VERSION=5.26.0 \
AWS_CLI_VERSION=1.36.40 \
CONTAINER_ENABLE_MESSAGING=TRUE \
CONTAINER_ENABLE_MONITORING=TRUE \
Expand Down Expand Up @@ -52,6 +53,7 @@ RUN source /assets/functions/00-container && \
mariadb-connector-c \
mongodb-tools \
ncurses \
openjdk17-jre-headless \
openssl \
pigz \
pixz \
Expand Down Expand Up @@ -97,6 +99,14 @@ RUN source /assets/functions/00-container && \
echo >&2 "Unable to build Influx 2 on this system" ; \
fi ; \
\
mkdir -p /usr/src/neo4j && \
curl -sSL https://dist.neo4j.org/neo4j-community-${NEO4J_VERSION}-unix.tar.gz | tar xzf - --strip-components=1 -C /usr/src/neo4j && \
mkdir -p /opt/neo4j/bin /opt/neo4j/lib && \
cp /usr/src/neo4j/bin/cypher-shell /opt/neo4j/bin/ && \
cp /usr/src/neo4j/lib/*.jar /opt/neo4j/lib/ && \
chmod +x /opt/neo4j/bin/cypher-shell && \
ln -sf /opt/neo4j/bin/cypher-shell /usr/local/bin/cypher-shell && \
\
clone_git_repo https://github.com/influxdata/influxdb "${INFLUX1_CLIENT_VERSION}" && \
go build -o /usr/sbin/influxd ./cmd/influxd && \
strip /usr/sbin/influxd && \
Expand Down
48 changes: 47 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

This will build a container for backing up multiple types of DB Servers

Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Postgres, Redis servers.
Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Neo4J, Postgres, Redis servers.

- dump to local filesystem or backup to S3 Compatible services, and Azure.
- multiple backup job support
Expand Down Expand Up @@ -62,6 +62,7 @@ Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Postgres, Red
- [MariaDB/MySQL](#mariadbmysql)
- [Microsoft SQL](#microsoft-sql)
- [MongoDB](#mongodb)
- [Neo4J](#neo4j)
- [Postgresql](#postgresql)
- [Redis](#redis)
- [Default Storage Options](#default-storage-options)
Expand All @@ -82,6 +83,7 @@ Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Postgres, Red
- [MariaDB/MySQL](#mariadbmysql-1)
- [Microsoft SQL](#microsoft-sql-1)
- [MongoDB](#mongodb-1)
- [Neo4J](#neo4j-1)
- [Postgresql](#postgresql-1)
- [Redis](#redis-1)
- [SQLite](#sqlite)
Expand Down Expand Up @@ -297,6 +299,38 @@ Encryption occurs after compression and the encrypted filename will have a `.gpg
| | This environment variable will be parsed and populate the `DB_NAME` and `DB_HOST` variables to properly build your backup filenames. | | |
| | You can override them by making your own entries | | |

###### Neo4J

| Variable | Description | Default | `_FILE` |
| -------------- | ----------- | ------- | ------- |
| `DEFAULT_PORT` | Neo4J Port | `7687` | x |

> Requires the [APOC](https://neo4j.com/docs/apoc/current/) plugin to be installed on the Neo4J server.
> `DB_NAME` should be set to the target database name (default Neo4J database is named `neo4j`).
> Uses the Bolt protocol via `cypher-shell` to export all data as Cypher statements.
>
> **Installing APOC on your Neo4J server:**
>
> *Docker (environment variable):*
> ```bash
> docker run \
> -e NEO4J_PLUGINS='["apoc"]' \
> -e NEO4J_apoc_export_file_enabled=true \
> -e NEO4J_dbms_security_procedures__unrestricted='apoc.*' \
> neo4j:latest
> ```
>
> *Docker Compose:*
> ```yaml
> environment:
> - NEO4J_PLUGINS=["apoc"]
> - NEO4J_apoc_export_file_enabled=true
> - NEO4J_dbms_security_procedures__unrestricted=apoc.*
> ```
>
> *Manual installation:*
> Download the APOC JAR from [Neo4J APOC Releases](https://github.com/neo4j/apoc/releases) and place it in your Neo4J `plugins/` directory.

###### Postgresql

| Variable | Description | Default | `_FILE` |
Expand Down Expand Up @@ -571,6 +605,18 @@ Encryption will occur after compression and the resulting filename will have a `
| | This environment variable will be parsed and populate the `DB_NAME` and `DB_HOST` variables to properly build your backup filenames. | | |
| | You can override them by making your own entries | | |

###### Neo4J

| Variable | Description | Default | `_FILE` |
| ------------------------ | --------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | | |
| `DB01_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | | |
| `DB01_NAME` | Database name e.g. `neo4j` | | x |
| `DB01_PORT` | Neo4J Bolt Port | `7687` | x |

> Requires the [APOC](https://neo4j.com/docs/apoc/current/) plugin to be installed on the Neo4J server.
> Uses `cypher-shell` via the Bolt protocol to export the entire database as Cypher statements.

###### Postgresql

| Variable | Description | Default | `_FILE` |
Expand Down
47 changes: 47 additions & 0 deletions install/assets/functions/10-db-backup
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,13 @@ EOF
dbtype=mssql
backup_job_db_port=${backup_job_db_port:-1433}
;;
neo4j* )
dbtype=neo4j
backup_job_db_port=${backup_job_db_port:-7687}
check_var backup_job_db_name DB"${v_instance}"_NAME "database name"
check_var backup_job_db_user DB"${v_instance}"_USER "database username"
check_var backup_job_db_pass DB"${v_instance}"_PASS "database password"
;;
postgres* | "pgsql" )
dbtype=pgsql
backup_job_db_port=${backup_job_db_port:-5432}
Expand Down Expand Up @@ -699,6 +706,38 @@ backup_mssql() {
esac
}

backup_neo4j() {
if var_true "${DEBUG_BACKUP_NEO4J}" ; then debug on; fi
prepare_dbbackup
backup_job_filename=neo4j_${backup_job_db_name}_${backup_job_db_host,,}_${now}.cypher
backup_job_filename_base=neo4j_${backup_job_db_name}_${backup_job_db_host,,}
compression
pre_dbbackup "${backup_job_db_name}"
write_log notice "Dumping Neo4J database: '${backup_job_db_name}' ${compression_string}"
if var_true "${DEBUG_BACKUP_NEO4J}" ; then debug on; fi
run_as_user ${play_fair} cypher-shell \
-a "neo4j://${backup_job_db_host}:${backup_job_db_port}" \
-u "${backup_job_db_user}" \
-p "${backup_job_db_pass}" \
-d "${backup_job_db_name}" \
--format plain \
${backup_job_extra_opts} \
${backup_job_extra_backup_opts} \
"CALL apoc.export.cypher.all(null, {stream:true, format:'cypher-shell'}) YIELD cypherStatements RETURN cypherStatements" \
| ${compress_cmd} | run_as_user tee "${temporary_directory}"/"${backup_job_filename}" > /dev/null
exit_code=$((PIPESTATUS[0] + PIPESTATUS[1] + PIPESTATUS[2]))
if var_true "${DEBUG_BACKUP_NEO4J}" ; then debug off; fi
check_exit_code backup "${backup_job_filename}"
timer backup finish
file_encryption
generate_checksum
move_dbbackup
check_exit_code move "${backup_job_filename}"
post_dbbackup "${backup_job_db_name}"
cleanup_old_data
if var_true "${DEBUG_BACKUP_NEO4J}" ; then debug off; fi
}

backup_mysql() {
if var_true "${DEBUG_BACKUP_MYSQL}" ; then debug on; fi
if var_true "${backup_job_mysql_events}" ; then
Expand Down Expand Up @@ -1014,6 +1053,14 @@ check_availability() {
write_log warn "MSSQL Host '${backup_job_db_host}' is not accessible, retrying.. ($counter seconds so far)"
done
;;
"neo4j" )
counter=0
while ! (run_as_user cypher-shell -a "neo4j://${backup_job_db_host}:${backup_job_db_port}" -u "${backup_job_db_user}" -p "${backup_job_db_pass}" -d "${backup_job_db_name}" "RETURN 1;" > /dev/null 2>&1) ; do
sleep 5
(( counter+=5 ))
write_log warn "Neo4J Host '${backup_job_db_host}' is not accessible, retrying.. ($counter seconds so far)"
done
;;
"pgsql" )
counter=0
until run_as_user pg_isready --host=${backup_job_db_host} --port=${backup_job_db_port} -q
Expand Down