Ff2Cql
A simple transfer data between NiFi and CQL (support Apache Cassandra, ScyllaDB, AstraDB, YugabyteDB, etc.). The implementation details:
- development NiFi v2 processor (with controller) and java application (support Java 17/21+)
- support Apache Cassandra v4/v5, ScyllaDB, AstraDB, YugabyteDB based on CQL (Cassandra Query Language)
1. The main motivation
- the Apache NiFi v2 does not support Apache Cassandra v4/v5 (NiFi v2 removed the Cassandra processor due to security vulnerabilities and unmaintained code. The processor supported only Cassandra v3 not newer.)
2. Usage in NiFi
You can use this preferred way (NiFi v2 processor with controller):
- ✅ PutCQL, put data to CQL solution, where inputs are FlowFiles
- ✅ GetCQL, get data from CQL solution, where outputs are FlowFiles
- NOTE:
- You can download the latest version, see 'nifi-cql-nar-*.nar' file
- Sample Flow definitions, see
or two other older alternative ways:
- ✅ ExecuteProcess with java application (see 'Ff2Cql-*.jar'), where inputs are CSV files
- ✅ ExecuteStreamCommand with java application (see 'Ff2Cql-*.jar'), where inputs are FlowFiles via stdin
3. Connection setting
You can see a sample of relevant setting for CQL controller here (these connection settings are used directly in unit tests):
3.1 Cassandra
- Cassandra setting
- access with IP addresses (name/password)
3.2 Scylla
- Scylla setting
- access with IP address (without name/password)
3.3 AstraDB
- AstraDB setting
- access with Security Connection Bundle (SCB) and Token
- NOTE about SCB: see the file with '.zip' suffix
- How to get the SCB? Go to the https://astra.datastax.com/..., Menu Databases/Your database/Region/Download SCB
- NOTE about token: see the file with 'token.json' suffix
- How to get the token? Go to the https://astra.datastax.com/..., Menu Tokens/Generate Token/Token Details/Download Token Details
3.4 YugabyteDB
- YugabyteDB setting
- access with IP address (without name/password)
4. Apache NiFi v2 & Scylla/Cassandra v5
