resurfaceio-importer

Import NDJSON into Resurface database

This open-source Java utility imports API calls (stored in NDJSON format) to a remote Resurface database. This command-line utility works with Resurface databases on Kubernetes or Docker, and includes a few helpful options for load testing.

Usage

Download executable jar:

wget https://dl.cloudsmith.io/public/resurfaceio/public/maven/io/resurface/resurfaceio-importer/3.6.1/resurfaceio-importer-3.6.1.jar

Importing a local file:

java -DFILE=XXX.ndjson.gz -DHOST=localhost -DPORT=443 -DBATCH_SIZE=128 -DLIMIT_MESSAGES=0 -DLIMIT_MILLIS=0 -DREPEAT=no -DSATURATED_STOP=no -Xmx512M -jar resurfaceio-importer-3.6.1.jar

⚠️ This utility reads files in .ndjson.gz format exclusively. This compressed file format can be exported from a Resurface database, or generated using the ndjson library.

Parameters

FILE: local .ndjson.gz file to import to remote database
HOST: machine name for remote database
PORT: network port for remote database (80 or 443 for Kubernetes, 7701 for Docker)

BATCH_SIZE: default is '128', messages sent in a single POST
LIMIT_MESSAGES: default is '0' (unlimited), quit after this many messages
LIMIT_MILLIS: default is '0' (unlimited), quit after this many milliseconds
REPEAT: default is 'no', import file until process is cancelled
SATURATED_STOP: default is 'no', quit if database is saturated
URL: override HOST and PORT with custom URL for remote database

Dependencies

• Java 17
• resurfaceio/ndjson

Installing with Maven

⚠️ We publish our official binaries on CloudSmith rather than Maven Central, because CloudSmith is awesome and free for open-source projects.

If you want to call this utility from your own Java application, add these sections to pom.xml to install:

<dependency>
    <groupId>io.resurface</groupId>
    <artifactId>resurfaceio-importer</artifactId>
    <version>3.6.1</version>
</dependency>

<repositories>
    <repository>
        <id>resurfaceio-public</id>
        <url>https://dl.cloudsmith.io/public/resurfaceio/public/maven/</url>
        <releases>
            <enabled>true</enabled>
            <updatePolicy>always</updatePolicy>
        </releases>
    </repository>
</repositories>

---

© 2016-2024 Graylog, Inc.