📡 How to Use Streams in ZIMX Studio

1. What Is NDJSON?

NDJSON (Newline Delimited JSON) is a simple, streaming-friendly text format where each line is a complete, self-contained JSON object. It is sometimes called JSON Lines.

Valid NDJSON example

{"db":"myapp","host":"dbhost","proc":"pGetUsers()","params":""}
{"db":"myapp","host":"dbhost","proc":"pInsertUser(lName,lEmail)","params":"Alice,alice@example.com"}
{"db":"myapp","host":"dbhost","proc":"pGetEvents(lType)","params":"login"}

What makes NDJSON great for streaming?

• Each record is self-contained — the receiver can parse it as soon as the line arrives.
• No need to buffer the entire response before parsing.
• Works perfectly over WebSockets where messages arrive one at a time.
• Human-readable and easy to debug in browser DevTools.

2. What Is the ws-dba Endpoint?

ws-dba is ZIMX's WebSocket endpoint for direct, real-time database interaction. Every message you send calls a named ZIM procedure on a specific database and host using NDJSON, streaming the results back line by line.

Standard message shape

{
  "db":     "myapp",                      // required — database name
  "host":   "dbhost",                     // required — host / server name
  "proc":   "pGetUsers(lLimit,lFilter)",  // required — procedure name with parameter names
  "params": "50,active"                   // required — comma-separated positional values (string)
}

Endpoint URL pattern

wss://<your-server>/api/ws-dba

Replace <your-server> with your ZIMX host. Ask your administrator if you are unsure of the correct address.

3. Connecting to a Stream in Stream Manager

wss://cloud.example.com/api/ws-dba

{"db":"myapp","host":"dbhost","proc":"pGetUsers()","params":""}

4. Sending NDJSON Commands

Each command calls a named ZIM procedure on a specific database and host. The message must be a single, valid JSON object followed by a newline (\n).

params vs payload

Call a procedure with no parameters

{"db":"myapp","host":"dbhost","proc":"pGetAllUsers()","params":""}

Call a procedure with parameters (params)

{"db":"myapp","host":"dbhost","proc":"pGetUsers(lLimit,lFilter)","params":"50,active"}

Insert via procedure — using params (positional)

{"db":"myapp","host":"dbhost","proc":"pInsertUser(lName,lEmail)","params":"Alice,alice@example.com"}

Insert via procedure — using payload (named object)

{"db":"myapp","host":"dbhost","proc":"pInsertUser(lName,lEmail)","params":"","payload":{"lName":"Alice","lEmail":"alice@example.com"}}

Update via procedure

{"db":"myapp","host":"dbhost","proc":"pUpdateUserEmail(lId,lEmail)","params":"42,alice@new.com"}

Delete via procedure

{"db":"myapp","host":"dbhost","proc":"pDeleteUser(lId)","params":"42"}

5. Receiving NDJSON Data

The server responds to your commands with NDJSON — one JSON object per line. Each object represents a single record or a status message.

Example: response to a procedure call

{"id":1,"name":"Alice","email":"alice@example.com"}
{"id":2,"name":"Bob","email":"bob@example.com"}
{"id":3,"name":"Carol","email":"carol@example.com"}

Each line is a separate record. There is no wrapping array and no commas between lines.

End-of-stream marker

When a query finishes, ZIMX typically sends a completion signal such as:

{"type":"done","count":3}

Listen for type === "done" (or an equivalent sentinel from your server configuration) to know when all records have been delivered.

Parsing in the Stream Manager

The Stream Manager automatically parses each incoming line as JSON and displays field names and values in its Fields tab. As data arrives, new column names are detected and shown in real time.

6. JavaScript Quick-Start

Use the browser's native WebSocket API to connect from any JavaScript environment.

// 1. Open a ws-dba connection
const sock = new WebSocket("wss://cloud.example.com/api/ws-dba");

// 2. Helper — send one NDJSON command
function send(obj) {
  sock.send(JSON.stringify(obj) + "\n");
}

// 3. When the connection opens, call a procedure
sock.addEventListener("open", () => {
  send({ db: "myapp", host: "dbhost", proc: "pGetUsers()", params: "" });
});

// 4. Parse each incoming line as JSON
sock.addEventListener("message", (event) => {
  const lines = event.data.split("\n").filter(Boolean);
  lines.forEach((line) => {
    try {
      const record = JSON.parse(line);
      console.log("Received:", record);
      // e.g. add to a table, chart, or dashboard widget
    } catch (err) {
      console.warn("Non-JSON line:", line);
    }
  });
});

// 5. Handle errors and close
sock.addEventListener("error", (err) => console.error("WebSocket error:", err));
sock.addEventListener("close", (ev)  => console.log("Closed:", ev.code, ev.reason));

7. Message Fields Reference

Example — using params (positional string)

{"db":"myapp","host":"dbhost","proc":"pGetUsers(lLimit,lFilter)","params":"50,active"}

Example — using payload (named object)

{"db":"myapp","host":"dbhost","proc":"pInsertUser(lName,lEmail)","params":"","payload":{"lName":"Alice","lEmail":"alice@example.com"}}

8. Troubleshooting & Tips

Connection refuses to open

• Verify the URL uses wss:// on HTTPS pages (browsers block mixed content).
• Confirm the server is running and the endpoint path is correct (e.g. /api/ws-dba).
• Check that your network or firewall allows WebSocket traffic on the server port.

Messages are ignored or cause a parse error

• Make sure every message ends with \n.
• Validate your JSON: use jsonlint.com or your browser's DevTools console (JSON.parse('...')).
• Do not send arrays ([ ]) — send a single object per message.
• Ensure db, host, and proc match exactly what the server expects — procedure names are case-sensitive.
• The number of values in params must equal the number of parameter names declared in proc.

Data stops arriving mid-stream

• The connection may have timed out (ZIMX default idle timeout is 30 seconds). Send a keepalive or reconnect automatically.
• Check the Events pane in the Stream Manager or the browser's DevTools Network → WS tab for close codes.

Seeing duplicate or out-of-order records

• Ensure the procedure on the server applies a deterministic sort order.
• Track received records in your client code and deduplicate before rendering.

Quick sanity-check commands

// Fetch a single known record
{"db":"myapp","host":"dbhost","proc":"pGetUserById(lId)","params":"1"}

// Confirm a write round-trip
{"db":"myapp","host":"dbhost","proc":"pInsertTestRecord(lMsg)","params":"hello"}
{"db":"myapp","host":"dbhost","proc":"pGetTestRecords()","params":""}