census serve

census serve answers requests on stdin, one JSON object per line, and writes responses and progress to stdout the same way. The Disk Census app runs it, and any program can. Scans it has read stay in memory, so queries answer in milliseconds.

printf '%s\n' '{"id":1,"args":["version"]}' | census serve

That prints one line: the version response, with "id":1.

Requests

A request is one line of JSON:

{"id":3,"args":["ls","/Users"]}

• args are the command-line arguments. --json is always added.
• id is an integer from 0 to 2^53 − 1, unique among the requests still in flight.

Responses

A response is the command’s envelope, as census … --json prints it, on one line, plus "id". With --progress in args, the request also gets progress records on stdout, each with its "id". Responses can arrive in a different order from the requests, so match them by id.

Cancel a request

{"id":4,"cancel":3}

Request 3 ends with status: cancelled. Request 4 gets status: ok and command: "cancel", even when request 3 had already finished. A cancel stops a cleanup between files; what was deleted so far is in the audit log the response names.

Long jobs

scan, duplicates, export, clean, review, apply, uninstall, maintain with tasks, optimize, empty-trash --yes, and any request with --refresh are long jobs. They run one at a time; another long job meanwhile gets busy (exit code 2). Queries still answer while one runs, from the last complete scan. history answers from the usage history and the disk’s live space.

Cleanup in serve

serve adds --json, so there is no terminal to ask in:

• clean needs --safe;
• uninstall, maintain with tasks, optimize and empty-trash need --yes;
• nothing asks.

Get the person’s approval in your own interface first, then send the request with the flag. maintain without tasks and empty-trash without --yes are queries. After a deletion, queries keep reading the scan taken before it until the next scan.

Errors and shutdown

• selftest, help, dump-tree and serve are refused with invalid_request.
• A line that isn’t a valid request, or is longer than 16 MiB, gets invalid_request, with "id": null unless the line had a valid id.
• When stdin ends, the engine answers every request it accepted, then exits 0. A running job is cancelled and answers within 2 seconds.
• On SIGTERM or SIGINT it cancels the running job, writes that response, and exits. Queued queries may go unanswered, but no line is ever cut off.

A client in Python

import json
import subprocess

engine = subprocess.Popen(
    ["census", "serve"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    text=True,
)

def call(request_id, *args):
    """Send one request and return its response, skipping progress records."""
    engine.stdin.write(json.dumps({"id": request_id, "args": list(args)}) + "\n")
    engine.stdin.flush()
    while True:
        message = json.loads(engine.stdout.readline())
        if message.get("id") == request_id and message.get("event") != "progress":
            return message

print(call(1, "version")["results"])  # {'version': '0.1.0', 'schema_version': 5}

top = call(2, "top", "--limit", "5")
for row in top["results"]:
    print(row["allocated_bytes"], row["path"]["display"])

engine.stdin.close()  # the engine answers what it accepted, then exits
engine.wait()

This client sends one request at a time. To keep several in flight, read stdout on its own thread and hand each message to the request with its id.