Design principles and development direction

Collectra aims to make it easier for researchers and institutions to turn collection images and documents into structured, searchable data. The design grew from collection-specific workflows such as Hespi and Grapto: researchers should be able to reuse their components and adapt them to new collections.

These principles reflect the vision presented at the ARDC Summer School. Collectra is still in development, and many of the features in that vision are still to come. The goals below describe the intended direction, rather than a promise that every workflow is available now. There is no release date implied for the planned features.

Modular, reusable pipelines

Each task should perform a distinct operation, with artefacts connecting its inputs and outputs to other tasks. Researchers can combine object detection, classification, OCR, and LLM prompts to fit their collection, reuse an existing pipeline, or implement a new task in Python. Multiple LLM tasks can turn an image into a transcription and then into structured text, for example.

Today, pipelines are configured in YAML and run through the CLI or Python. A visual editor for connecting and configuring tasks is planned. The intended task range also includes landmark/pose detection, segmentation, and oriented bounding boxes; these are not documented as available built-in tasks. See Tasks for the current tasks and their commands.

Result bundles that preserve context

Extracted data should retain its relationship to the source material. A crop, its transcription, and a classification should remain connected so that a researcher can inspect the evidence behind a result.

Collectra currently saves artefacts and their parent relationships in result directories containing results.yaml and associated files. Predictions and training annotations use the same result format. This allows reviewed results to become training examples when they have the annotations required by the chosen task.

RO-Crate-compatible packaging is a design goal. The current Collectra result format should not be assumed to be a complete RO-Crate. See Artefact Types for the current artefact types and commands.

Human review as part of the process

AI predictions need the judgement of researchers and curators. Reviewing and correcting results is part of the intended workflow, before those results are used in research, added to a database, or reused for training.

The current GUI supports inspecting results and editing annotations, including bounding boxes and text. The broader goal is a connected workflow from prediction through verification to export into collection databases; general database export is planned work.

An iterative path to better models

Researchers should be able to start with pretrained models, predict on their own material, correct the results, and fine-tune on those reviewed examples. Keeping predictions and annotations in the same format reduces the conversion work between these stages. Held-out validation data is still needed to assess whether retraining improves the model.

Supported detection and classification tasks currently expose CLI train commands. Training a selected task directly from the GUI is planned. See the Quickstart for preparing annotations and training through the CLI.

Exploration that guides annotation

Clustering should help researchers discover groups of similar items, such as specimen labels with the same layout or recurring depictions in artwork. Embeddings can also guide which examples to review first: the proposed furthest-first traversal would prioritise diverse examples, helping annotators cover variation in the collection with less repeated effort.

Interactive clustering, visual exploration of those groups, and diversity-based annotation ordering are planned workflows. They are not currently exposed as the GUI’s proposed Cluster action. Existing internal similarity utilities do not constitute this complete workflow.

Working with Collectra today

Start with the Quickstart, choose from the available Tasks, and use the GUI to inspect saved results. The task and artefact command pages describe current interfaces; the design goals above explain where the project is heading.