The Export Format
As of now, the only supported format for exporting data out of the system is JSON. Below is a detailed description of the JSON format . It outlines the structure, key fields, and example data to help users understand and utilize the exported data effectively.
JSON Structure
The JSON export contains a hierarchical structure representing the data. Below is a high-level overview of the structure:
{
"info": { ... },
"media_items": [ ... ]
}
info: Contains information about the export process and datasetmedia_items: An array of objects, each representing a record of a single Image or Video Frame
Info
The info section includes details about the export, such as the export date, version, and other relevant information.
Example:
{
"schema_version": "1.1",
"dataset": "ImageNet 1K Enriched",
"dataset_url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data?page=1",
"dataset_creation_time": "2023-01-01 12:34:56",
"export_time": "2024-06-08 12:34:56",
"description": "Export from Dataset34 at Visual Layer",
"exported_by": "Elad Yaakov",
"total_media_items": 3
}
schema_version: The version of the export format schemadataset: The Dataset from which data was exporteddataset_url: The link to the Dataset from which data was exporteddataset_creation_time: The date and time of the Dataset's creationexport_time: The date and time when the data was exporteddescription: A general description of this export fileexported_by: The person who performed this export operationtotal_media_items: The total number of records in the export
Media Items
The media_items section contains an array of media items records (Images/Video Frames). Each record is an object with fields representing different attributes of the media item.
Example:
{
"media_items": [
{
"media_id": "7dd03beb-961a-43bd-b68a-585b6abc37de",
"media_type": "image",
"file_name": "some-image.jpg",
"file_path": "imagenet1k/some-image.jpg",
"file_size": "3.38MB",
"height": 333,
"width": 500,
"url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data/image/520f7405-9af3-4781-8423-c4092fd40ef5",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326a21",
"metadata_items": []
},
{
"media_id": "b570353c-1216-4b73-9a83-57cb4791503e",
"media_type": "video_frame",
"file_name": "some-video-frame1.jpg",
"file_path": "imagenet1k/some-video-frame.jpg",
"file_size": "3.95MB",
"height": 480,
"width": 640,
"url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data/image/b570353c-1216-4b73-9a83-57cb4791503e",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326a21",
"metadata_items": []
}
// Additional records...
]
}
media_id: A unique identifier for the media item recordmedia_type: The type of media item (Image / Video Frame)file_name: The full name of the media item filefile_path: A full path of the media item filefile_size: The file system size of media item fileheight: The height of the media item in pixelswidth: The width of the media item in pixelsurl: The link to the media itemcluster_id: A unique identifier for the cluster containing this media item
Metadata Items
Each Media Item may have many metadata properties. These properties are represented in each Media Item object under a metadata_items section that contains an array of metadata item records. Each Metadata Item has a type and a set of properties.
Example:
{
"metadata_items": [
{
"metadata_type": "issue",
"properties": {
"issue_type": "duplicate",
"confidence": "0.88",
"duplicate_group_id": 43
}
},
{
"metadata_type": "user_tag",
"properties": {
"tag_name": "reviewed",
"assigned_date": "2023-01-01 12:34:56",
"assigned_by": "Elad Yaakov"
}
},
{
"metadata_type": "image_label",
"properties": {
"category_id": 34,
"category_name": "sky",
"source": "user",
}
}
]
}
metadata_type: The metadata Item type (e.g. Issue / Image Label / Object Label / User Tag / etc.)properties: A set of properties associated with this Metadata Item (the attributes of the metadata item could differ from one another depending on the metadata type)
Objects
Objects are special entities in the Visual Layer platform, and this is well represented in the export JSON format.
One hand they are considered metadata items of Media Items, and can be found under the metadata_items section with the metadata_type set to "object_label".
However, on the other hand they may also have a Metadata Items section of their own, containing various metadata items such as Issues and User Tags associated with this particular Object instance.
Example:
{
"metadata_type": "object_label",
"properties": {
"category_id": 36,
"category_name": "tree",
"bbox": [50, 50, 150, 150],
"url": "https://app.visual-layer.com/dataset/f246c942-2bcb-11ef-9e12-4e52fc95a50e/data/image/8087f287-0c61-496e-a0a2-1021b3a330ae?t=%5B%22Objects%22%5D&object_id=307c463d-d502-4b88-9b89-117e3e470edc",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326678",
"metadata_items": [
{
"metadata_type": "issue",
"properties": {
"issue_type": "mislabel",
"confidence": "0.88",
"actual_label": "tree",
"suggested_label": "building",
"suggestion_confidence": "0.78"
}
},
{
"metadata_type": "user_tag",
"properties": {
"tag_name": "accepted",
"assigned_date": "2023-01-01 12:34:56",
"assigned_by": "Elad Yaakov"
}
}
]
}
}
Example JSON
Below is a complete example of an exported JSON file:
{
"info": {
"schema_version": "1.1",
"dataset": "ImageNet 1K Enriched",
"dataset_url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data?page=1",
"dataset_creation_time": "2023-01-01 12:34:56",
"export_time": "2024-06-08 12:34:56",
"description": "Export from Dataset34 at Visual Layer",
"exported_by": "Elad Yaakov",
"total_media_items": 3
},
"media_items": [
{
"media_id": "e234801a-b02b-4ad7-96db-3a6c7ef7e1cf",
"media_type": "image",
"file_name": "some-image1.jpg",
"file_path": "imagenet1k/some-image1.jpg",
"file_size": "7.35MB",
"height": 480,
"width": 640,
"url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data/image/520f7405-9af3-4781-8423-c4092fd40ef5",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326a21",
"metadata_items": [
{
"metadata_type": "issue",
"properties": {
"issue_type": "duplicate",
"confidence": "0.88",
"similarity_level": "0.98",
"duplicate_group_id": 43
}
},
{
"metadata_type": "user_tag",
"properties": {
"tag_name": "reviewed",
"assigned_date": "2023-01-01 12:34:56",
"assigned_by": "Elad Yaakov"
}
}
]
},
{
"media_id": "3a4c5d5b-f6d1-4539-ab69-0df3094c715d",
"media_type": "image",
"file_name": "some-image2.jpg",
"file_path": "imagenet1k/some-image2.jpg",
"file_size": "5.56MB",
"height": 480,
"width": 640,
"url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data/image/520f7405-9af3-4781-8423-c4092fd40ef5",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326a21",
"metadata_items": [
{
"metadata_type": "issue",
"properties": {
"issue_type": "duplicate",
"confidence": "0.88",
"duplicate_group_id": 43
}
},
{
"metadata_type": "image_label",
"properties": {
"category_id": 34,
"category_name": "sky",
"source": "user",
}
},
{
"metadata_type": "object_label",
"properties": {
"category_id": 36,
"category_name": "tree",
"bbox": [50, 50, 150, 150],
"url": "https://app.visual-layer.com/dataset/f246c942-2bcb-11ef-9e12-4e52fc95a50e/data/image/8087f287-0c61-496e-a0a2-1021b3a330ae?t=%5B%22Objects%22%5D&object_id=307c463d-d502-4b88-9b89-117e3e470edc",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326678",
"metadata_items": [
{
"metadata_type": "issue",
"properties": {
"issue_type": "mislabel",
"confidence": "0.88",
"actual_label": "tree",
"suggested_label": "building",
"suggestion_confidence": "0.78"
}
},
{
"metadata_type": "user_tag",
"properties": {
"tag_name": "accepted",
"assigned_date": "2023-01-01 12:34:56",
"assigned_by": "Elad Yaakov"
}
}
]
}
}
]
},
{
"media_id": "b570353c-1216-4b73-9a83-57cb4791503e",
"media_type": "video_frame",
"file_name": "some-video-frame1.jpg",
"file_path": "imagenet1k/some-image1.jpg",
"file_size": "3.95MB",
"height": 480,
"width": 640,
"url": "https://app.visual-layer.com/dataset/2056644a-31fe-11ef-a543-ce15e99033a1/data/image/520f7405-9af3-4781-8423-c4092fd40ef5",
"cluster_id": "e3e0e019-79a3-4f4e-b19c-7eedc7326678",
"metadata_items": [
{
"type": "video_info",
"properties": {
"video_name": "some-video.mov",
"frame_timestamp": "01:42:00"
}
},
{
"metadata_type": "image_label",
"properties": {
"category_id": 33,
"category_name": "parking lot",
"source": "vl_enrichment",
}
},
]
}
]
}
If you have any questions or need further assistance, please contact our support team at [email protected].
Updated 2 months ago