Database

ARC Metadata Registry application leverages the abstract ISA-JSON format to maintain the experimental metadata. Also, the application maintains the metadata related to the ARCs from individual DataHUBs. The backend database contains three important document collections

ARC Metadata Collection

Each document within ARC Metadata collection contains the file-level information of each ARCs. Each file-level change to the ARCs in the DataHUB is maintained as separate ARC version to present the evolution of the individual ARCs by keeping track of the history of the ARC updates.

{
            "arcID": {
                "type": "keyword"
            },
            "arcVersion": {
                "type": "integer"
            },
            "arcCreationDate": {
                "type": "date"
            },
            "arcLastModifiedDate": {
                "type": "date"
            },
            "investigationFilename": {
                "type": "keyword"
            },
            "arcSize": {
                "type": "long"
            },
            "gitLabUserID": {
                "type": "long"
            },
            "gitLabUserName": {
                "type": "text"
            },
            "gitLabUserUserName": {
                "type": "keyword"
            },
            "gitLabUserEmail": {
                "type": "keyword"
            },
            "gitLabRepoVisibility": {
                "type": "keyword"
            },
            "gitLabHost":{
                "type": "keyword"
            },
            "gitLabHostURL":{
                "type": "text"
            },
            "gitLabHostLocation":{
                "type": "text"
            },
            "gitLabHostVersion":{
                "type": "text"
            },
            "investigations": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            },
            "assays": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            },
            "studies": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            },
            "workflows": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            },
            "externals": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            },
            "runs": {
                "properties": {
                    "file": {
                        "type": "keyword"
                    },
                    "status": {
                        "type": "keyword"
                    },
                    "creationDate": {
                        "type": "date"
                    },
                    "lastModifiedDate": {
                        "type": "date"
                    },
                    "lastModifiedBy": {
                        "type": "text"
                    }
                }
            }
        }

Parameter-Values Collection

This collection maintains each parameter-value from both the ARC and experimental metadata. The collection maintains the characteristics, components, factors, study and assay parameters values from the ISA data of individual ARCs.

{
    "arcID": {
        "type": "keyword"
    },
    "arcVersion": {
        "type": "integer"
    },
    "investigationFilename": {
        "type": "keyword"
    },
    "gitLabHost": {
        "type": "keyword"
    },
    "gitLabHostLocation": {
        "type": "text"
    },
    "studyIdentifier": {
        "type": "keyword"
    },
    "assayIdentifier": {
        "type": "keyword"
    },
    "protocolName": {
        "type": "keyword"
    },
    "processSequenceName": {
        "type": "keyword"
    },
    "derivesFrom": {
        "type": "keyword"
    },
    "isaPath": {
        "type": "text"
    },
    "valuesType": {
        "type": "keyword"
    },
    "name": {
        "type": "text"
    },
    "nameTermSource": {
        "type": "keyword"
    },
    "nameTermAccession": {
        "type": "text"
    },
    "nameTermAnnotationValue": {
        "type": "text"
    },
    "value": {
        "type": "text"
    },
    "valueTermSource": {
        "type": "keyword"
    },
    "valueTermAccession": {
        "type": "text"
    },
    "valueTermAnnotationValue": {
        "type": "text"
    },
    "valueUnit": {
        "type": "text"
    },
    "valueUnitTermSource": {
        "type": "keyword"
    },
    "valueUnitTermAccession": {
        "type": "text"
    },
    "valueUnitTermAnnotationValue": {
        "type": "text"
    }
}

Investigation Data Collection

Note

In the ISA Metadata Framework specification, there are instances which are not assigned to a specific data type, instead, a complex type with “anyOf” marker. In order to maintain these fields in the database collections, they are translated and stored by a field with the same name and type as suffix.

investigation_schema.json

This schema implements Investigation from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA investigation schema",
	"description": "JSON-schema representing an investigation in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"filename": {
			"type": "string"
		},
		"identifier": {
			"type": "string"
		},
		"title": {
			"type": "string"
		},
		"description": {
			"type": "string"
		},
		"submissionDate": {
			"type": "string",
			"format": "date-time"
		},
		"publicReleaseDate": {
			"type": "string",
			"format": "date-time"
		},
		"ontologySourceReferences": {
			"type": "array",
			"items": {
				"$ref": "ontology_source_reference_schema.json#"
			}
		},
		"publications": {
			"type": "array",
			"items": {
				"$ref": "publication_schema.json#"
			}
		},
		"people": {
			"type": "array",
			"items": {
				"$ref": "person_schema.json#"
			}
		},
		"studies": {
			"type": "array",
			"items": {
				"$ref": "study_schema.json#"
			}
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

study_schema.json

This schema implements Study from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"title": "Study JSON Schema",
	"description": "JSON Schema describing an Study",
	"@context": {
		"@base": "http://purl.org/isaterms/",
		"xsd": "http://www.w3.org/2001/XMLSchema#"
	},
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"filename": {
			"type": "string"
		},
		"identifier": {
			"type": "string"
		},
		"title": {
			"type": "string"
		},
		"description": {
			"type": "string"
		},
		"submissionDate": {
			"type": "string",
			"format": "date-time"
		},
		"publicReleaseDate": {
			"type": "string",
			"format": "date-time"
		},
		"publications": {
			"type": "array",
			"items": {
				"$ref": "publication_schema.json#"
			}
		},
		"people": {
			"type": "array",
			"items": {
				"$ref": "person_schema.json#"
			}
		},
		"studyDesignDescriptors": {
			"type": "array",
			"items": {
				"$ref": "ontology_annotation_schema.json#"
			}
		},
		"protocols": {
			"type": "array",
			"items": {
				"$ref": "protocol_schema.json#"
			}
		},
		"materials": {
			"type": "object",
			"properties": {
				"sources": {
					"type": "array",
					"items": {
						"$ref": "source_schema.json#"
					}
				},
				"samples": {
					"type": "array",
					"items": {
						"$ref": "sample_schema.json#"
					}
				},
				"otherMaterials": {
					"type": "array",
					"items": {
						"$ref": "material_schema.json#"
					}
				}
			}
		},
		"processSequence": {
			"type": "array",
			"items": {
				"$ref": "process_schema.json#"
			}
		},
		"assays": {
			"type": "array",
			"items": {
				"$ref": "assay_schema.json#"
			}
		},
		"factors": {
			"type": "array",
			"items": {
				"$ref": "factor_schema.json#"
			}
		},
		"characteristicCategories": {
			"description": "List of all the characteristics categories (or material attributes) defined in the study, used to avoid duplication of their declaration when each material_attribute_value is created. ",
			"type": "array",
			"items": {
				"$ref": "material_attribute_schema.json#"
			}
		},
		"unitCategories": {
			"description": "List of all the unitsdefined in the study, used to avoid duplication of their declaration when each value is created. ",
			"type": "array",
			"items": {
				"$ref": "ontology_annotation_schema.json#"
			}
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

assay_schema.json

This schema implements Assay from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"title": "Assay JSON Schema",
	"name": "Assay JSON Schema",
	"description": "JSON Schema describing an Assay",
	"@context": {
		"@base": "http://purl.org/isaterms/",
		"xsd": "http://www.w3.org/2001/XMLSchema#"
	},
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		},
		"filename": {
			"type": "string"
		},
		"measurementType": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"technologyType": {
			"type": "object",
			"properties": {
				"ontologyAnnotation": {
					"$ref": "ontology_annotation_schema.json#"
				}
			}
		},
		"technologyPlatform": {
			"type": "string"
		},
		"dataFiles": {
			"type": "array",
			"items": {
				"$ref": "data_schema.json#"
			}
		},
		"materials": {
			"type": "object",
			"properties": {
				"samples": {
					"type": "array",
					"items": {
						"$ref": "sample_schema.json#"
					}
				},
				"otherMaterials": {
					"type": "array",
					"items": {
						"$ref": "material_schema.json#"
					}
				}
			}
		},
		"characteristicCategories": {
			"description": "List of all the characteristics categories (or material attributes) defined in the study, used to avoid duplication of their declaration when each material_attribute_value is created. ",
			"type": "array",
			"items": {
				"$ref": "material_attribute_schema.json#"
			}
		},
		"unitCategories": {
			"description": "List of all the unitsdefined in the study, used to avoid duplication of their declaration when each value is created. ",
			"type": "array",
			"items": {
				"$ref": "ontology_annotation_schema.json#"
			}
		},
		"processSequence": {
			"type": "array",
			"items": {
				"$ref": "process_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

comment_schema.json

This schema implements the ability to annotate objects with user-defined comments.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA comment schema - it corresponds to ISA Comment[] construct",
	"description": "JSON-schema representing a comment in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"value": {
			"type": "string"
		}
	},
	"additionalProperties": false
}

data_schema.json

This schema implements Data from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA data schema",
	"description": "JSON-schema representing a data file in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"type": {
			"type": "string",
			"enum": [
				"Raw Data File",
				"Derived Data File",
				"Image File"
			]
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

factor_schema.json

This schema implements Study factor from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA factor schema",
	"name": "ISA factor schema",
	"description": "JSON-schema representing a factor value in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"factorName": {
			"type": "string"
		},
		"factorType": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

factor_value_schema.json

This schema implements Factor value given to a node corresponding to a declared Factor.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA factor value schema",
	"description": "JSON-schema representing a factor value in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"category": {
			"$ref": "factor_schema.json#"
		},
		"value_string": {
			"type": "string"
		},
		"value_number": {
			"type": "number"
		},
		"value_ontologyAnnotation": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"unit": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

material_attribute_schema.json

This schema is used in a Material node to declare an attribute (Characteristic).

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA material attribute schema",
	"description": "JSON-schema representing a characteristics category (what appears between the brackets in Charactersitics[]) in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"characteristicType": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

material_attribute_value_schema.json

This schema is used in a Material node to hold an attribute value (value of a Characteristic).

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA material attribute schema",
	"description": "JSON-schema representing a material attribute (or characteristic) value in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"category": {
			"$ref": "material_attribute_schema.json#"
		},
		"value_string": {
			"type": "string"
		},
		"value_number": {
			"type": "number"
		},
		"value_ontologyAnnotation": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"unit": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

material_schema.json

This schema implements Material nodes from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA material node schema",
	"description": "JSON-schema representing a material node in the ISA model, which is not a source or a sample (as they have specific schemas) - this will correspond to 'Extract Name', 'Labeled Extract Name'",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"type": {
			"type": "string",
			"enum": [
				"Extract Name",
				"Labeled Extract Name"
			]
		},
		"characteristics": {
			"type": "array",
			"items": {
				"$ref": "material_attribute_value_schema.json#"
			}
		},
		"derivesFrom": {
			"type": "array",
			"items": {
				"$ref": "material_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

ontology_annotation_schema.json

This schema implements Ontology from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA ontology reference schema",
	"name": "ISA ontology reference schema",
	"description": "JSON-schema representing an ontology reference or annotation in the ISA model (for fields that are required to be ontology annotations)",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"annotationValue_string": {
			"type": "string"
		},
		"annotationValue_number": {
			"type": "number"
		},
		"termSource": {
			"type": "string",
			"description": "The abbreviated ontology name. It should correspond to one of the sources as specified in the ontologySourceReference section of the Investigation."
		},
		"termAccession": {
			"type": "string",
			"format": "uri"
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

ontology_source_reference_schema.json

This schema implements Ontology from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA ontology source reference schema",
	"name": "ISA ontology source reference schema",
	"description": "JSON-schema representing an ontology reference in the ISA model",
	"type": "object",
	"properties": {
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		},
		"description": {
			"type": "string"
		},
		"file": {
			"type": "string"
		},
		"name": {
			"type": "string"
		},
		"version": {
			"type": "string"
		}
	},
	"additionalProperties": false
}

person_schema.json

This schema implements Contact from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA person schema",
	"description": "JSON-schema representing a person in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"lastName": {
			"type": "string"
		},
		"firstName": {
			"type": "string"
		},
		"midInitials": {
			"type": "string"
		},
		"email": {
			"type": "string",
			"format": "email"
		},
		"phone": {
			"type": "string"
		},
		"fax": {
			"type": "string"
		},
		"address": {
			"type": "string"
		},
		"affiliation": {
			"type": "string"
		},
		"roles": {
			"type": "array",
			"items": {
				"$ref": "ontology_annotation_schema.json#"
			}
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

process_parameter_value_schema.json

This schema is used in a Process node to hold a parameter value (value of a Protocol parameter).

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA process parameter value schema",
	"description": "JSON-schema representing a Parameter Value (associated with a Protocol REF) in the ISA model",
	"type": "object",
	"properties": {
		"category": {
			"$ref": "protocol_parameter_schema.json#"
		},
		"value_string": {
			"type": "string"
		},
		"value_number": {
			"type": "number"
		},
		"value_ontologyAnnotation": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"unit": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

process_schema.json

This schema implements Process nodes from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA process or protocol application schema, corresponds to 'Protocol REF' columns in the study and assay files",
	"description": "JSON-schema representing a protocol application in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"executesProtocol": {
			"$ref": "protocol_schema.json#"
		},
		"parameterValues": {
			"type": "array",
			"items": {
				"$ref": "process_parameter_value_schema.json#"
			}
		},
		"performer": {
			"type": "string"
		},
		"date": {
			"type": "string",
			"format": "date-time"
		},
		"previousProcess": {
			"$ref": "process_schema.json#"
		},
		"nextProcess": {
			"$ref": "process_schema.json#"
		},
		"inputs_sample": {
			"type": "array",
			"items": {
				"$ref": "sample_schema.json#"
			}
		},
		"inputs_source": {
			"type": "array",
			"items": {
				"$ref": "source_schema.json#"
			}
		},
		"inputs_data": {
			"type": "array",
			"items": {
				"$ref": "data_schema.json#"
			}
		},
		"inputs_material": {
			"type": "array",
			"items": {
				"$ref": "material_schema.json#"
			}
		},
		"outputs_sample": {
			"type": "array",
			"items": {
				"$ref": "sample_schema.json#"
			}
		},
		"outputs_data": {
			"type": "array",
			"items": {
				"$ref": "data_schema.json#"
			}
		},
		"outputs_material": {
			"type": "array",
			"items": {
				"$ref": "material_schema.json#"
			}
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

protocol_parameter_schema.json

This schema is used in a Protocol to describe a protocol parameter.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA protocol parameter schema",
	"description": "JSON-schema representing a parameter for a protocol (category declared in the investigation file) in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"parameterName": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

protocol_schema.json

This schema implements Protocol from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA protocol schema",
	"name": "ISA protocol schema",
	"description": "JSON-schema representing a protocol in the ISA model",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		},
		"name": {
			"type": "string"
		},
		"protocolType": {
			"$ref": "ontology_annotation_schema.json#"
		},
		"description": {
			"type": "string"
		},
		"uri": {
			"type": "string",
			"format": "uri"
		},
		"version": {
			"type": "string"
		},
		"parameters": {
			"type": "array",
			"items": {
				"$ref": "protocol_parameter_schema.json#"
			}
		},
		"components": {
			"type": "array",
			"items": {
				"type": "object",
				"properties": {
					"componentName": {
						"type": "string"
					},
					"componentType": {
						"$ref": "ontology_annotation_schema.json#"
					}
				}
			}
		}
	},
	"additionalProperties": false
}

publication_schema.json

This schema implements Publication from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA investigation schema",
	"name": "ISA investigation schema",
	"description": "JSON-schema representing an investigation in the ISA model",
	"type": "object",
	"properties": {
		"comments": {
			"type": "array",
			"items": {
				"$ref": "comment_schema.json#"
			}
		},
		"pubMedID": {
			"type": "string"
		},
		"doi": {
			"type": "string"
		},
		"authorList": {
			"type": "string"
		},
		"title": {
			"type": "string"
		},
		"status": {
			"$ref": "ontology_annotation_schema.json#"
		}
	},
	"additionalProperties": false
}

sample_schema.json

This schema implements Sample from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA sample schema",
	"description": "JSON-schema representing a sample in the ISA model. A sample represents a major output resulting from a protocol application other than the special case outputs of Extract or a Labeled Extract.",
	"type": "object",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"characteristics": {
			"type": "array",
			"items": {
				"$ref": "material_attribute_value_schema.json#"
			}
		},
		"factorValues": {
			"type": "array",
			"items": {
				"$ref": "factor_value_schema.json#"
			}
		},
		"derivesFrom": {
			"type": "array",
			"items": {
				"$ref": "source_schema.json#"
			}
		}
	},
	"additionalProperties": false
}

source_schema.json

This schema implements Source from the ISA Abstract Model.

{
	"$schema": "http://json-schema.org/draft-04/schema",
	"title": "ISA source schema",
	"description": "JSON-schema representing a source in the ISA model. Sources are considered as the starting biological material used in a study.",
	"properties": {
		"@id": {
			"type": "string",
			"format": "uri"
		},
		"name": {
			"type": "string"
		},
		"characteristics": {
			"type": "array",
			"items": {
				"$ref": "material_attribute_value_schema.json#"
			}
		}
	},
	"additionalProperties": false
}