This is an old revision of the document!
Data Format in the polyDB
MongoDB Structure
MongoDB stores data (documents in MongoDB terms) in collections organized in databases (so the structure has two levels). There is one special database admin
for user information, permissions and passwords. Some query operations can span collections in the same database, but usually not collections across databases.
Therefore polyDB
uses just two databases inside a MongoDB instance: Data is stored in collections inside the database polyDB
. Users and permissions are stored in admin
.
polyDB Structure
In the MongoDB instance the polyDB
uses just one database and organizes everything in collections within this database.
In polyDB
objects are stored in polyDB collections, and these collections can be organized in (nested) sections. For each data collection there is an accompanying collection containing meta information on the data. The name of the MongoDB collection is composed from the collection and the chain of sections the collection is in. A family of objects collection organized in the subsection sub of section section is stored in the MongoDB collection section.sub.collection
. The meta information is in _collectionInfo.section.sub.collection
.
For example, the family of smooth reflexive polytopes is in the collection SmoothReflexive
in the subsection Lattice
of the section Polytopes
. In terms of MongoDB, the dtaa is stored in the in the MongoDB collection Polytopes.Lattice.SmoothReflexive
of the database polyDB
, and the meta information is in _collectionInfo.Polytopes.Lattice.SmoothReflexive
in the same database.
Any documentation for the sections and collections are in the collections _collectionInformation.section.sub.collection
and _sectionInfo.section
, _sectionInfo.section.sub
etc.
Data in the data collection is described by two documents in the meta collection:
- a document for meta information. This document should have the following entries
description
: A short description of the data, only for internal use, this will not be displayed.maintainer
: Maintainer of the data in polyDB, will be added to documents in the collectioncreator
: Name of person creating the data, will be added to documents in the collectioncontributor
: Name of the person who prepared the data for inclusion onto polyDB, will be added to documents in the collectionfields
: A list of data fields contained in a document in the collection. This can be used to produce a list of data that can be queried from the collection.polydb_version
: the version of polyDB used to store the datapackages
: Here software packages can store additional information they need to access the data
- a json schema: A json schema that completely describes the data. Each document in the collection should verify against this schema. MongoDB comes with its own internal schema verification methods. However, in
polyDB
we do not use this as it is based on an old draft of the schema language and has modifications from the standard. The schema is stored in a schema document with the three entriessection
: the section of the collection the schema applies tocollection
: the collection the schema applies toschema
: The actual json schema. Both json schemas and MongoDB use$
as a special character. We need to replace this in the schema for stroring as this would lead to conflicts otherwise. InpolyDB
we use__
(two underscores) for this.
The MongoDB _id
of the info document is info.<polydb version>
and of the schema document schema.<polydb version>
, where <polydb version.
is the polydb version number for which the two documents apply.
Example of a meta information document