Introduction to polyDB
This tutorial explains how to access the polyDB database from within polymake using the extension polyDB
. It comes bundled with polymake
, so there is no need to install extra software, except for the MongoDB.pm perl package. (This tutorial is for polymake version 4 and later. The old version is here, but this needs also an old version of the database). If you encounter any errors or problems concerning polyDB
or the extension, please don't hesitate to ask in the forum.
The polymake
extension is not necessary to use the data. You can access the data also
- via the web interface
- using the mongo shell directly or via any gui (see below for more details (to be written))
- using the polyDB REST API.
Software developers can also include access to polyDB using any of the many MongoDB interfaces and use the data directly in their programs. The few structural assumptions made in the database that you need to follow in your development are explained below (to be written).
Initializing Access
You need a database connection before you can work with polyDB
(and a working internet connection if you want to access a database that is not stored in your local computer). For the main instance of polyDB
you can just call
$polydb = polyDB();
If you have a personal account in the database (e.g. if you maintain a collection or work on a provate project), you should pass your username and password:
$polydb = polyDB(username=><username>, passwd=><password>);
You can also store this in the custom variables
$PolyDB::default::db_user
and
$PolyDB::default::db_pwd
… Then you don't have to specify them in the connection method, they will be picked up automatically and are preserved over sessions.
Learning which Data is Available
The data in polyDB is stored in collections, which can be organized in nested sections. Access to a certain collection requires the full path to it, i.e. both the (nested) section containing the collection and the collection name. E.g. the collection of smooth reflexive Fano polytopes up to dimension nine is in the collection SmoothReflexive
contained in the Lattice
subsection of the section Polytopes
. The section separator is a dot (.
), so you find the data in
- section:
Polytopes.Lattice
- collection:
SmoothReflexive
The info
method gives you a list of collections you can read, ordered by sections. If you have a private account in the database then this command will also list the additional collections you have been granted access to.
$polydb->info(); =============== available polydb collections =============== SECTION: Manifolds This database contains combinatorial manifolds COLLECTION: DIM2_3 This is a collection of combinatorial 2- and 3-manifolds with up to 10 vertices given as triangulations and calculcated by Frank Lutz found at: http://page.math.tu-berlin.de/~lutz/stellar/mixed.html. [...]
You can restrict the search by
- providing a chain of sections and subsections in the argument
section
. This can also just be an initial substring, so$polydb->info(section=>"Polytopes.La")
lists all collections within the section
Polytopes
whose first subsection starts withLa
- given a specific section, you can filter the collections by an initial substring, so
$polydb->info(section=>"Polytopes.Lattice", collection=>"Smoo")
lists all collections within the section
Polytopes.Lattice
whose name starts withSmoo
.
Reading Data
Before you can access a collection you need to establish a connection handle with the method get_collection
. For example, if you want to acess the list of 0/1-polytopes up to combinatorial equivalence you would call
$collection=$polydb->get_collection("Polytopes.Combinatorial.01Polytopes");
Now you can query this ist the various methods provided:
count
: to count the number of objects satisfying the provided queryfind_one
: to get one example satisfying the provided queryfind
: to obtain a cursor on objects satisfying the provided querydistinct
: to get an array of distinct values for a property among all objects satisfying the provided queryaggregate
: to apply complex aggregation pipelines on a collection
The main argument of the first three functions is a MongoDD query hash. You can use the full MongoDB query syntax as decribed here. Note that polymake
uses the perl interface interface of MongoDB, so the query should be given as a perl hash instead of a json document (it mostly suffices to use ⇒
instead of :
). For some perl examples see here. Basic queries for one or more parameter look like
{"N_VERTICES"=>10} {"DIM"=>5, "N_FACETS"=>7}
Bounds or ranges can be defined with the operators >
, >e
, <
and <e
. For example
{"N_VERTICES"=> { ">e" => 5, "<e" => 10 } }
returns documents where the number of vertices is between five and ten (including the boundaries). More operators can be found here. You can also query for elements in arrays either somewhere in the array or at a specific position.
The last function allows to pass an aggregation pipeline as described here (note again that the pipeline needs to be passed as a perl hash instead of a json document).
All queries need the name of the section and collection. You can either pass this via the options section
and collection
or, in particular if you query the same collection several times in a row, via the variables $PolyDB:“default::db_section_name
and $PolyDB:“default::db_collection_name
. If you set them via set_custom. this is even persistent over polymake
sessions. So a query could look like
$cur=$collection->find({"DIM"=>4}, section=>"Polytopes.Lattice", collection=>"SmoothReflexive");
or
$PolyDB::default::db_section="Polytopes.Lattice"; $PolyDB:"default::db_collection="SmoothReflexive"; $cur=$collection->find({"DIM"=>4});
find
returns a cursor over the list of results, count
just counts the number of objects matching your query. You can iterate over the result of find
via
$cur=$collection->find({"DIM"=>4}, section=>"Polytopes.Lattice", collection=>"SmoothReflexive"); while ( $cur->has_next() ) { $p=$cur->next(); print $p->N_VERTICES; }
There are further options to control the return:
skip⇒$n
: skips the first$n
documentslimit⇒$n
: returns at most$n
documents
You can reset the cursor with
$cur->reset
if you want to iterate over the results again.
Inserting new Data
Note that you need write access to write data into polyDB
. Also, initiating a new collection needs administrator permissions on the database. Please contact us (directly or via the forum) to obtain write acces to a (new) collection and have us do the basic initialization of a new collection.
Inserting data requires several steps:
- You need to prepare a json document with meta information about you collection in the following form
{ "_id" : "info.2.1", "description" : "0/1 Polytopes", "collection" : "01Polytopes", "section" : [ "Polytopes", "Combinatorial" ], "creator" : "Oswin Aichholzer", "contributor" : "Andreas Paffenholz", "maintainer" : "Andreas Paffenholz", "polydb_version" : "2.1", "packages" : { "polymake" : { "version" : "3.4", "type" : "polytope::Polytope<Rational>" } }, "uri" : "https://polymake.org" }
- You need to provide a full json schema describing your data. If you have a polymake object with the data you want, then the function
create_restrictive_schema
can help you with this and provide an initial template. Here is part of the schema for 0/1-Polytopes.{ "type": "object", "$schema": "http://json-schema.org/draft-07/schema#", "properties": { "SELF_DUAL": { "$ref": "#/definitions/common-Bool" }, "VERTICES": { "$ref": "#/definitions/common-Matrix-Rational-NonSymmetric" }, "_ns": { "additionalProperties": false, "properties": { "polymake": { "type": "array", "additionalItems": false, "items": [ { "const": "https://polymake.org" }, { "const": "3.5" } ] } }, "type": "object" } }, "additionalProperties": false, "required": [ "_ns", "SELF_DUAL", "VERTICES" ], "definitions": { "common-Rational": { "pattern": "^-?(\\d+(/\\d+)?|inf)$", "type": "string" }, "common-Matrix-Rational-NonSymmetric": { "type": "array", "items": { "oneOf": [ { "$ref": "#/definitions/common-Vector-Rational" }, { "type": "object", "properties": { "cols": { "type": "integer", "minimum": 0 } }, "required": [ "cols" ], "additionalProperties": false } ] } }, "common-Bool": { "type": "boolean" }, "common-Vector-Rational": { "items": { "$ref": "#/definitions/common-Rational" }, "type": "array" } } }
This schema needs one special property
_polyDB
, which specifies some information on the document. Among the properties you should have"_polyDB": { "required": [ "collection", "creation_date", "section", "uri", "version" ], "type": "object", "properties": { "uri": { "type": "string" }, "collection": { "type": "string" }, "version": { "type": "string", "pattern": "^[0-9]{1,2}.[0-9]{1,2}$" }, "creation_date": { "pattern": "^[1-9][0-9]{3}-[0-9]{2}-[0-9]{2}$", "type": "string" }, "section": { "type": "string" } } }
and should be a required property with an entry also in the properties listed in
_attr
as"_attrs": { "additionalProperties": false, "properties": { "_polyDB": { "properties": { "attachment": { "const": true } } } }, "type": "object" }
- If you want you collection to be included in the
db_info
command you need a json document describing you collection in the form{ "collection" : "TOM", "section" : [ "DocTropical" ], "maintainer" : "Andreas Paffenholz", "contributor" : "Silke Horn", "author" : "Silke Horn", "polydb_version" : "2.1", "description" : "All known non-realisable tropical oriented matroids with parameters n=6, d=3 or n=d=4. They were computed using polymake with the tropmat extension and Topcom. You need the extension tropmat for this.", "short_description" : "All known non-realisable tropical oriented matroids with parameters n=6, d=3 or n=d=4.", "webpage" : [ { "description" : "polymake extension tropmat", "address" : "http://solros.de/polymake/tropmat" } ] }
- If you want to place this also in a new section, then also this (and all new subsections created) need a description document. However, a description for a section can only be edited by an administrator, so just send us the description and we will add it.
Meta information, schema and documentation are stored with the methods
$collection->set_info($meta); $collection->set_schema($schema); $collection->set_collection_doc($doc, replace=>true/false, update=>true/false);
where the first argument is the data as a perl hash.
Insertion of data is done with the method insert
. This function either takes a file, a single polymake
big object or an array of such as first argument and writes this data into the collection specified by the options section
and collection
(these can also be specified with the same custom variables as for queries). This has some more options, see
help "insert";
in the polymake shell.
Administrative Tasks (only for polyDB admins)
Starting a new collection
A new collection is started with the command
$polydb->initiate_collection(section=><section>, collection=><collection>);
If the collection should not be public, then also pas the option public⇒false
. For a public collection the read access role of the new collection is added to the default role polymakeUser
which is granted to every user of polyDB
. One can add this later if one wants to build up and test the collection befor making it publicly available.
If this creates new intermediate subsections you should set the section documentation with
$polydb->set_section_doc($doc, section=>...);
so that the new collection appears in the list printed by db_info
for all users with sufficient permissions.
Note that the first command essentially only creates two new roles in MongoDB, one for read access to the collection (and all sections up to the root) and one for write access to the collection (and only to the collection, not to the sections). The actual collections are only created once the first document is written into the collection. This implies that collections will not be listed with db_info
if any of the intermediate sections has no documentation, as then the collection where this is stored is not created.
You can add users for read acces with the method add_user_to_collection
. Note that this is only useful if the collection is not public. Any user that has the write access rule (which you can assign with $polydb→add_user_to_collection(user⇒…, collection⇒…, admin⇒true
) can insert, delete and modify documents in the collection, the meta information, the schema and to documentation of the collection. Note that in this method the collection must be given fully qualified, e.g. as Polytopes.Combinatorial.01Polytopes
.