Introduction to polyDB [polymake wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Introduction to polyDB ======

The polyDB extension provides access to the [[:polydb|polyDB database]]. It comes bundled with ''polymake'', so there is no need to install extra software, except for the MongoDB.pm perl package. If you encounter any errors or problems concerning polyDB, please don't hesitate to [[https://forum.polymake.org/|ask in the forum]].

(For the new version of polyDB see [[user_guide:howto:polydb_tutorial|here]] (currently only for developers, will be released soon).
===== The polymake database =====

The database is hosted on the polymake server (at [[http://db.polymake.org|db.polymake.org]], where you also find a web interface). The objects are arranged in collections which reside in so-called databases. See [[:polydb|here]] for a list of available databases and collections.

From the polymake shell the function ''[[https://polymake.org/release_docs/latest/common.html#common__db_info__284|db_info]]'' prints information about available databases and collections.

<code>
polytope > db_info();
DATABASE: LatticePolytopes
This database contains various classes of lattice polytopes.

Collection: SmoothReflexive
All smooth reflexive lattice polytopes in dimensions up to 8 up to lattice equivalence. The lists were computed with the algorithm of Mikkel Oebro (see [[http://arxiv.org/abs/0704.0049|arxiv: 0704.0049]]) and are taken from the [[http://polymake.org/polytopes/paffenholz/www/fano.html|website of Andreas Paffenholz]].

In dimensions up to 7 the database contains the properties:
H_STAR_VECTOR, REFLEXIVE, CONE_DIM, date, _id, LATTICE_CODEGREE, N_INTERIOR_LATTICE_POINTS, SMOOTH, N_LATTICE_POINTS, FACET_WIDTHS, VERTICES, FACETS, CENTROID, N_VERTICES, contributor, LATTICE_DEGREE, LATTICE_VOLUME, EHRHART_POLYNOMIAL_COEFF, N_BOUNDARY_LATTICE_POINTS, ESSENTIALLY_GENERIC, VERY_AMPLE, F_VECTOR, GORENSTEIN, FEASIBLE, LINEALITY_SPACE, AFFINE_HULL.

In dimension 8 it only contains the minimal set:
LINEALITY_SPACE, CONE_DIM, date, _id, VERTICES, FEASIBLE, N_VERTICES, contributor.

__________________

DATABASE: Tropical
This database contains tropical objects.

Collection: TOM
All known non-realisable tropical oriented matroids with parameters n=6, d=3 or n=d=4. They were computed using polymake with the tropmat extension and Topcom. You need the extension [[http://solros.de/polymake/tropmat|tropmat]] for this.

__________________
</code>
Note that this command only lists those databases for which you have read access. The command takes the options ''db'' to list only collections in a particular database, ''collection'' to list only one collection (in this case ''db'' must also be given), and ''username'', if you have access to some collections not (yet) publicly available. 


It is also possible to host your own database on your server. This is, however, not yet documented.




===== Basic database access =====

In this tutorial we are going to use the collection SmoothReflexive in the database LatticePolytopes, which contains all smooth reflexive lattice polytopes in dimensions up to 8.
==== Queries ====

Database queries are given as perl hashes. In its most basic form a query has the form
   { "PROPERTY1" => <value1>, "PROPERTY2" => <value2>, ... }
for example
   { "DIM" => 3, "N_VERTICES" => 7 }
Note that the database is pretty strict with types, so strings always need to be quoted, while integers must not be. The following would fail:
   { "DIM" => "3" }
The empty query is allowed:
   {}
returns all objects of a collection. You can query elements in an array with their index, e.g.
   { "F_VECTOR.1" => 12 }
gives polytopes with 12 edges (the entry at position one of the f-vector).

For more sophisticated querying see the [[user_guide:howto:poly_db_tutorial#advanced_queries|section on advanced queries]].


=== db_query ===

Note that you should **not** use this function if you expect a large number of objects to match your search criteria. ([[https://polymake.org/release_docs/latest/common.html#common__db_count__285|Check this with db_count]].) In this case create a DatabaseCursor instead, as explained [[user_guide:howto:poly_db_tutorial#using_a_databasecursor|here]].

The function ''[[https://polymake.org/release_docs/latest/common.html#common__db_query__283|db_query]]'' expects a query, and as options a database and a collection name. The following code returns an Array<LatticePolytope> of all 3-dimensional polytopes in our collection of smooth reflexive lattice polytopes.

  polytope > $a = db_query({"DIM" => 3}, db=>"LatticePolytopes", collection=>"SmoothReflexive");

Note that (depending on server capacity and network speed) this may take a while. Sometimes it might even result in a time out. In this case just try again (and feel free to contact us if the problem persists).

Check the [[user_guide:howto:poly_db_tutorial#custom_variables|section on custom variables]] to omit having to enter ''db'' and ''collection'' on every query.

Now we can browse through the elements of ''$a'' and inspect their properties, e.g.:

<code>
polytope > print $a->[0]->SMOOTH;
1
polytope > print $a->[0]->REFLEXIVE;
1
polytope > print $a->[0]->VERTICES;
1 1 1 0
1 1 0 0
1 0 1 -1
1 1 -2 1
1 1 1 1
1 0 1 1
1 0 -3 1
1 -4 1 -3
polytope > print $a->size;
18
</code>

=== List queryable entries ===

You can print a list of all queryable entries of a database collection:
<code>
polytope > db_print_searchable_fields(db=>"LatticePolytopes",collection=>"SmoothReflexive");
Entries for smooth reflexive polytopes in dimensions 1 to 7
----------------------------
AFFINE_HULL  [dense, int]
CENTROID
CONE_AMBIENT_DIM  [int]
CONE_DIM  [int]
EHRHART_POLYNOMIAL_COEFF
FACETS  [dense, int]
FACET_WIDTHS  [int]
(...)

Entries for smooth reflexive polytopes in dimension 8
----------------------------
(...)

Entries for smooth reflexive polytopes in dimension 9
----------------------------
(...)
</code>

=== Request a limited number of objects ===

You can use the ''limit'' option:

<code>
polytope > $a=db_query({'DIM' => 7}, db=>"LatticePolytopes", collection=>"SmoothReflexive", limit => 10);

polytope > print $a->size;
10
</code>

=== Request possible values of a property ===

You can use the option ''distinct'', e.g.

<code>
$a=db_query({'DIM' => 3}, distinct=>"N_VERTICES");
</code>

returns an array that contains all values ''N_VERTICES'' can take for a 3-dimensional polytope. 


=== Request only one representative of a collection ===

You can use the ''db_query'' function with the option ''representative'' if you are only looking for one arbitrary example matching certain query criteria, e.g.:

  polytope > $p = db_query({"CONE_DIM"=>4, "N_VERTICES"=>10}, representative=>1);
  
  polytope > print $p->_id;
  F.3D.0002
  polytope > print $p->DIM;
  3
  polytope > print $p->N_VERTICES;
  10  

Note that you might get a different polytope (with another _id) if you repeat the above code.

You can also use this function to extract a polytope with a certain ''_id'':

  polytope > $p = db_query({"_id"=>"F.4D.0123"});

=== Custom variables ===

If you are going to access the same collection all the time, you can set the custom variables ''$db_name'' and ''$collection_name'':

  $PolyDB::default::db_name = "LatticePolytopes";
  $PolyDB::default::collection_name = "SmoothReflexive";

These variables are used as default for the database and collection name if you omit the options of ''db_query'' and ''db_count'', e.g.:

  $a = db_query({"CONE_DIM"=>5, "N_VERTICES"=>15});

If you do

  set_custom $PolyDB::default::db_name = "LatticePolytopes";
  set_custom $PolyDB::default::collection_name = "SmoothReflexive";

these settings will be saved in your preference file and still be available after restarting polymake.

In the code examples below it is assumed that $PolyDB::default::db_name and $PolyDB::default::collection_name are set thus.

=== Metadata ===

In addition the "normal" polymake properties, database objects have two additional properties: 
   * Every object in a collection has a unique ''_id''. 
   * Every object has a metadata section that can contain
      * the contributor
      * the creation date
      * the name of the ''application'' in ''polymake'' 
      * the version of ''polyDB'' the object was inserted with
      * the collection it is contained in
      * the database of the collection
      * the type of the object (currently only ''object'' is allowed, matrices etc cannot be stored in the db yet)

You can access the metadata with the ''db_print_metadata'' function

<code>
polytope > print $a->[1]->_id;
F.3D.0001
polytope > db_print_metadata($a->[1]);
collection: SmoothReflexive
contributor: Andreas Paffenholz
creation_date: 2016-08-30
database: LatticePolytopes
id: F.7D.000001
tag: object
version: 3.0
</code>


==== poly_db_count ====

The function ''[[https://polymake.org/release_docs/latest/common.html#common__db_count__285|poly_db_count]]'' returns the number of objects matching your query criteria, e.g.:

  polytope > print db_count({"DIM"=>3});
  18
  polytope > print db_count({"DIM"=>4});
  124
  polytope > print db_count({"DIM"=>5});
  866
  polytope > print db_count({"DIM"=>6});
  7622
  polytope > print db_count({"DIM"=>7});
  72256
  polytope > print db_count({"DIM"=>8});
  749892

==== db_ids ====

There is another function ''db_ids'' that is very similar to ''db_query'' but only returns an array of IDs, e.g.:

  polytope > print db_ids({'CONE_DIM' => 8}, db=>"LatticePolytopes", collection=>"SmoothReflexive", limit => 10);
  F.7D.009999 F.7D.000000 F.7D.000001 F.7D.000002 F.7D.000003 F.7D.000004 F.7D.000005 F.7D.000006 F.7D.000007 F.7D.000008

==== Advanced queries ====
You can use the full query syntax of MongoDB, as explained [[https://docs.mongodb.com/manual/reference/operator/query/|here]].

=== Ranges ===

Instead of specifying the exact value of a property we might be interested in specifying ranges for numerical properties.

The following queries the polytopes with (strictly) less than 100 vertices:

  polytope > print db_count({"N_VERTICES" => { '$lt' => 100 }});
  107917
 
You can also query those with less than 100 but (strictly) more than 50 vertices:
  polytope > print db_count({"N_VERTICES" => { '$lt' => 100, '$gt' => 50 }});
  97595

The operators ''$lte'' (for less than or equal) and ''$gte'' (for greater than or equal) are also supported.

=== Regex search ===

You can also use regular expressions to search for strings. Since there are not so many string-valued properties around, let's try with the _id. The following is another way to count 4-dimensional polytopes (those with ''_id'' beginning with F.4D).

  polytope > print db_count({"_id" => qr/F.4D/});
  124
  
We could also count those objects added in August 2013:

  polytope > print db_count({"date" => qr/2013-08-/});
  80886  

===== Using a DatabaseCursor =====

If you expect your query to match a large number of objects (which you can find out using ''db_count'' on your query), you should construct a database cursor instead of using the function ''db_query''. A PolyDB cursor accepts the same arguments as a normal ''db_query'', i.e. you can specify database and collections, you can narow the search with a query, and you can set ''limit'' and ''skip''. The options ''representative'' and ''distinct'' don't work here. The request for a PolyDB cursor returns a pointer into the database, and each call to ''next'' retrieves another object. For large result sets, this saves memory locally and reduces the time until you can start with computations. Note though that you have to create a new cursor if you want to start from the beginning.

<code>
  $c = db_cursor({"DIM"=>4}, db=>"LatticePolytopes", collection=>"SmoothReflexive");
</code>

A cursor has four special methods:
   * ''next'': retrieves the next object from the database
   * ''has_next'': Checks if there are still objects that satisfy the query in the database that have not yet been retrieved.
   * ''at_end'': similar to ''has_next'', returns true if no further object can be retrieved.
   * ''count'': tells you the number of results that match your query

<code>
  polytope > print $c->count;
  124
</code>

<code>
polytope > while ($c->has_next) { print $c->next->N_VERTICES." "; }
11 8 17 13 15 18 15 16 12 12 15 16 16 12 16 16 16 13 12 12 9 21 17 17 23 30 13 17 15 18 16 15 12 17 21 21 18 24 18 17 16 13 13 12 15 12 9 20 24 20 16 16 20 20 24 20 24 20 20 16 20 24 20 21 20 24 24 20 20 16 16 16 16 12 25 30 36 20 24 20 16 16 16 20 16 12 17 16 16 16 16 20 16 12 15 18 15 20 24 18 15 15 16 11 11 12 12 12 13 16 16 12 12 12 8 8 12 9 16 12 8 8 9 5
</code>

==== Sorting, etc. ====

If you want the objects to be retrieved in a certain order, the optional parameter ''sort_by'' allows to specify a sorting order, using 1 for ascending and -1 for descending. E.g. assume you want to sort ascending by ''_id'' ...

  polytope > $c = db_cursor({"DIM"=>4}, db=>"LatticePolytopes", collection=>"SmoothReflexive", sort_by=>{'_id'=>1});
  
  polytope > for ($i=1; $i<=10; ++$i) { print $c->next->name." "; }
  F.4D.0000 F.4D.0001 F.4D.0002 F.4D.0003 F.4D.0004 F.4D.0005 F.4D.0006 F.4D.0007 F.4D.0008 F.4D.0009

... or descending by number of vertices:


<code>
polytope > $c = db_cursor({"DIM"=>4}, db=>"LatticePolytopes", collection=>"SmoothReflexive", sort_by=>{'N_VERTICES'=>-1});

polytope > while($c->has_next){print $c->next->N_VERTICES." ";}
36 30 30 25 24 24 24 24 24 24 24 24 24 23 21 21 21 21 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 18 18 18 18 18 18 17 17 17 17 17 17 17 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 15 15 15 15 15 15 15 15 15 15 13 13 13 13 13 13 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 11 11 11 9 9 9 9 8 8 8 8 8 5
</code>


If you want to skip the first few objects, you can do so by specifying ''skip'':

  polytope > $c = db_cursor({"DIM"=>4}, db=>"LatticePolytopes", collection=>"SmoothReflexive", sort_by=>{'_id'=>1}, skip=>100);
  
  polytope > for ($i=1; $i<=10; ++$i) { print $c->next->name." "; }
  F.4D.0100 F.4D.0101 F.4D.0102 F.4D.0103 F.4D.0104 F.4D.0105 F.4D.0106 F.4D.0107 F.4D.0108 F.4D.0109