Customization

Naiad can be customized to better suit your need in various ways:

  • adding custom properties to the indexed granules and tiles
  • generating your self the granule and tile files

Granule and tile properties

By default, Naiad only index the spatial and temporal boundaries of the granules and tiles registered into the system (done by naiad-tile command).

However it is possible to index more metadata for both granules and tiles in order to later perform search on additional criteria rather than just a geographic area and time frame. For instance, you would want only granules or granule subsets over an area that have a mean solar zenital angle greater than 90 degrees (nighttime data).

This can be done by implementing a python class inheriting from Tiler class, where you will override two methods:

  • process_granule_properties() : add additional properties to granule metadata
  • process_tiles_properties() : add custom properties to the spatial/temporal properties (or metadata) of a granule’s tiles

In the following example, we design a specific tiler class for GHRSST products: we want to store the mean solar zenital angle for each granule’s tile to later select only night time granule subsets. To do so, we create a new class inheriting from :class:Tiler class where we implement the two forementioned methods.

To apply this class in the tiling process, there is no need to write a new command line tool replacing naiad-tile. Just call naiad-tile with --tile-class <your class> option, which in above case would give:

naiad-tile 20100123-EUR-L2P_GHRSST-SSTsubskin-AVHRR_METOP_A-eumetsat_sstmgr_metop02_20100123_000103-v01.7-fv01.0.nc GHRSSTNCFile --show --xstep=540 --ystep=540 --tile-class GHRSSTTiler

Creating custom granule and tile files

The easiest way to produce the granule and tile metadata to be indexed into Naiad is using the naiad-tile command line tool. However it gives you little flexibility to produce your own metadata content.

We showed above a first method to customize your metadata, by heritage of the Tiler class.

This is not the only way : you can also edit or create these metadata yourself by writing your own tile files (that you will then register with naiad-register command) or even writing json documents that you will directly index in elasticsearch (this is for very advanced users as you have to know how to work with elasticsearch and be fully aware of how naiad stores the granule and tile metadata into elasticsearch indices.

Writing your own tile files is very easy. A tile file will generally contain a line containing the metadata of your granule file and as many lines as you have tiles breaking down this granule into smaller pieces.

Here is an example:

S3A_SL_1_RBT____20160523T083711_20160523T084011_20160523T102935_0179_004_249_4499_MAR_O_NR_001.SEN3;2016-05-23T08:37:11;2016-05-23T08:40:11;POLYGON (( -70.660400 -95.301800,-71.078900 -95.895700,-71.495400 -96.515400,-
71.912800 -97.172300,-72.321800 -97.839300,-72.731400 -98.546800,-73.142900 -99.314000,-73.543800 -100.068000,-73.949600 -100.893000,-74.344400 -101.741000,-74.738600 -102.641000,-75.128900 -103.586000,-75.518000 -104.
584000,-75.900000 -105.630000,-76.276500 -106.738000,-76.649400 -107.898000,-77.016100 -109.133000,-77.377400 -110.419000,-77.729700 -111.785000,-78.074000 -113.249000,-78.413600 -114.772000,-78.742800 -116.407000,-79.
063100 -118.113000,-79.369900 -119.935000,-79.673200 -121.864000,-79.957400 -123.895000,-80.231300 -126.044000,-80.490900 -128.346000,-80.736200 -130.722000,-80.966300 -133.261000,-80.991400 -133.564000,-83.205600 -123
.193000,-85.041100 -104.256000,-85.903700 -71.995200,-85.232200 -38.290600,-85.180700 -38.721700,-84.777400 -41.230800,-84.364700 -43.400200,-83.951100 -45.278100,-83.527300 -46.917300,-83.096100 -48.325000,-82.661900
-49.621600,-82.229600 -50.719800,-81.791300 -51.744700,-81.348900 -52.660900,-80.904600 -53.463300,-80.456300 -54.201600,-80.013000 -54.892600,-79.566100 -55.509400,-79.114900 -56.075500,-78.665100 -56.606600,-78.21570
0 -57.077100,-77.760100 -57.530500,-77.311300 -57.957800,-76.858500 -58.348800,-76.404900 -58.712900,-75.950400 -59.054900,-75.492600 -59.366100,-75.037700 -59.667700,-74.584400 -59.962600,-74.129100 -60.222600,-73.673
200 -60.485900,-73.214200 -60.713700,-72.757600 -60.941500,-72.300700 -61.157900,-72.476400 -70.102700,-72.246100 -79.020600,-71.625100 -87.534400,-70.660400 -95.301800 ));Swath;{"version": "2.3"}
S3A_SL_1_RBT____20160523T083711_20160523T084011_20160523T102935_0179_004_249_4499_MAR_O_NR_001.SEN3;2016-05-23T08:37:11;2016-05-23T08:40:11;POLYGON (( -70.660400 -95.301800,-71.078900 -95.895700,-71.495400 -96.515400,-
71.912800 -97.172300,-72.321800 -97.839300,-72.731400 -98.546800,-73.142900 -99.314000,-73.543800 -100.068000,-73.949600 -100.893000,-74.344400 -101.741000,-74.738600 -102.641000,-75.128900 -103.586000,-75.518000 -104.
584000,-75.900000 -105.630000,-76.276500 -106.738000,-76.649400 -107.898000,-77.016100 -109.133000,-77.377400 -110.419000,-77.729700 -111.785000,-78.074000 -113.249000,-78.413600 -114.772000,-78.742800 -116.407000,-79.
063100 -118.113000,-79.369900 -119.935000,-79.673200 -121.864000,-79.957400 -123.895000,-80.231300 -126.044000,-80.490900 -128.346000,-80.736200 -130.722000,-80.966300 -133.261000,-80.991400 -133.564000,-83.205600 -123
.193000,-85.041100 -104.256000,-85.903700 -71.995200,-85.232200 -38.290600,-85.180700 -38.721700,-84.777400 -41.230800,-84.364700 -43.400200,-83.951100 -45.278100,-83.527300 -46.917300,-83.096100 -48.325000,-82.661900
-49.621600,-82.229600 -50.719800,-81.791300 -51.744700,-81.348900 -52.660900,-80.904600 -53.463300,-80.456300 -54.201600,-80.013000 -54.892600,-79.566100 -55.509400,-79.114900 -56.075500,-78.665100 -56.606600,-78.21570
0 -57.077100,-77.760100 -57.530500,-77.311300 -57.957800,-76.858500 -58.348800,-76.404900 -58.712900,-75.950400 -59.054900,-75.492600 -59.366100,-75.037700 -59.667700,-74.584400 -59.962600,-74.129100 -60.222600,-73.673
200 -60.485900,-73.214200 -60.713700,-72.757600 -60.941500,-72.300700 -61.157900,-72.476400 -70.102700,-72.246100 -79.020600,-71.625100 -87.534400,-70.660400 -95.301800 ));Swath;0;1199;0;1499
...

The first line describes a granule, with the following sequence of fields, separated by ;:

  • the granule name, here S3A_SL_1_RBT____20160523T083711_20160523T084011_20160523T102935_0179_004_249_4499_MAR_O_NR_001.SEN3
  • the granule start time, here 2016-05-23T08:37:11
  • the granule end time, here 2016-05-23T08:40:11
  • the granule spatial shape, as a wkt string, here POLYGON ((-70.660400 -95.301800,-71.078900 -95.895700,-71.495400 -96.515400,-71.912800 -97.172300,-72.321800 -97.839300,-72.731400 -98.546800,-73.142900 -99.314000,-73.543800 -100.068000,-73.949600 -100.893000,-74.344400 -101.741000,-74.738600 -102.641000,-75.128900 -103.586000,-75.518000 -104.584000,-75.900000 -105.630000,-76.276500 -106.738000,-76.649400 -107.898000,-77.016100 -109.133000,-77.377400 -110.419000,-77.729700 -111.785000,-78.074000 -113.249000,-78.413600 -114.772000,-78.742800 -116.407000,-79.063100 -118.113000,-79.369900 -119.935000,-79.673200 -121.864000,-79.957400 -123.895000,-80.231300 -126.044000,-80.490900 -128.346000,-80.736200 -130.722000,-80.966300 -133.261000,-80.991400 -133.564000,-83.205600 -123.193000,-85.041100 -104.256000,-85.903700 -71.995200,-85.232200 -38.290600,-85.180700 -38.721700,-84.777400 -41.230800,-84.364700 -43.400200,-83.951100 -45.278100,-83.527300 -46.917300,-83.096100 -48.325000,-82.661900 -49.621600,-82.229600 -50.719800,-81.791300 -51.744700,-81.348900 -52.660900,-80.904600 -53.463300,-80.456300 -54.201600,-80.013000 -54.892600,-79.566100 -55.509400,-79.114900 -56.075500,-78.665100 -56.606600,-78.215700 -57.077100,-77.760100 -57.530500,-77.311300 -57.957800,-76.858500 -58.348800,-76.404900 -58.712900,-75.950400 -59.054900,-75.492600 -59.366100,-75.037700 -59.667700,-74.584400 -59.962600,-74.129100 -60.222600,-73.673200 -60.485900,-73.214200 -60.713700,-72.757600 -60.941500,-72.300700 -61.157900,-72.476400 -70.102700,-72.246100 -79.020600,-71.625100 -87.534400,-70.660400 -95.301800 ))
  • the granule observation pattern among Swath, Grid and Trajectory, here Swath
  • additional metadata properties, as a dictionary of (property name, property value), here {"version": "2.3"}

The tiles follow the same sequence of fields, except that before the additional metadata properties you must have the offsets in the file of the data subset corresponding to the tile:

  • min column, max column, min line, max line for Grid or Swath patterns, here 0;1199;0;1499
  • min indice, max indice along the time axis for a Trajectory pattern

You can then register the tile files with the usual naiad-register` command.