Retrieving data =============== Any PostgreSQL client application can directly access the data stored in the relational database. Any text-parsing application can read the archival logs. The database schema is documented below; for users familiar with command-line KTL tools such as ``show`` and ``modify``, the use of ``gshow`` is recommended, as it will translate a basic query on the command line into the proper format to retrieve the data from a keyword history database. For users interested in turning keygrabber data into FITS headers, the ``keyheader`` tool abstracts away all of the database interaction. .. _schema: Database schema --------------- Within the configured database, a single table will be created for each service monitored by keygrabber. Each individual table has the following structure: =========== ================== ============= Column Type Description =========== ================== ============= time double precision UNIX timestamp keyword text KTL keyword name binvalue text Binary keyword value ascvalue text ASCII keyword value repeated integer Count of repeated broadcasts for this keyword value discarded integer Count of discarded broadcasts that arrived too quickly between the recording of this keyword value and the subsequent keyword value in the database =========== ================== ============= Here are some sample records:: time | keyword | binvalue | ascvalue | repeated | discarded ------------------+-----------+------------------+------------------+----------+----------- 1329178093.22804 | DISPSTA | 0 | Ready | 0 | 0 1329178093.23297 | DISPSTOP | 0 | No | 0 | 0 1329178093.29152 | TEMP | 15.5 | 15.5 | 0 | 0 1329178093.29531 | UPTIME | Heartbeat active | Heartbeat active | 0 | 0 1329178094.62863 | SETPOINT | 0.0 | 0.0 | 0 | 0 1329179561.85276 | TEMP | 15.4 | 15.4 | 0 | 0 1329179562.37642 | TEMP | 15.5 | 15.5 | 0 | 0 Using gshow ----------- Beyond replicating the basic functionality of ``show`` and ``cshow``, ``gshow`` can also query a keyword history database. Such queries are triggered by supplying one or more ``-date`` arguments on the command line. This usurps the regular control flow in the script; rather than issue requests via the regular KTL machinery, ``gshow`` queries the history database according to the parameters provided, then resumes its normal execution path using the query results as its data source. The format of the ``-date`` argument is any date string recognized by the Tcl ``clock scan`` function. Examples:: gshow -s hamiodine SETPOINT TEMP -date "September 27, 2011 11:23" gshow -s hamiodine SETPOINT TEMP -date "September 27, 2011 11:23" -date "September 27, 2011 12:23" gshow -s hamiodine SETPOINT TEMP -date "September 27, 2011 11:23" -window 1h gshow -s hamiodine SETPOINT TEMP -date "September 27, 2011 11:23" -window 1h -csv 1min gshow -s hamiodine SETPOINT TEMP -date "2 days ago" gshow -s hamiodine SETPOINT TEMP -date "yesterday" gshow -s hamiodine SETPOINT TEMP -date "11AM" -date "3PM" All three of these queries are retrieving the SETPOINT and TEMP keywords from the hamiodine service. The first query will retrieve all results from the date specified until now; the second query will retrieve only the results between the two dates specified. The third query, using the ``-window`` option, is another way to retrieve the same time range. The fourth query will output the results in a comma-separated values (.csv) format, only outputting one value per minute of available data. Subsequent queries demonstrate a limited number of the many alternate ways that dates can be specified. For further information, refer directly the ``gshow`` documentation, which one can review by running either of the following commands:: gshow -h gshow -H .. _keyheader: FITS headers with keyheader --------------------------- The ``keyheader`` command-line tool will generate valid FITS header segments defined by a flat-text configuration file. A typical invocation of ``keyheader`` will specify a named *snapshot* of keywords to retrieve, and a *timestamp* for a snapshot of those keyword values. It is thus possible to retrieve any FITS header segment at any time after the keygrabber data is in place; for example, a data taking daemon may have three blocks defined in the configuration file:: [ExposureBegin] [ExposureEnd] [ReadoutEnd] With appropriate timestamps in-hand, the data taking daemon could then request each of the three snapshots with three separate invocations of ``keyheader``, all right before writing the final FITS file to disk:: keyheader -c ./keyheader.conf -s ExposureBegin -t 12345678 keyheader -c ./keyheader.conf -s ExposureEnd -t 12345679 keyheader -c ./keyheader.conf -s ReadoutEnd -t 12345680 One major benefit of constructing FITS headers in this fashion is to decouple the collection of data from the generation of the FITS header. By doing so, the data taking daemon need not be in the business of monitoring KTL services; that is all abstracted away by leveraging keygrabber. And, as long as the data was properly stored, the FITS headers can be regenerated at any time, days, months, or years after the exposure was taken. The keyheader configuration file affords substantial flexibility in how the final FITS header segment will be formatted. A sample configuration file follows: .. include:: ../fits/sample.conf :literal: