DMAC Support

After August 3, 2009...

DMAC is charged by IOOS to maintain data and metadata of all data that passes through the system.  DMAC supports the Prince William Sound 2009 field experiment by acquiring, storing, and making available the data from ten PIs and a suite of sampling platforms and models.  DMAC makes available data in formats to reduce time to inject data into the processing and visualization routines.  In general, data formats that are easily readable are: ASCII (flat, CSV), XML, HDF(4, 5, EOS), NetCDF, Grib1 and Grib2.  Files may be compressed using ZIP, Gzip, compress, and Bzip2.  Tar and compressed tar collections are also permitted. PIs may opt to send data as-is, however, the format must be well documented and understood but cannot accept proprietary data that are in formats that cannot be converted without the use of commercial tools. Metadata templates will be made available to participants so that datasets and products are appropriately identified and documented.  This is particularly useful at later stages of the research project (publications). 

 

Ingesting Data

The Data Management team received data from ten Investigators; this data comes via multiple streams in several different formats. See Data by Investigator for the list. There are presently 70 items listed. Many more items will be coming in at the close of the 'in field' portion of the experiment.

Processing Data

Select steps are necessary to process any data item. At the very least these materials must be cataloged and entered in the database. Select data streams need to be conditioned then formatted then put into a meaningful directory structure on top of being cataloged and entered into the database. Eg:

  • Mooring data is available in CSV format for Salinty and Water Temperature for two moorings (with one more coming soon.) The link to "Salinity" runs agains the SOS service loaded on one of the Data Management servers. The buoy (Port San Juan), the property (Salinity) and the type (CSV) get passed to the program. You then see the results. Learn more from the SOS wiki.

Providing Imagery

Many data products came into the AOOS Data Warehouse hands during phase one of the Prince William Sound Field Experiment. See more information on the end-to-end delivery that was provided that included imagery. Not all products received real-time images by the AOOS DM team, but all products were ingested into the database, in several cases making it possible for other groups to provide excellent graphics. We've taken steps all along to try to point viewers to all materials being generated.

Please forward to dm@aoos.org any materials you wish to have represented here.