Skip to topic | Skip to bottom
Home
Main
Main.ObsRSSr1.10 - 16 Jan 2007 - 04:30 - JeremyCothrantopic end

Start of topic | Skip to actions

ObsRSS

See also ObsKML

Lately (September 2006) I've been wondering if the below xml rss type feeds might be more suitable to simple sharing of latest observation data from platforms. ObsRSS is shorthand for Observations RSS.

A common concern is collecting latest observations in near real-time from various platforms and many different entities are trying both to provide and consume these latest data via both the usual http/ascii/xml type feeds and web services/SOAP/REST type feeds.

In addition to a more complex query based web services approach(SOAP or REST based) which can be time consuming/resource intensive to develop and maintain, what's needed for this simpler publishing/report issue is an rss type approach where there is agreement on the rss/xml format and methods for providing cross-organizational mapping for the content elements.

If existing data feeds were mapped by platform to a more community standard rss/xml type feed below once then the duplicated effort of understanding/mapping to the various individual feeds could be reduced.

Below are listed what I would consider the critical elements to achieving a useful rss type feed listed in the 'minimal form' section.

  • position
    • Latitude and longitude are represented in a georss style tag ( http://georss.org )which is a simplified version of gml. Latitude and longitude are representative for all observations listed with this platform or rss message.
    • Elevation is referenced by the 'elev' tag (also borrow from georss) and has units of meters and a frame of reference as positive where this is height above the earth or ocean surface(sea level) and negative where this is depth below the earth or ocean surface(sea level). Elevation is listed with each observation.

  • time
    • use ISO8601 format in the 'dateTime' element listed with each observation including time zone to hour resolution

  • observation description
    • an observation needs to be described in terms of its type such as sea surface temperature, air pressure, etc. , unit of measure (uom) and observed value.

  • platform handle
    • we need a platform handle to display and to be used for mapping against other possibly duplicate data sources

Note in the below that xml namespace elements (xmlns) are given which further reference the list of valid element values and their definitions(such as http://nautilus.baruch.sc.edu/obsrss/observation_types.xml which was created using the perl DBIx::XML_RDB package code example). These additional reference documents could be used to create further higher level mappings and ontologies and reused as reference sources for others additional data feeds.

The filename includes a reference to the platform handle, unique timestamp for the file creation date or latest obs and convention version.

The platform handle in the below reference is a concatenation of the organization name, platform name and platform type(like carocoops:SUN2:buoy ). The platform type can be used to break up different messages from different parts of the platform like '..._buoy','..._met','..._waves',etc. This has been a useful convention for my purposes, but this convention does not have to be followed as long as the handle provides some description and forms a unique string key.

A new obs rss file would be produced with each latest set of (say hourly) observation data. These files could accumulate for say the previous day's worth of data or possibly just the latest obs only is shared depending on the data provider. The xml could be extended to reference a subscription time, frequency and possible secondary source.

Minimal form

#filename convention like follows:
#<platformHandle>_<datetime=YYYYMMDDHHMM>_latest_obs_v1.xml
#carocoops:SUN2:buoy:latest.xml

<platform xmlns:georss="http://www.georss.org/georss"
   xmlns:platforms="http://nautilus.baruch.sc.edu/obsrss/platforms.xml"   
   xmlns:observationTypes="http://nautilus.baruch.sc.edu/obsrss/observation_types.xml"   
   xmlns:uomTypes="http://nautilus.baruch.sc.edu/obsrss/uom_types.xml">

   <georss:point>33.83 -78.48</georss:point>
   
   <platforms:platformHandle>carocoops:SUN2:buoy</platforms:platformHandle>  

   <observation>
      <observationTypes:type>sea_surface_temperature</observationTypes:type>
      <dateTime>2006-08-17T14:00:00Z</dateTime> #ISO8601 time format GMT
      <value>20<value>
      <uomTypes:uom>celcius</uomTypes:uom>
      <elev>-1</elev>
   </observation>
   
   <observation>
      <observationTypes:type>air_pressure</observationTypes:type>
      <dateTime>2006-08-17T14:00:00Z</dateTime> #ISO8601 time format GMT
      <value>1016<value>
      <uomTypes:uom>millibar</uomTypes:uom>
      <elev>3</elev>
   </observation>

</platform>

Full form

Below is the 'full form' version of the above. I've added useful additional reference tags such as platform and data url's, quality flags and a breakout of the elements which form the platform handle. Other additional tags could be added for additional metadata or data product specific elements.

#filename convention like follows:
#<platformHandle>:latest.xml
#carocoops:SUN2:buoy:latest.xml

<platform xmlns:georss="http://www.georss.org/georss"
   xmlns:organizations="http://nautilus.baruch.sc.edu/obsrss/organizations.xml"   
   xmlns:platformTypes="http://nautilus.baruch.sc.edu/obsrss/platform_types.xml"   
   xmlns:platforms="http://nautilus.baruch.sc.edu/obsrss/platforms.xml"   
   xmlns:observationTypes="http://nautilus.baruch.sc.edu/obsrss/observation_types.xml"   
   xmlns:uomTypes="http://nautilus.baruch.sc.edu/obsrss/uom_types.xml"   
   xmlns:qualityFlagTypes="http://nautilus.baruch.sc.edu/obsrss/quality_flag_types.xml">

   <georss:point>33.83 -78.48</georss:point>
   
   <organizations:organizationName>carocoops</organizations:organizationName>
   <platformName>SUN2</platformName>
   <platformTypes:platformType>buoy</platformTypes:platformType>
   <platforms:platformHandle>carocoops:SUN2:buoy</platforms:platformHandle>  
   <platformURL>http://nautilus.baruch.sc.edu/carocoops_website/buoy_detail.php?buoy=buoy6</platformURL>

   <observation>
      <observationTypes:type>sea_surface_temperature</observationTypes:type>
      <dateTime>2006-08-17T14:00:00Z</dateTime> #ISO8601 time format GMT
      <value>20<value>
      <uomTypes:uom>celcius</uomTypes:uom>
      <elev>-1</elev>
      <qualityFlagTypes:qualityFlag>3</qualityFlagTypes:qualityFlag>
      <dataURL>http://nautilus.baruch.sc.edu/carocoops_website/buoy_graph.php?buoy=buoy6&graph_type=temperature_surface</dataURL>
   </observation>
   
   <observation>
      <observationTypes:type>air_pressure</observationTypes:type>
      <dateTime>2006-08-17T14:00:00Z</dateTime> #ISO8601 time format GMT
      <value>1016<value>
      <uomTypes:uom>millibar</uomTypes:uom>
      <elev>3</elev>
      <qualityFlagTypes:qualityFlag>3</qualityFlagTypes:qualityFlag>
      <dataURL>http://nautilus.baruch.sc.edu/carocoops_website/buoy_graph.php?buoy=buoy6&graph_type=air_pressure</dataURL>
   </observation>

</platform>

WFS form

Since OGC WFS (Web Feature Service) has been useful in the past for GML tagging recordsets, here's a draft at a possible WFS type feed record. Platform elements are repeated with each observation record.

<GML elements>

<platformHandle>carocoops:SUN2:buoy</platformHandle>  
<observationType>sea_surface_temperature</observationType>
<dateTime>2006-08-17T14:00:00Z</dateTime> #ISO8601 time format GMT
<value>20<value>
<uom>celcius</uom>
<elev>-1</elev>

CSV form

Since CSV (Comma Separated Value) is a simple and common way of passing data, here's a draft at a possible CSV type feed record. Platform elements are repeated with each observation record.

platformHandle,observationType,time,latitude,longitude,z,measurement

carocoops:SUN2:buoy,sea_surface_temperature,2006-08-17T14:00:00Z,33.83,-78.48,-1,20

For folks familiar with SWE(Sensor Web Enablement) efforts and getObservation web service, the following mapping of terms would apply:

platformHandle=offering
observationType=observationProperty
dateTime=time

CSV/SOS form

SOS (Sensor Observing Service) is part of the integrated ocean observing system effort documented at http://www.oostethys.org/sos-cgi-cookbook-v0.1 and http://twiki.sura.org/twiki/bin/view/Main/OOSTechSWE

The below CSV (Comma Separated Value) uses the platformHandle name and observationType as part of the filename. Each file contains only one platformHandle and observationType. The below files could also reference a sos_config.xml file detailed at http://www.oostethys.org/sos-cgi-cookbook-v0.1

Within filename <platformHandle>_<observationType>.csv
time,latitude,longitude,z,measurement

2006-08-17T14:00:00Z,33.83,-78.48,-1,20

Future development

  • develop shapefile based derivations for traditional desktop based GIS users

  • a pathing and filenaming convention could be used to support larger archival datasets. Any of the above more verbose xml or csv forms could be additionally zipped for compression/bandwidth needs.

  • Please email me at jcothran[at]asg.sc.edu if interested in providing feedback on the above type latest observation rss type feed or something similar and if you would be interested in providing such feeds or are interested in visualizations/products derived from such feeds.

  • Place rdf:ID tags in namespaces?

Other earlier and ongoing work

The above xml rss forms are related to and derive from earlier and ongoing work related to ocean observing system web services development documented at

http://twiki.sura.org/twiki/bin/view/Main/OosTechServiceDefinition
http://twiki.sura.org/twiki/bin/view/Main/OOSTechSWE

Also at the relational database level I'm working to provide a generalized table schema which would be used to collect and share data/products from documented at

http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/XeniaPackage

Data feeds

Currently the latest observations from the Seacoos database (1000+ platforms, federal and institution level) are hourly provided as both:

zipped xml file
http://nautilus.baruch.sc.edu/gearth/oostech.xml.zip

kmz file for display in google earth
http://nautilus.baruch.sc.edu/gearth/latest_placemarks.kmz

Smaller bounding boxes, individual platforms and observations can be queried via examples of the REST oriented web service documented at
http://twiki.sura.org/twiki/bin/view/Main/SoapliteSeacoos

Work done to convert data from their hourly raw ascii feeds to other formats is documented at
http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/CarolinasCoastGetData

-- JeremyCothran - 07 Sep 2006
to top


You are here: Main > ObsRSS

to top

Copyright © 1999-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding DMCC? Send feedback