Skip to topic | Skip to bottom
Home
Main
Main.OOSTechKMLr1.9 - 26 Jun 2007 - 17:52 - JeremyCothrantopic end

Start of topic | Skip to actions
NOTE - If you are arriving at this webpage from the Google Earth gallery link http://services.google.com/earth/kmz/realtime_ocean_data_n.kmz please continue forward to the webpage ObsKML Here are links to the original source ObsKML data and the styling script used to generate this KML file.


This is a webpage to discuss KML (Keyhole Markup Language used for 3D geospatial rendering in Google Earth and other viewers) related products which are leveraged/derived from OOSTech and OOSTethys related discussions and data and services.

JeremyCothran

KML placemark generator with layers by operator and observation and ranged color coding of observation values

Description

The above google earth image shows ocean observing data gathered to the SEACOOS database which is refreshed 5 minutes past the top of each hour available using this kml link.

The KML is generated from files used to support the OOSTethys effort detailed here

Features

  • placemarks contain links to operator(gomoos, etc), platform and data provided by sos_config.xml file
  • data layer for viewing by operator/platform all observations at each platform
  • data layer for viewing by observation property(sea_water_temperature, etc) using a ranged color coding of the data
    • color viewing of observation data helps visually identify suspicious/bad data
    • observation measurement range limits can be user-defined by supplying a http url to a style.xml file (see below usage sections) which allows users flexibility in how data is visually filtered
    • new observation properties can be added to units.xml and style.xml to support their display
  • layers/placemarks have been timeline enabled (requires google earth version 4 or later) with description of archival products listed here
  • can be called as a service or scripts can be used and modified elsewhere

Web service / browser client usage

SOS cookbook users

For users familiar with the SOS cookbook efforts documented at http://www.oostethys.org/sos-cgi-cookbook-v0.1 , the additional steps needed to provide the kml output from your existing configuration and data files are as follows

  • zip your sos_config.xml file to sos_config.xml.zip
  • zip your data files (referenced in the 'SOSDataFile' element) to a single zip file with filename now.zip . All the data files should be at the top level with no subfolders
  • make these 2 files, sos_config.xml.zip and now.zip HTTP accessible

The calling php page uses the http url's to gather the above 2 files and return a KMZ(a zipped kml) file which should automatically load into Google Earth (if GE is installed) and there are not errors with the supplied input files.

The required http arguments expected are

  • zip_config_url (like zip_config_url=http://dev.gomoos.org:8081/sos/metadata/sos_config.xml.zip)
  • zip_data_url (like zip_data_url=http://dev.gomoos.org:8081/sos/data/now/now.zip)

An optional http argument is style_url which lists the observation range limits(high and low) applied to the visualization. If this argument is not specified then the default style.xml is applied.

  • style_url (like style_url=http://carocoops.org/gearth/style.xml)

The below is an example http url using zipped config and data files from GOMOOS (thanks EricBridger?!) which produces the below visualization showing salinity observations. All the points are close shades of green since all the salinity observations are relatively close around 32 psu on a 20 to 40 psu range limit.

Note that for the below GOMOOS example, the response time before receiving a kmz file back is around 0.5 to 1 second per platform depending on the server load. The GOMOOS file contains 10 platforms so expect a 5-10 second wait.

http://carocoops.org/gearth/genPlacemarksSOS.php?zip_config_url=http://dev.gomoos.org:8081/sos/metadata/sos_config.xml.zip&zip_data_url=http://dev.gomoos.org:8081/sos/data/now/now.zip

In regards to automatically zipping files, here's a snippet of some code I use on my end towards this purpose.

##zip sos_config.xml file
`cd /sos/metadata; zip sos_config.xml.zip sos_config.xml`;
 
##zip latest(now) data files
`cd /sos/data/now; find . -name "*.csv" -print | zip now.zip -@`;

New users

New users are encouraged to review the SOS cookbook documents at http://www.oostethys.org/sos-cgi-cookbook-v0.1 which explain how to setup a OGC SOS(Sensor Observing System) setup for your data using a comination of xml metadata files, data files, templates and scripts which tie things together. While this KML product only requires the sos_config.xml file and data files, users are encouraged to use the SOS cookbook to provide a OGC SOS service and share/register this service for additional benefit.

The inputs required to this KML product are

I decided to go with zip files for this product because I have a large number of platforms(>1000) and observations(say 10 per platform) and making thousands of individual http requests using the defined OGC SOS response/request methods would be prohibitive. Hopefully at some point we'll(the OOSTech/OOSTethys groups) figure out the orthodox OGC SOS request/response methods for large amounts of data.

Note the following

  • zip files(sos_config.xml.zip, now.zip) contain no pathing or subfolders
  • in the sos_config.xml file for each observation listed the observation property is derived from the 'observationProperty' element and the path to the associated data file is in the 'SOSDataFile' element. For the KML product the pathing information in the 'SOSDataFile' element is ignored, but the filename is used in referencing the files in now.zip.
  • data filenames can follow the convention <operator>_<localPlatformName>_<sensorPackageIdentifier>:<observedProperty>
  • data content fields are in the following order
    • time (ISO8601 like 2006-10-10T14:00:00Z)
    • latitude in decimal degrees
    • longitude in decimal degrees
    • elevation in meters (negative is depth below sea level/elevation reference, positive is height above sea level/elevation reference)
    • measurement value (the supported observation property types and their unit of measures expected and converted to in the KML is determined by units.xml which can be changed or extended)

See the notes above regarding using the existing service to generate a kml product or the below notes on installing these scripts locally.

Server side install and usage

The scripts which support the KML generation are a php page to forward the http url arguments to a called perl script which uses the LWP::Simple and XML::XPath packages to retrieve and parse the provided input files and converts them to the returned KMZ (a zipped KML) file reference. I've also included the script which I use on the server side internally for my own automated generation purposes which is pretty much the same file as earlier with just a few internal server rather than http references to data so the zipping process isn't required. The server side script also generates content for the archival script genDayArchive.pl

For installation you'll also need a copy of units.xml file which is used to support unit conversion and labeling for the scripts and style.xml file which defines the color coded range limits.

Proving GE timeline support was very easy, just added the following element tags with the observation time to each of my placemarks.

<TimeStamp><when>2006-10-17T14:00:00Z</when><TimeStamp>


KML placemark generator for catalog effort

Description

After a discussion on some minimal metadata fields necessary to support an IOOS observations based catalog(not getting into sensor or data details), I developed a KML product which used a CSV (Comma Separated Value) file input to generate a KML output visualization of the the supplied catalog data detailed at ObsCatalog and with a sample visualization shown above.

Fortunately the new sos_config.xml file used in the SOS/Oostethys effort has incorporated those same catalog elements. Since only the format of the input data has changed to the KML catalog visualization, I've added some code which processes the sos_config.xml.zip (a zipped sos_config.xml file) and translates that to a CSV file which is then used by the earlier scripts to produce KML.

This product could be merged in with the above observations based KML product, but I'm leaving them separate for now for the following reasons

  • the catalog observation type metadata should be less frequently changing/referenced metadata compared to sensor/observation/measurement data
  • the catalog product focuses on displays specific to the catalog such as the operational/planned/offline status and coverage areas for observation types like hf radar

Web service / browser client usage

http://carocoops.org/gearth/catalog/zip2kml.php?zip_url=http://dev.gomoos.org:8081/sos/metadata/sos_config.xml.zip

The calling php page uses the http url to the sos_config.xml.zip file and return a KML file which should automatically load into Google Earth (if GE is installed) and there are not errors with the supplied input files.

The required http argument expected is

  • zip_url (like zip_url=http://dev.gomoos.org:8081/sos/metadata/sos_config.xml.zip)

The below is an example http url using zipped config and data files from GOMOOS which produces a kml catalog visualization of the available observations.

http://carocoops.org/gearth/catalog/zip2kml.php?zip_url=http://dev.gomoos.org:8081/sos/metadata/sos_config.xml.zip

In regards to automatically zipping files, here's a snippet of some code I use on my end towards this purpose.

##zip sos_config.xml file
`cd /sos/metadata; zip sos_config.xml.zip sos_config.xml`;

Server side install and usage

The scripts which support the KML generation are a php page to forward the http url argument to a called perl script which translates the xml to csv. The csv file is passed forward to the earlier csv to kml translation perl script which produces the returned kml.

Related Links

http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/JCNotes#Google_Earth
http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/XeniaPackage
http://nautilus.baruch.sc.edu/twiki_dmcc/bin/view/Main/JCNotes#July_27_2005_General_Info_Links

Update Notes

November 8, 2006: The dated attached files on this page were changed to incorporate the use of the much faster XML::LibXML references instead of XML::XPath

-- JeremyCothran - 14 Oct 2006
to top

I Attachment Action Size Date Who Comment
genPlacemarksSOS.pl.txt manage 19.1 K 08 Nov 2006 - 20:56 JeremyCothran NA
genPlacemarksSOS_server.pl.txt manage 19.0 K 08 Nov 2006 - 20:57 JeremyCothran NA
genDayArchive.pl.txt manage 4.8 K 08 Nov 2006 - 20:57 JeremyCothran NA
processCatalogZip.pl.txt manage 2.7 K 08 Nov 2006 - 20:58 JeremyCothran NA

You are here: Main > OOSTechKML

to top

Copyright © 1999-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding DMCC? Send feedback