Skip to topic | Skip to bottom
Main.RSSWeatherr1.1 - 05 Dec 2004 - 16:17 - JeremyCothrantopic end

Start of topic | Skip to actions

This webpage was copied from author Kellan Elliott-McCrea

I copied the source because I thought there was a lot of good information here and was worried that the link or content of the page might change - JeremyCothran

Recent thoughts/discoveries on the intersection of weather forecasts and RSS.

  • The Weather as XML

    Catching up on my backlog of blogging, I have to say lately I've been really impressed by my government. I'll wait while you all recover from the shock of that last statement.

    Last week I was impressed by the U.S. Census Bureau and the TIGER/Line database which make possible projects like, and the forthcoming "politcoder" (more on that soon). And sitting in my "to be blogged" queue for nearly a month now, the National Weather Service has ramped up their public XML offerings!

    The NWS is now offering

    This is stuff I'm happy to spend tax dollars on!

    Slashdot had a story on this, The Future of Free Weather Data on the Internet, claiming that the president of Accuweather is lobbying the NWS to shut down this useful public service because it threatens Accuweather's bottom line. (why anyone should expect to build a business on the back of data collected with public funds is beyond me) I can't comment on that directly, but it corresponds with what I know about NWS's concerns about releasing their RSS feeds last Fall. Also, almost beyond belief, there is a whole rash of intelligent, thoughtful Slashdot comments on that story, read'em for the novelty.

    Like last time, the NWS is soliciting feedback until August 1st. So if you think it makes sense for the government to be collecting important scientific/health and safety related data, and making it available to us in creative, and accessible fashions, make sure to let them know, and let them know any creative, and useful civic projects you are using the data for.

  • National Weather Service's XML Feeds

    The NWS recently decloaked their experimental RSS feeds of weather watches, warnings, and alerts. They are soliciting feedback until Dec. 30th.

    Check it out, and if you think it is cool as I do, and you think that the National Weather Service should be producing weather information in a public, easily accessible format, instead of having to buy that information from some commercial company, then you should contact them and tell them how excited you are to see this service. (because you know the vendors are going to scream when they see it)

    Now that the NWS is almost ready to start issuing weather data on a 5kmx5km grid, Bob has said that they see their "future more like you give us a lat/long and we feed back xml for that 'gridpoint'." How cool is that?

  • rssWeather

    Bah, too many projects, too little time. Until the day when I get back to this project, take a look at these other alternatives.

    Someone else has built a nice, clean implementation of weather-over-RSS, see rssWeather. They are using HAMWeather as the backend, rather then a scraper, which is certainly a conceptual improvement, though it seems that HAMWeather is limited to using ICAO codes for non-US locations. (though that is probably scriptable around) Nicely browsable, good web integration.

    I think I'll probably revive my own weather-rss project at some point. (it moved forward a bit last weekend) In particular I still think there is an opportunity to format the weather info, in a more useful, and more semantic fashion. Once I've got my working example, I'll probably be knocking on Tony of OpenWeather's and's door, see if we can get some coherence.

  • Weather RSS and the Dangers of Screen Scraping

    So the Weather RSS service is down right now. The whole thing is driven by screen scraping because the freely available sources of weather info suck (or at least the ones I can find do).

    I have a working screen scraper for but unforunately their URLs are relatively obscure. I have some code to mechanize their search form, but they got a bit jumpy, and temporarily blocked my IP while I was tuning it. So I put them on hold, and went with Wunderground.


    Well the Wunderground has US and international weather, so that is a plus, and nicely predictable URLs, which really helps when you're screen scraping. But they HTML sucks. Its very much circa 1997, but more cluttered. Its not a field I have a lot of expirence with, but I thought my scraper was pretty good, but 2 week later it is broken. Ugh.

    Haven't tried yet, years ago when I was last playing with weather, they blocked my IP for scraping (and I'm being well behaved I promise!), don't know if they still do that, my instinct is not, at least not within reason. However their URLs seem totally arbitrary, probably pegged to an internal numbering scheme. If anyone knows differently that would be great. (Also their editorial voice is insipid, the differnece I suppose between people who study weather for a living, and those who sell it for a living, e.g "Sunny" becomes "Plentiful sunshine".)

    More Domain Knowledge and Directories

    Looks like I'm going to have to move away form Wunderground, their HTML just isn't reliable enough. To do that I'll need to understand the identifiers the NOAA and are using. The NOAA identifiers are standard I think, but I havne't found a good documentation of them. The identifiers could probably be fetched by walking their website (or their syndicated Yahoo weather site which has a more directory like structure) The one problem with moving to using the identifiers rather then a search interface is rather then allowing free form entry, one would want to present the users with a deeply nested list of possible choices, which has never worked all that well on the web. I just downloaded WeatherPop (sweet little app btw), who I suspect is also a screen scraper (though I should probalby examine my outgoing traffic before making that claim) to see how it handles this problem, and presenting a drill down list is exactly what it does.

    Anyone has other suggestions for data in a workable format, let me know. God knows I hate screen scraping in this day and page (CSS makes it easier, but it feels so backwards)

    That and I never quite got conditional gets supported, it was working in Magpie, and at least one of the Windows clients (don't remember which one) but not in NNW, or most of the other readers.

    And last but not least, there is the lovely "Divsion by zero" error I get when trying to view Denver's weather.

  • Brief Weather RSS (un)Update

    Still doesn't work with US cities, stop subscribing to those feeds. All you'll see is 2 month old data for Providence, RI.

    Feedback would be really helpful, for example, one of the most commonly requested feeds Bristol, UK doesn't work at all. Your responsibility as beta-testers of an experimental tool is to tell me these things.

    This has been a pre-coffee blogging session.

-- JeremyCothran - 05 Dec 2004
to top

You are here: Main > RSSWeather

to top

Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding DMCC? Send feedback