Grandpa Ham – Grandpa Can do IT!

The Fiscal Year 2010 “Interoperable Communications Grant Program, Guidance and Application Toolkit” has just been published. My first question on seeing the grant language was, Did they mandate real interoperable data standards for software purchased using grant money?

They did. From Page 20:

Grant-funded systems, developmental activities, or services related to emergency response information sharing should conform as much as possible with the OASIS Emergency Data Exchange Language (EDXL) suite of data messaging standards and National Incident Management System (NIMS) guidelines. Additional information on data messaging standards and their applicability may be found at www.oasis-open.org. The NIMS Supporting Technology Evaluation Program (NIMS STEP) provides objective evaluations of commercial software and hardware products, and reports on product conformity to standards and NIMS guidelines. Findings from evaluations may be accessed through the Responder Knowledge Base (RKB) website to assist grantees in making purchases. More information on the NIMS STEP can be found at https://www.rkb.us/contentdetail.cfm?content_id=219711.

And Again from page 28 under Technology:

National Information Exchange Model (NIEM). FEMA requires all grantees to use the latest NIEM specifications and guidelines regarding the use of Extensible Markup Language (XML) for all grant awards. Further information about the required use of NIEM specifications and guidelines is available at http://www.niem.gov.

NIEM is XML. EDXL is XML. What gives? Who has precedence? Why is EDXL mentioned in the Funding Restrictions section and NIEM in the Administrative Requirements section?

In reality, you can ignore the apparent confusion. The requirements are valid and complimentary. For the most part, EDXL standards are accepted by NIEM as “approved external standards.” So you do not violate the NIEM requirements by using them, provided you use them as-is, in their entirety. If you use use individual elements (or a subset of elements) from an EDXL schema) in a way that does not validate against one of the schema standards, you are actually violating both EDXL and NIEM unless you document the use of those elements using the formal NIEM Information Exchange Package Documentation (IEPD) methodology as defined at niem.gov. So if you want to use a system that uses EDXL-Common Alerting Protocol (CAP), EDXL- Distribution Element (EDXL-DE), EDXL-Resource Messaging (EDXL-RM), and/or EDXL-Hospital Availability (EDXL-HAVE), go ahead. You are within the terms of the grant language. But if you modify (aka “improve”) the standards in any way, you must go through a formal IEPD process.

If, however you have requirements for information exchange that are not met by existing standards, NIEM offers you the opportunity to reuse existing NIEM IEPDs, build a new IEPD from existing NIEM data definition resources, or build an IEPD from a combination of data definition resources. It is a well-defined process that is designed to maximize reuse and minimize redundancy in data structure definitions supporting emergency management dat exchange requirements.

So, to summarize, if the software you are considering for purchase/development with your grant money reuses EDXL Exchange Standards and/or NIEM IEPDs, you are home free. If not, the system needs to define its exchanges with other systems following NIEM IEPD development rules as found at NIEM.gov

The link below is to a blog entry by Rick Wimberly concerning all of the alerting systems shown at the IAEM conference in Orlando this week.

http://www.emergencymgmt.com/emergency-blogs/alerts/The-Best-Notification-System.html

The basic premise is that there is no “best” alerting system and that the best alerting system is system of systems for alerting purposes that each have different traits and capabilities. I AGREE WHOLEHEARTEDLY. In fact, the activity where I currently work, FEMA’s Disaster Management Open Platform for Emergency Networks (DM-OPEN), is designed to allow communication between different alerting systems, such that they work together as a system of systems. At the IAEM conference, 10 different systems were using DM-OPEN to share the alerting function and it worked well because all were using the OASIS Common Alerting Protocol as a basis for exchange.

DM-OPEN also showed the ability of multiple systems to share OASIS Emergency Data Exchange Language Distribution Element (EDXL-DE) wrapped content. This content included NIEM IEPD Content (Amber Alerts) and OASIS Hospital Availability, but could also have included any defined data structure known to parties on at least two ends of the exchange. So, does this make DM-OPEN the best emergency information network? I might want to think so, but my thoughts are actually similar to Rick’s. I believe that no single network solution can legitimately call itself the best. Instead, it takes a constantly improving “network of networks” in combination to provide emergency managers with the best information available. In this arena, DM-OPEN does have a place. Because DM-OPEN connectivity is based on publicly available standards, it can connect network to network, as well as system to system as long as those systems are open to standards-based connectivity. So, DM-OPEN is not THE network or THE system. But if anyone else tells you theirs is THE solution, I would say they are blowing smoke, and that they need to learn to work with others.

Respectfully,
Gary “Grandpa” Ham

Attended combined NIEM National Training Event and Oasis Interoperability Summit in Baltimore last week. What a week!!

All of the following Items are from that event:
1. Participated in live demonstrations of interoperability by 11 separate commercial vendors, all using the DM-OPEN Backbone. Messages included Common Alerting Protocol sent from an actual Chorine sensor, NWS Tornado Warnings in CAP with the full polygon showing on maps used by multiple vendors, EDXL-DE wrapped Hospital Availability Messages, and EDXL-DE wrapped NIEM Amber Alert Messages, with accompanying Style sheet and reference base-64 encoded picture data used in full display. A professional videographer filmed the demonstration activities and interviewed key players. The edited video will be made available by OASIS. I will post the link when it is available.
2. Moderated NIEM NTE panel titled “Coordinating the Development and Adoption of Emergency Data Standards With the Ongoing Development of NIEM.” A format of 5 separate questions with short answer to each question by all panel members in turn was well received, both by the panel and the audience. Answers were lively and interesting. There were many audience questions as well. The NIEM organization recorded all panel sessions, so this panel will be available for review in its entirety.
3. Acted as a panel member in a second NIEM NTE panel titled: “Playing Well With Others”—NIEM and External Standards. This was a half session panel that stirred lots of interest and did not afford adequate time for all audience questions. Its recording will also be made available by the NIEM organization. Both panels made is clear that there is real cooperation between standards bodies and progress is being made to ensure that the value of all standards is recognized as a federation real capabilities. While some technical and “turf” issues need to be understood better, the folks involved look forward to the future with a positive attitude and a real belief in success.

Special Thanks to Donna Roy (NIEM Director) and her crew for a great event, and to Bill Kalin (Contractor to DHS Science and Tecnology) and Jane Harnad (OASIS) for organizing a superb demonstration and to all of the vendors for showing real interoperability in action. Standards do work!!!

The italics below contain a Facebook entry by Neil Bourgeois to his friends, repeated with his permission:

Remembering those who sacrificed all on 9/11. Also remembering everyone who made DMIS Fastrack happen after that. You would be pleased to know that the next generation DMIS and INTEROP has a bright future, which was built off a firm foundation that many of you worked so hard to put in place. Also remembering Charlie Bell, he would also have been pleased to see how far things have gone.

Charlie Bell was the original Marine Corps Program Manager for what is now FEMA’s DM-OPEN and DM-Framework. Charlie died before he could see his vision fully operational, but it was his vision that started the program, even before 9/11. Charlie, your vision became our passion on 9/11. It is now well on its way to becoming a national asset. You can be proud. We miss you, but we know you are watching.

The following announcement (copied from the FEMA Disaster Management Program govdelivery message stream) will be of interest to all who want to know how to use the Distribution Element properly: (NIEM users take note. This is one external Standard that makes IEP transport both easier and more effective, especially if your IEPD includes an XSLT in its documentation.)

OASIS EDXL Distribution Element Primer
Wednesday September 16, 12:00 Noon Eastern

In follow up to last month’s DM-OPEN SIG program, this month’s program will feature a new publication from OASIS, designed to provide developers with the A-B-Cs of the EDXL Distribution Element (EDXL-DE). “The Distribution Element: The Basic Steps to Package and Address Your Emergency Information,” is a white paper intended to function as an introduction–not as a comprehensive technical explanation.

Our guests will include Elysa Jones, Warning Systems Inc. and Chair of the OASIS Emergency Management Technical Committee (EM-TC). She will be joined by other members of the committee to provide an overview and respond to questions. We also plan to provide further information about the upcoming OASIS Interoperability Summit, and general status of DM-OPEN development efforts.

This program is intended primarily for software application developers, especially those new to EDXL-DE. Please make plans to join us via conference bridge and Live Meeting.

IMPORTANT: If you have not logged into Live Meeting before, check out the following connection instructions and participant guidelines prior to next week’s meeting:

http://www.disasterhelp.gov/disastermanagement/library/documents/LiveMtgInstruct.pdf

(1) Login to MS Live Meeting for visuals: The following login link can only be used 30 minutes prior to the scheduled meeting time: https://www.livemeeting.com/cc/eiip/join?id=DMprogram&role=attend

(2) Call into the Conference Bridge number as follows: 1 (800)366-7242 PIN 3647 6736#.

If you are unable to attend this month’s meeting due to other commitments, a recording will be accessible from the DM-OPEN SIG Presentations Archive at http://www.disasterhelp.gov/disastermanagement/library/archive/open-presentations.shtm

I received e-mail asking about my ideas for incorporating external structures like IEEE 1512 or Cursor on Target (Cot) into NIEM.  I think my response may be of general interest:

I am not a believer in the “incorporation of external structures like 1512 or Cot in NIEM.”  The cost of merging the maintenance  and update process of multiple separate standards related organizations would be too high. The resulting governance process would induce such brittleness that progress would simply stop.   I am, however, a believer in using external standards in the IEPD process and in the use of external standards on their own when and wherever they make sense.  Developing a brand new NIEM structure to replace a well-defined standard that is already in use is ludicrous.  And I believe that the NIEM management team agrees with me when I say that.

There are actually four cases to consider that involve the use of external standards:

1. The external standard meets the mission of an IEPD on its own. In this case just use it.  Document it as the content of the exchange and do not spend time trying to “NIEMify” its content. (Examples are CAP1.1 and perhaps some of 1512).

2. The external standard is used to “wrap” NIEM content (likely case for EDXL-DE and Cot).  In this case again the standard is simply used to wrap a NIEM IEP (as defined in an IEPD for content).  There is no need to add the external standard elements into the internal content IEPD process.  The content IEPD should define how the external standard(s) could be used as a wrapping for transport, but that is different than defining the content itself and should be encapsulated in an entirely separate section of the IEPD from the content definition.

3. The external standard contains components that need to be used in an IEPD in combination with NIEM components (GML, KML, and perhaps 1512 are examples).  It would seem that we could simply add the appropriate namespace origin and mix them into our exchange schema with the content from a NIEM extension schema.  The problem here is that representations of these components are almost guaranteed to violate NIEM Naming and Design Rules (NDR).  For example: Even a simple type such as xsd:string must be converted to a nc:TextType, which is in turn a niem-xsd:string that acts as a proxy for the XSD type but adds an optional set of attributes to the string.  So, even a simpleType from schema becomes a complexType in NIEM.   Luckily, there is an out in the NDR called the adapter. An adapter is a simple, single element wrapper that is used in NIEM to contain the external component. It is defined in the NIEM NDR as follows:  “An adapter type is a NIEM conformant type that adapts external components for use within NIEM.  An adapter type creates a new class of object that embodies a single concept composed of external components.  A NIEM conformant schema defines an adapter type.” It basically adds the attributes at the adapter level that are needed for NIEM conformance without fussing with the representations and naming issues of its contained content.  An adapter can contain:
a) an entire external schema (assuming it wraps the root element type)
b) a complex component element type from a schema
c) a single type from the schema.
The use of adapters is described in section 7-7 of the NIEM NDR.  Serious users of NIEM should read the NDR in its entirety.  The final point on this discussion is very simple.  You do not incorporate an external structure into NIEM, but you can define a NIEM conformant adapter in your IEPD as a container for that external structure that will allow you to import the external structure “as-is.”

4. The NIEM is not complete and needs extension. There are concepts that are needed in NIEM that may exist in external standards already.  In this case the concepts (but not the standards themselves) actually do need to be “incorporated” into NIEM.  So, how do we bring in the concept without creating a duplication of the external standard?  We could “coordinate” with the external standards organization(s), each of which have their own representation for the concept. This is indeed a hard task, and creates brittleness in and out of NIEM.  It can be done to some extent, but it is not a good idea to put it into any form of formal governance.  The cost is just too high.  Luckily, NIEM already provides a workable solution: the use of Abstract Elements with Multiple Representations already exists for dates and units of measure.  This is extended to the use of external namespaces for code lists.  Why not do the exact same thing for concepts that originate in an external standard?
a) Bring in the concept.  Name and define it in accordance with NIEM NDR.
b) Identify the Concept as an abstract element with multiple representations.
c) Identify one of the representations as the one in the external standard using the external standard’s namespace. The NDR may require that it be wrapped in a single element Adapter.
d) Identify a NIEM compliant and/or other standard representations as appropriate.
This avoids the duplication of concept and organizes multiple representations of the concept in a manner that that both controls those representations and makes translation between representations easier to accomplish. Yet it does not induce “coupling” between NIEM and other standards organizations. The separation remains clear and clean.

Note: Items 1, 2, and 3 on this list are already in place as how you should work with NIEM.  I put them in this message as a way of organizing the different issues that may be in play. Number 4 is new.  It is a combination of my recent completion of the on-line Practical Implementer’s course for NIEM and discussions with FEMA’s Emergency management domain lead and a couple of Georgia Tech Research Institute (GTRI) representatives at the recent NIEM Business Architecture Council meeting in Baltimore.  I have also had off-line discussion with FEMA technical representatives and members of the OASIS Emergency Management Technical Committee.

A final note: Although they do conform to the NDR, not everyone likes adapters. They add one more tag to an already deep chain of tags in a NIEM compliant schema.  They allow the use of non-NIEM content (although it is content that is recognized by NIEM).  My take is that adapters make cooperation between NIEM and the external standards bodies possible without the intense combined governance that could have a negative impact on progress on all sides.

I wrote a poller for DM-OPEN that posts alerts received in DM-OPEN to the prototype Unified Incident Command and Decision Support System (UICDS).  This gives posters of Common Alerting Protocol (CAP) alerts the option of using DM-OPEN as a mechanism for also posting to UICDS for use by systems connected to that capability.  Two successful demonstrations to date: a month or so ago at the Virginia Department of Emergency Management and today in McClean, Virginia for some folks from DHS.  We posted alerts from NC4’s E-Team, CellCast’s Eagle, MyStateUSA, and DMIS Tools (also a DM offering) to UICDS where the alerts were provided to to a UICDS RSS feed and plotted on maps using Alert Sense (where proper locations were identified in the input CAP message).

The message:  DM-OPEN can be a “connection multiplier” for its interoperability partners.  In this case a single connection yielded 4 new partners (and possibly many more in the future).

It works — Mostly.  I had one of my team members working with me on Disaster Management Interoperability Services post OASIS Common Alerting Protocol (CAP) Alerts to a test organization (aka COG for collaborative Operations Group) from 4 very different tools (E-Team, CellCast, MyStateUSA, and DMIS Toolset). He made these posts while I was traveling to California at 36000 feet on a commercial airline that offers wi-fi internet access.  At the same time I was running an application on my laptop inside the airplane that I wrote last week that that polls the DM-OPEN COG which my team member was posting to.  This polling software picked up each of the Alerts and posted them as tweets to my Twitter account (grandpaham). This blog (grandpaham.com) also picks up my tweets in a plug-in on its sidebar from Twitter.   There was one anomaly.  I know that all four of my tweets reached Twitter successfully, because my blog picked them all up successfully. However, one of the Tweets (DMIS  Toolset) did not show on my direct Twitter page.

?????????? My blog retrieved the headline from Twitter, so I know it got there. But my Twitter page did not show it.  So, I have an unanswered question, for sure. Why did Twitter not display the last tweet?

I recently added “tweeting” to my prototyping code base.  I can now poll the DM-OPEN Server for an organization and tweet any new alerts to followers of a given Twitter member.  You can see the results by following “grandpaham” on Twitter.  In fact, you can see it live if you have access to any Emergency Management Software that is compatible with DM-OPE’s CAP alert interface.   If you post an alert to the Interoperability COG, it should show up as a “grandpaham” twitter.  Kudos to the folks who developed twitter4j. The code and examples work as advertised.  Using Twitter can act as a DM-OPEN notification service.

I wrote a little ditty that explains the value of what FEMA’s Disaster management program offers to vendors, open source developers and even contract developers in the Emergency Management and Public Warning Domains. It is a question that users of such software might ask their vendors. Take a look.
See my Contact Info if you would like some help getting started

Are you OPEN? [1]

Can you connect using standards?
Are you open to all?
Or are you a silo?
Using “standards” to stall?

Our open web service
Connects all kinds of apps.
A middleware instance
To share more than “CAPs.”[2]

We have a web service
Based on EDXL [3]
That helps apps connect
Yet encapsulate well.

Hard wired integration
Is not what we do.
You connect via service
As captured by you.

You decide layout
And your design form
But connect to all others
Using standards as norm.

We make it straightforward.
Your connection is clean.
The boundaries work well.
You control what is seen.

You have the power.
We provide pipes,
For transferring data
Of all defined types.

With data described
Using DE [4]
So intelligent routing
Can come to be.

We provide access.
You set the rules
In the tags that you set
In the DE through your tools.

We then connect others
As desired by you.
And they get your data
As you want them to.

They can format the layout
In their own way.
Connected, yet separate
With their own say

Into how to display
And how to reuse
And so can all others
Unless you refuse.

You can work independent,
Yet use standards to share.
The best choice to ensure
Service to all; everywhere.

Gary A. Ham – May 14, 2009

[1] FEMA – Disaster Management Program – Open Platform for Emergency Networks
[2] OASIS Common Alerting Protocol
[3] OASIS Emergency Data Exchange Language
[4] OASIS EDXL Distribution Element