Grandpa Ham – Grandpa Can do IT!

I received e-mail asking about my ideas for incorporating external structures like IEEE 1512 or Cursor on Target (Cot) into NIEM.  I think my response may be of general interest:

I am not a believer in the “incorporation of external structures like 1512 or Cot in NIEM.”  The cost of merging the maintenance  and update process of multiple separate standards related organizations would be too high. The resulting governance process would induce such brittleness that progress would simply stop.   I am, however, a believer in using external standards in the IEPD process and in the use of external standards on their own when and wherever they make sense.  Developing a brand new NIEM structure to replace a well-defined standard that is already in use is ludicrous.  And I believe that the NIEM management team agrees with me when I say that.

There are actually four cases to consider that involve the use of external standards:

1. The external standard meets the mission of an IEPD on its own. In this case just use it.  Document it as the content of the exchange and do not spend time trying to “NIEMify” its content. (Examples are CAP1.1 and perhaps some of 1512).

2. The external standard is used to “wrap” NIEM content (likely case for EDXL-DE and Cot).  In this case again the standard is simply used to wrap a NIEM IEP (as defined in an IEPD for content).  There is no need to add the external standard elements into the internal content IEPD process.  The content IEPD should define how the external standard(s) could be used as a wrapping for transport, but that is different than defining the content itself and should be encapsulated in an entirely separate section of the IEPD from the content definition.

3. The external standard contains components that need to be used in an IEPD in combination with NIEM components (GML, KML, and perhaps 1512 are examples).  It would seem that we could simply add the appropriate namespace origin and mix them into our exchange schema with the content from a NIEM extension schema.  The problem here is that representations of these components are almost guaranteed to violate NIEM Naming and Design Rules (NDR).  For example: Even a simple type such as xsd:string must be converted to a nc:TextType, which is in turn a niem-xsd:string that acts as a proxy for the XSD type but adds an optional set of attributes to the string.  So, even a simpleType from schema becomes a complexType in NIEM.   Luckily, there is an out in the NDR called the adapter. An adapter is a simple, single element wrapper that is used in NIEM to contain the external component. It is defined in the NIEM NDR as follows:  “An adapter type is a NIEM conformant type that adapts external components for use within NIEM.  An adapter type creates a new class of object that embodies a single concept composed of external components.  A NIEM conformant schema defines an adapter type.” It basically adds the attributes at the adapter level that are needed for NIEM conformance without fussing with the representations and naming issues of its contained content.  An adapter can contain:
a) an entire external schema (assuming it wraps the root element type)
b) a complex component element type from a schema
c) a single type from the schema.
The use of adapters is described in section 7-7 of the NIEM NDR.  Serious users of NIEM should read the NDR in its entirety.  The final point on this discussion is very simple.  You do not incorporate an external structure into NIEM, but you can define a NIEM conformant adapter in your IEPD as a container for that external structure that will allow you to import the external structure “as-is.”

4. The NIEM is not complete and needs extension. There are concepts that are needed in NIEM that may exist in external standards already.  In this case the concepts (but not the standards themselves) actually do need to be “incorporated” into NIEM.  So, how do we bring in the concept without creating a duplication of the external standard?  We could “coordinate” with the external standards organization(s), each of which have their own representation for the concept. This is indeed a hard task, and creates brittleness in and out of NIEM.  It can be done to some extent, but it is not a good idea to put it into any form of formal governance.  The cost is just too high.  Luckily, NIEM already provides a workable solution: the use of Abstract Elements with Multiple Representations already exists for dates and units of measure.  This is extended to the use of external namespaces for code lists.  Why not do the exact same thing for concepts that originate in an external standard?
a) Bring in the concept.  Name and define it in accordance with NIEM NDR.
b) Identify the Concept as an abstract element with multiple representations.
c) Identify one of the representations as the one in the external standard using the external standard’s namespace. The NDR may require that it be wrapped in a single element Adapter.
d) Identify a NIEM compliant and/or other standard representations as appropriate.
This avoids the duplication of concept and organizes multiple representations of the concept in a manner that that both controls those representations and makes translation between representations easier to accomplish. Yet it does not induce “coupling” between NIEM and other standards organizations. The separation remains clear and clean.

Note: Items 1, 2, and 3 on this list are already in place as how you should work with NIEM.  I put them in this message as a way of organizing the different issues that may be in play. Number 4 is new.  It is a combination of my recent completion of the on-line Practical Implementer’s course for NIEM and discussions with FEMA’s Emergency management domain lead and a couple of Georgia Tech Research Institute (GTRI) representatives at the recent NIEM Business Architecture Council meeting in Baltimore.  I have also had off-line discussion with FEMA technical representatives and members of the OASIS Emergency Management Technical Committee.

A final note: Although they do conform to the NDR, not everyone likes adapters. They add one more tag to an already deep chain of tags in a NIEM compliant schema.  They allow the use of non-NIEM content (although it is content that is recognized by NIEM).  My take is that adapters make cooperation between NIEM and the external standards bodies possible without the intense combined governance that could have a negative impact on progress on all sides.

• Love the term "self-inflicted denial of service attack" to describe some security policy. #ogi #
• Interfaces for using standards based exchange of emergency information. See http://www.disasterhelp.gov/disastermanagement/open #ogi #
• DM-OPEN is simple. Multiple emergency management messaging standards supported by a total of two interface functions. #ogi #
• Good enough is Good enough. (most of the time) #ogi #
• For a view that is in contradiction to OGI Conference view of data.gov go to: http://www.ijis.org/EDblog/ — #ogi #
• Me experience is that competition often means "Who has the overhead space to absorb the proposal wins the competition." Smalls must sub only #
• #NIEM based Substitution Groups and abstract Representation elements can be used to avoid concept duplication when adding outside content. #

Powered by Twitter Tools.

• OPEN Alert, Cell Cast Alert – TEST ALERT ONLY!! High Water on Bull Run Road #
• OPEN Alert, Alert – TEST ALERT ONLY!! High Water on Route 1 at Rappahannock River Bridge #
• OPEN Alert, Previstar Alert -TEST ALERT ONLY!! – High Water on Route 620 at Kellys Ford #
• RT @trevbhatt thinks it is a dishonor to billy may's memory that the new oxy clean commercials are narrated by a pleasant-voiced woman. #

Powered by Twitter Tools.

I have been studying the National Information Exchange Model (NIEM) fairly extensively over the last few weeks. I have even read the NIEM Naming and Design Rules document from beginning to end. I will admit that I went into it with something of a jaundiced view.  As a veteran contributor to the DoD data model and an outside observer of the GJXDM (recently), and a large scale IBM model (a long time ago), I have real reservations about the usability and maintainability of any all-knowing, all-seeing model.  I have, at least at this point, become a believer in NIEM.  Why?   Because NIEM accepts the notion that a federation between separately name-spaced models makes sense, both within NIEM, and with external standards defined outside the heavy NIEM NDR discipline (or defined with a different heavy discipline).   The notion of defining an Adapter for NIEM use of other standards is a brilliant concept.  This, combined with the Information Exchange Package Documentation (IEPD) methodology for documenting the contextual use of data used in exchanges has made me a fan.

The problem with this “federation of standards” concept is that it makes tools (and “auto-magic” validation) harder to build.  As a result there is a tendency to try and force all of the standards back into the all-knowing, all-seeing model. It is a seductive idea, but not a good idea.  Let’s look at a very simple example: EDXL Resource Management uses the Customer Information Quality (CIQ standard) for Person Names. This allows internationalization for all kinds of different Naming structures and for a wide variety of Addressing schemes.  NIEM (as a national model) is much more U.S. centric, particularly in the use of PersonName tag. Both CIQ and NIEM are appropriate in their respective namespaces (and the NIEM NDR respects this fact by allowing for the adapter wrapper for external standards).   If we try to combine the two standards by defining CIQ elements as NIEM elements directly in order to make the subschema generator work more easily, we blur important distinctions that were developed for good reason.

So, we need to use NIEM IEPD methods. They are excellent. But we must resist the desire to force single definitions for concepts that may appear to be the same, but actually differ due to the context in which they were defined.  In other words, do not force a merger of conceptual domains, unless they actually are the same.  NIEM lets us federate in the building of an IEPD.   We should take advantage of that capability.

• For those with Bible study backgrounds: Reading the NIEM Naming and Design Rules is like reading Leviticus in the Old Testament. #
• @trevbhatt Worse than King James. Think of crossing of super geek data concepts with IRS tax law (like Leviticus mixes tax and life rules) in reply to trevbhatt #
• For those who may wonder based on previous tweets – I do think NIEM style IEPDs make a lot of sense. They actually make NIEM valuable. #
• Just completed the online coursework for the NIEM Practical Implementers Course. Good material. You need to know XML Schema well first. #
• The Capstone Exercise for NIEM appears to include an incomplete subset schema compared to what the instructions say. Makes it hard to do. #
• @NIEMExecDir My objective in taking the NIEM coursework and reading the entire NDR is to make EDXL/NIEM work together. – OASIS EM-TC founder in reply to NIEMExecDir #

Powered by Twitter Tools.

Interesting phenomena.  I put out a tweet likening the NIEM Naming and Design Rules Document to a cross between Leviticus and IRS regulations.  Almost immediately, I was followed by 1) a Bible Study Group asking for money 2) a NIEM consultant, and 3) a Tax Consultant.  Apparently they each have some sort of bot in place that looks for key words. They then follow in tthe hope of being followed.  It worked for the NIEM consultant, not the others. Shortly thereafter, I was folowed by ithe IJIS Institute, a justice related non-profit with a strong interest in NIEM.  I now follow them as well.

• Sitting in a Starbucks in Down town DC – waiting for a 1400 meeting to start. The drive home will be hell tonight. #

Powered by Twitter Tools.

• 5 hour rainstorm – no baseball – even though the week end was beautiful after wards. You would think they could fix the fields. but no…. #
• A tweeter asked "What is the opposite of time?" Answer: Time off. Time lasts forever. Time off is gone in a flash. #
• Working from the cafeteria at Walter Reed Hospital. Between Routine appointments. With today's cell broadband you can work anywhere..Darn!! #
• @Podkeyne Fresh live mussels and shrimp boiled in coconut milk with ginger, peppers, and mustard greens. in reply to Podkeyne #
• @Podkeyne they were alive when theyhit the coconut milk. in reply to Podkeyne #

Powered by Twitter Tools.

I wrote a poller for DM-OPEN that posts alerts received in DM-OPEN to the prototype Unified Incident Command and Decision Support System (UICDS).  This gives posters of Common Alerting Protocol (CAP) alerts the option of using DM-OPEN as a mechanism for also posting to UICDS for use by systems connected to that capability.  Two successful demonstrations to date: a month or so ago at the Virginia Department of Emergency Management and today in McClean, Virginia for some folks from DHS.  We posted alerts from NC4’s E-Team, CellCast’s Eagle, MyStateUSA, and DMIS Tools (also a DM offering) to UICDS where the alerts were provided to to a UICDS RSS feed and plotted on maps using Alert Sense (where proper locations were identified in the input CAP message).

The message:  DM-OPEN can be a “connection multiplier” for its interoperability partners.  In this case a single connection yielded 4 new partners (and possibly many more in the future).

It is always nice to know that you web services are up and running strong.  Since Twitter is the current rage, I though I would see if it could actually be useful. I set up a new Twitter account, wrote a poller to DM-OPEN that pings it on a regular basis and sends me a direct message to my grandpaham Twitter account upon the first instance of a successful ping,  the first instance of a failure following a series of successful pings, and the first instance of a successful ping after one or more failures.  Failures do not happen often, but now I will be the first to know if they do. Cool.