Grandpa Ham – Grandpa Can do IT!

I wrote the following Comment on a LinkedIn NIEM Thread and thought it might be worth sharing:

In my opinion (perhaps not so humble :-)) NIEM is very good if you use it right. But, if you think of NIEM as a standard by itself, it is NOT GOOD. Too much stuff. Hard to keep organized. It is actually incredible how well organized it is given its volume, but as a standard by itself, it is just too BIG to be anything but unwieldy.

Now, as a model for building standard exchanges, it has TREMENDOUS value. It is like a mine. You have to refine the ore through an appropriate standards development process. (The IEPD process works for this.) But the ore is super high quality when compared with building standards from scratch, because the NEIM model has been preprocessed in the sense that most definition and relationships are well defined and understood. Even so, you still have to refine NIEM input using the IEPD process to actually create a usable exchange or a useable standard exchange.

Bottom line, if you expect NIEM to be “auto-reusable,” you will be disappointed. But, If you use NIEM appropriately, it has a lot to offer in the way of documented data structure and definition.

There have been at least two situations that I have heard of recently that claim NIEM Compliance and External Standards Conformance in the same breath. While it can be done, neither actually did so. In one case, NIEM elements were mixed into non-NIEM schemas, but the NIEM attributes were removed. This is clearly not the correct approach (although at least NIEM concept re-use was achieved). In another approach, external standard concepts were “NEIMified” in a tool and mixed with NIEM elements in a combined IEPD without the use of adapters. This approach is NIEM conforming, but NOT standards conforming. This approach can claim to use input from existing standards but cannot claim to actually use those standards. Especially, this approach cannot claim to conform to them. I firmly believe in NEIM as a methodology and as an excellent model for concept re-use, but building a NIEM conforming schema is not the same as building a standard. A standard involves acceptance of the exchange schema by a formal standards body or by a wide body of users as a reusable exchange. When you build a NIEM complient IEPD you have 3 options:

1. Build it as a NIEM IEPD and ignore other standards.
2. Build an IEPD and use pieces of external standards but ignore validation or conformance to the the standards that are reused.
3. Build an IEPD with standards conforming components in adapters and add any other needed data using NIEM want list methods or current NIEM elements .

In all three cases, the result is not a standard until it is recognized as one, either de facto or through a recognized standards body. So, the IEPD is just the beginning. Before it becomes a standard, it must be recognized, either by a formal standards body, or through industry dominant re-use.

Bottom line: If you choose path 2, you should not advertise it as standards conforming. While you can claim NIEM Conformance, you cannot honestly claim conformance to the external standards used as input in any way whatsoever.

Attended combined NIEM National Training Event and Oasis Interoperability Summit in Baltimore last week. What a week!!

All of the following Items are from that event:
1. Participated in live demonstrations of interoperability by 11 separate commercial vendors, all using the DM-OPEN Backbone. Messages included Common Alerting Protocol sent from an actual Chorine sensor, NWS Tornado Warnings in CAP with the full polygon showing on maps used by multiple vendors, EDXL-DE wrapped Hospital Availability Messages, and EDXL-DE wrapped NIEM Amber Alert Messages, with accompanying Style sheet and reference base-64 encoded picture data used in full display. A professional videographer filmed the demonstration activities and interviewed key players. The edited video will be made available by OASIS. I will post the link when it is available.
2. Moderated NIEM NTE panel titled “Coordinating the Development and Adoption of Emergency Data Standards With the Ongoing Development of NIEM.” A format of 5 separate questions with short answer to each question by all panel members in turn was well received, both by the panel and the audience. Answers were lively and interesting. There were many audience questions as well. The NIEM organization recorded all panel sessions, so this panel will be available for review in its entirety.
3. Acted as a panel member in a second NIEM NTE panel titled: “Playing Well With Others”—NIEM and External Standards. This was a half session panel that stirred lots of interest and did not afford adequate time for all audience questions. Its recording will also be made available by the NIEM organization. Both panels made is clear that there is real cooperation between standards bodies and progress is being made to ensure that the value of all standards is recognized as a federation real capabilities. While some technical and “turf” issues need to be understood better, the folks involved look forward to the future with a positive attitude and a real belief in success.

Special Thanks to Donna Roy (NIEM Director) and her crew for a great event, and to Bill Kalin (Contractor to DHS Science and Tecnology) and Jane Harnad (OASIS) for organizing a superb demonstration and to all of the vendors for showing real interoperability in action. Standards do work!!!

I have been studying the National Information Exchange Model (NIEM) fairly extensively over the last few weeks. I have even read the NIEM Naming and Design Rules document from beginning to end. I will admit that I went into it with something of a jaundiced view.  As a veteran contributor to the DoD data model and an outside observer of the GJXDM (recently), and a large scale IBM model (a long time ago), I have real reservations about the usability and maintainability of any all-knowing, all-seeing model.  I have, at least at this point, become a believer in NIEM.  Why?   Because NIEM accepts the notion that a federation between separately name-spaced models makes sense, both within NIEM, and with external standards defined outside the heavy NIEM NDR discipline (or defined with a different heavy discipline).   The notion of defining an Adapter for NIEM use of other standards is a brilliant concept.  This, combined with the Information Exchange Package Documentation (IEPD) methodology for documenting the contextual use of data used in exchanges has made me a fan.

The problem with this “federation of standards” concept is that it makes tools (and “auto-magic” validation) harder to build.  As a result there is a tendency to try and force all of the standards back into the all-knowing, all-seeing model. It is a seductive idea, but not a good idea.  Let’s look at a very simple example: EDXL Resource Management uses the Customer Information Quality (CIQ standard) for Person Names. This allows internationalization for all kinds of different Naming structures and for a wide variety of Addressing schemes.  NIEM (as a national model) is much more U.S. centric, particularly in the use of PersonName tag. Both CIQ and NIEM are appropriate in their respective namespaces (and the NIEM NDR respects this fact by allowing for the adapter wrapper for external standards).   If we try to combine the two standards by defining CIQ elements as NIEM elements directly in order to make the subschema generator work more easily, we blur important distinctions that were developed for good reason.

So, we need to use NIEM IEPD methods. They are excellent. But we must resist the desire to force single definitions for concepts that may appear to be the same, but actually differ due to the context in which they were defined.  In other words, do not force a merger of conceptual domains, unless they actually are the same.  NIEM lets us federate in the building of an IEPD.   We should take advantage of that capability.

Interesting phenomena.  I put out a tweet likening the NIEM Naming and Design Rules Document to a cross between Leviticus and IRS regulations.  Almost immediately, I was followed by 1) a Bible Study Group asking for money 2) a NIEM consultant, and 3) a Tax Consultant.  Apparently they each have some sort of bot in place that looks for key words. They then follow in tthe hope of being followed.  It worked for the NIEM consultant, not the others. Shortly thereafter, I was folowed by ithe IJIS Institute, a justice related non-profit with a strong interest in NIEM.  I now follow them as well.