IJIS data exchange standards

The history of interoperability in law enforcement information systems is instructive and provides the basis for a ‘model of pitfalls’ that must be avoided as industry wide data exchange standards are written and adopted.  When NCIC created the first message based transaction handling system there was a single set of communications protocols and data exchanges that described how states would interoperate with NCIC.  As state agencies moved to implement NCIC access, there emerged a wide variety of in-state telecommunications protocols and data standards based mostly on the proprietary communication protocols of the company from which the state acquired its hardware and system software.  The result was that companies engaged in supplying information systems had to develop many different data interfaces to handle the variety of protocols and data standards that were implemented.  The problem was compounded at the local level, where another layer of conflicting protocols and data standards were adopted at the county level, to which municipal and city police systems were to be connected.  A company active in this industry was forced to create as many as 85 separate and distinct communications protocols and data interchange standards.  The path taken by some states made it nearly impossible to implement data exchange capabilities, and there are still many situations where interoperability is, at the very least, prohibitively expensive.

In some states, the programs written to permit communications within the state were not fully compliant with even the industry standards at the time, having been constructed by individuals who for various reasons chose to only partially or not at all implement current industry standards.  In these states, the cost of creating customized protocol handlers and different data formats was then borne by the local agencies who paid companies considerable sums to create the unique interconnection required by the state.  Companies have gone out of business after suffering significant losses in attempting to making these interfaces work.

If the standards to be recognized for integrated justice information exchange are not done differently then in the past, the same waste will occur, and the justice agencies will suffer both the expense and the complexities of integration that will result in delays, performance problems, inflexible applications and ultimately dissatisfied end users. 

The changes that have occurred  in industry over the past 30 years is remarkable in that today’s technology drivers take into serious account the need for shared, open industry driven (as opposed to vendor driven) standards on which to base system interoperability.  The technical standards being created today across all industry sectors have the joint backing of many companies that have recognized that information exchange and the resulting gains in productivity and efficiency are at the heart of improved system performance.  

It is also important to recognize that the law enforcement and justice community does not drive underlying technology to the extent that commercial enterprises do.  We seek instead to leverage the massive research and development efforts of the information technology industry and find ways to apply the resulting technology to the justice systems we build.  Accordingly, it is imperative that the construct of standards takes into account the emerging technologies that will shape the future of information technology, irrespective of the industry to which they are applied.  In other words, integrated justice information exchange standards will inevitably be built on top of commonly accepted and supported information technology standards.

In this context, the web and the technologies it has spawned is irrevocably linked to the future of integrated justice information systems.  The IJIS Industry Working Group, among others, has forcefully concluded that the vision of integrated justice information systems is that of a virtual system rather than a monolithic host computer under which agencies must sub optimize their system designs to accommodate the lowest common denominator in satisfying user agencies.  As we seek to encourage and promote the integration effort in conjunction with the vision of virtual systems, the emerging web based technologies will play a key role in shaping the future. 

Having drawn this conclusion, it appears that the standards that must be developed and adopted should build upon the foundations of common industry technology in such a way as to exploit these underlying principles.  To do so requires that integrated justice standards are at least consistent with and do not conflict with the way that the information technology industry is implementing these standards.  Further, the work that has been done and is ongoing in developing information exchange standards should be used as a basis for building industry specific standards such as those proposed for integrated justice systems.  Finally, the integrated justice interchange standards should anticipate where the technology is going rather than where it has been as agencies such as OJP and Global attempt to forge standards that will remain strategic.

The most notable and widely adopted work in the development of standards has been and is the World Wide Web Consortium (W3C), which has, at its heart, the objective of building open (non-proprietary) standards for technology that will facilitate information system interoperability and the exchange of information.  Given the breadth of companies who rely on the W3C for standards development, and the proposals that are being submitted now for future adoption, it is clear that there are several trends which should be considered as a technological foundation on which integrated justice data and communications standards should be based.

There are three emerging technologies that are shaping the work of most of the nation’s future system software.  These technologies should be used as a basis for building new standards unique to the justice industry.  They all fall under the general category of using XML as a basis for intersystem communications and data exchange.  These three important trends are:(1) the use of XML schemas to reflect the definition of data types,  (2) the use of emerging XML protocols for defining the envelope for interchange, and (3) the introduction of web services as a model for integrating systems.

The final proposed recommendation of the W3C on XML schemas  were approved in 2001, and its adoption as a recommendation indicates its feasibility.  It is highly likely that most of the information technology industry, in any attempt at system interoperability, will embrace this standard and use it upon which to create more robust ways to exchange information.  Standards developed for integrated justice information should be based on extending the XML schema standard to the specifics of integrated justice.

There has also emerged a standard protocol for exchanging XML based data in the form of what is now being called XML Protocols (aka SOAP).  The history of how SOAP has been transformed into a standard is interesting from the perspective of the ways in which industry participants overcame proprietary interests and ultimately concluded that the adoption of an open protocol standard would be in the best interests of companies and customers. The story is well documented by Don Box in his history of SOAP .  The adoption of this open standard in integrated justice would further accelerate the development of systems and would in the end lower costs to agencies. 

One of the reasons that SOAP is an important part of the eventual standards that will facilitate the construction of integrated justice systems is that the W3C has recently received a proposal for creating a standard for public key management that will make it much easier to introduce the security required for justice information exchanges particularly as the stakeholders expand beyond conventional criminal justice agencies.  A public key infrastructure is already acknowledged as a basic and ongoing requirement for developing trusted systems that exchange information, and this security issue is paramount in the development of systems that exchange information in the justice world.  The proposed standard for public key management will build on the standards that the W3C has adopted for XML schemas and SOAP .

Given the broad support for the development of web services by member companies in the Industry Working Group, and the strong desire of justice agencies to take advantage of web technology, standards developed for the exchange of information among agencies should also anticipate the use of web services.  Once the standards are defined, there exists the potential for state or local agencies to embrace this technology and create web services that can quickly and easily incorporate criminal history checks, fingerprint transmissions, and database interactions that very much support the NASIRE architectural model which has been widely endorsed.  In order to take advantage of this emerging technology, standards developed for the justice information exchange purpose should anticipate the existence of web services by incorporating reference to the UDDI (Universal Description, Discovery and Integration Service) standards. 

While UDDI is not yet an open standard, the W3C is working on a variety of specifications that will eventually allow agencies to programmatically embed web services and create interfaces to external and legacy systems with minimal effort.  Both Microsoft and IBM have endorsed the development of open standards for UDDI which is likely to result in wide spread adoption of this model for registering web services.  As state departments of justice and local justice systems open their repositories to public access, the UDDI model will provide a basis for easy information exchange.  It is likely that UDDI will provide a structure for making web services available in conjunction with the SOAP model. Even in a limited access secured environment such as a state-wide integrated justice information system, the UDDI model would make it much easier to introduce web services and make them available to local systems.

In these days of internet time rather than conventional time, the time it takes to develop and gain acceptance of a standard is longer than the cycle time of information technology revolutions, and it is therefore important to build on the likely technologies that will persist as standards are written.  Otherwise, they will be obsolete before they are published.  Aiming at the future is harder, but more productive if the end point can result in standards that live longer and tolerate change.

Paul Wormeli

June, 2001

© Copyright 2005 Paul Wormeli. Last update: 3/16/2005; 9:25:19 PM.