0

DDS Interoperability with commercial and open source based systems

Tim 11 years ago in DDS updated by Paul Ourada 9 years ago 13
MDPnP et al,
I wanted to find out where the focus in regards to the DDS would currently and in the future?
  • Will RTI-DDS and Prismtech-DDS be implemented together? 
  • or one or the other DDS technologies used provided by the above stated vendors?
  • Are RTI-DDS, Primstech-DDS and OpenDDS interoperable with each other?
  • Does OpenDDS not satisfy all the requirements compared to RTI and Primstech DDS technologies?
  • According to OMG specification the DDS is embedded on top of UDP or TCP, is that correct?
  • How is DDS different from RTP/RTCP/RTSP and IEC 61850-GOOSE, if we are trying to achieve and implement real-time communication layer?
  • Have you guys had any thoughts of looking into Transport Information Collection Protocol (TICP) http://planete.inria.fr/ticp/ on top of IP?
Thanks
Tim
+1
Under review
Hey Tim,

Lots of questions packed into one question here...
  1. "Will RTI-DDS and Prismtech-DDS be implemented together?" - Yes. The base standard of DDS is interoperable. The demo-apps package is built using the RTI DDS community edition. We have also used the PrismTech OpenSplice community edition to run apps on the network! Applications running both RTI and PrismTech community distributions were interoperable. It is the choice of the developer which distribution they would like to use. In the future state, OpenICE apps will be implemented using both distributions.
  2. "or one or the other DDS technologies used provided by the above stated vendors?" - The demo-apps distribution comes packaged with RTI DDS included. The user could replace the RTI libraries with any other vendor if interested.
  3. "Are RTI-DDS, Primstech-DDS and OpenDDS interoperable with each other?" - Yes, in theory. We have never used OpenDDS though because it does not support dynamic data types and has poor support for Java.
  4. "Does OpenDDS not satisfy all the requirements compared to RTI and Primstech DDS technologies?" - OpenDDS does not implement x-types, does not support dynamic data in their API, is tightly coupled with CORBA, the build process for Java is complex, and does not have the same support offered by RTI and PrismTech. It is very usable, it just lacks certain features we have implemented.
  5. "According to OMG specification the DDS is embedded on top of UDP or TCP, is that correct?" - DDS is an API for the  RTPS protocol which has standardized implementation using UDP.
  6. "How is DDS different from RTP/RTCP/RTSP and IEC 61850-GOOSE, if we are trying to achieve and implement real-time communication layer?" - What about these protocols interests you? DDS seems like a good fit for our purposes. 
    1. http://en.wikipedia.org/wiki/Real-time_Transport_Protocol
    2. http://en.wikipedia.org/wiki/RTP_Control_Protocol
    3. http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol
    4. http://en.wikipedia.org/wiki/Generic_Substation_Events#Generic_Object_Oriented_Substation_Events
  7. "Have you guys had any thoughts of looking into Transport Information Collection Protocol (TICP) http://planete.inria.fr/ticp/ on top of IP?" - Never heard of this; seems interesting. Thanks for the link. Is there anything about this that caught your eye?

Thanks,
Jeff
Hello Jeff,
Thank you for replying to my questions. I appreciate your quick response. You stated in item 3. that interoperability between multiple vendor DDS system did exist theoretically. I was more interested in the practical and real scenario terms. Has your group encountered any issues between RTI-DDS and Prismtech-DDS from the interoperability perspective?

  1. "Have you guys had any thoughts of looking into Transport Information Collection Protocol (TICP)http://planete.inria.fr/ticp/ on top of IP?" - Never heard of this; seems interesting. Thanks for the link. Is there anything about this that caught your eye? Well the architecture is very robust and the overhead is much less than DDS. Any bench test comparisons made between TICP and DDS would be really interesting?
Thanks
Tim
+1
Under review
Hi Tim,

The base standard for DDS is interoperable between vendors. We are constantly trying to inform the community of any issues with interoperability that we discover. For example, OpenICE no longer uses colons (::) in the Topic names because it is not approved by the standard and breaks interoperability. The intent of our current work is not to comprehensively test the DDS standard implementations. We do however test our system and ensure that DDS works.

I do not have a working knowledge of TICP - or any knowledge past what you can read on the link you provided. I'm not sure what aspects of the functionality DDS provides would be analogous to the functionality that TICP provides. I am confident enough in my understanding to say that they are different technologies with different purposes. DDS aims to provide an API into the underlying global data space, essentially abstracting many aspects of the underlying distributed system architecture. From my understanding, TICP is a transport protocol you can build an application on top of.

Our current development actions (6 month timeframe) are geared towards prototyping the system as best as we can understand it now. There are requirements that we are discovering as we build. If/when our requirements cannot be met by our current stack, we will research what tools will actually meet our needs. Frankly, there is no reason to rebuild work that has been completed successfully at this point. In the future, when optimization is a priority and we have a thorough and comprehensive understanding of what requirements our platform must meet, we may 'bench test' TICP among other transport layer protocols. Until that point, we will be using DDS.

If you have platform/system requirements in mind that you feel we do not currently meet, please please share them. We'd love to hear from you.
Thanks,
Jeff
Hello Jeff,
A lot of progress has been going on from the pub sub perspective as to which technology would be best suitable on a long term basis that is far less complex than DDS technology.

IHMO DDS is like CORBA. Too complex. A big company might embrace it, but they really don't have a reason to do so. With the existing protocols already well established, DDS doesn't bring "new" functionality to the table. DDS doesn't help the problem of connectivity -- it is too complex and configurable in incompatible ways. This is not the first technology product to be hampered by complexity. CORBA itself is a prime example. A little closer to home, 11073 suffers a similar problem. These things are still very useful, but they are not game changers.

One technology you might look at is ZeroMQ. It is one of a set of message brokers that includes better-known cousins such as ActiveMQ, RabbitMQ, Redis, even SOAP based systems.

ZeroMQ stands out in this crowd as modern, simple, and elegant. Not the universal solution for everything, but a good solution for some Pub-Sub type systems. Already well past critical mass, and continues to gain popularity. Here is an overview comparison:

http://www.bravenewgeek.com/dissecting-message-queues/

ZeroMQ is also data agnostic. It provides the infrastructure of message passing and subscriptions, but the data payload itself needs to be encoded separately. Protobuf is a good match for that, as well as Drager's SOA recommendations of binary encoding of XML messages using EXI.

Just my 2 cents.
One of the underlying pillars of our work here at MD PnP is to discover and document what requirements and specifications have to be met for a universal medical device interoperability platform to be adopted and successful. Enterprise is complex because the requirements are often complex. One requirement I certainly did not foresee at the outset of my tenure here was the requirement from medical device manufacturers (paraphrasing here) that the technology stack used shall be standards based. DDS is standards based; ZMQ et al are not (with the exception of MQTT which recently became an Oasis standard). The standards based middleware requirement, amongst other requirements related to distributed system functionality, initially informed the decision to prototype with DDS. The OpenICE platform is defined by more than a single piece of technology in the stack - it aims to be a lot of things to a lot of people. As our prototyping efforts continue, we mount evidence for and against the requirements and obviously change technologies when they do not work. For example, initially the web demo was backed by Apache Camel but as the site was developed, Camel collapsed when trying to serve multiple realtime waveforms. The tech was swapped out for Node.js which has been working great.

We are not married to DDS, but in the near term (6 month time frame) for us to walk away from what has been developed already would be a massive undertaking and require a very strong case to justify it. This case would be composed of requirements, use cases, and/or functionality that could not be met or provided by the DDS layer of OpenICE. There is not enough evidence against the use of DDS to motivate us to stop what we are doing and rebuild the platform with a different middle layer. I'm encouraging you, and anyone else reading this, to provide input into what requirements and functionality they expect a medical device interoperability platform to meet and provide respectively.

"I require the platform to have message latency of less than 10ms between an x-ray unit and a ventilator so that I can synchronize the radiation exposure to the phase of respiration of the patient." This is kind of input I dream about because we can derive a lot of requirements from it about the OpenICE platform, the hospitals infrastructure, the ICE application, etc. Help us ensure that what we are building will meet the community's needs.

You eluded to but did not directly state a requirement that applications developed to run on the OpenICE platform should have minimal technological complexity to ease adoption. This is a very good point and one we have been considering internally. I would love to see us develop an application host environment that is decoupled from the middleware layer and/or transport layer protocol to provide exactly the minimal complexity you mention. The challenge is doing so in a way that allows the application developer to leverage the full power and potential of the platform without abstracting away too much. This is something I can't wait to work on! Exciting stuff...

With respect to bench tests: we could run bench tests for every message queue system (ZMQ, RabbitMQ, etc) and separately test every distributed system middleware (DDS, etc) but what message latency is acceptable? What raw message throughput is acceptable (at what average message size)? What requirements does this place on the infrastructure? Can we minimize the network requirements to maximize adoptability? (Remember that many small to mid sized hospitals have very limited infrastructure, if any.) We are prototyping to explore and document what requirements and specifications are required. We need reason for our development efforts and have to justify what we build with evidence, not anecdote.

I am not here to defend DDS. For the record, from my initial exposure to ZMQ, I like it a lot. I'd love to spend some time exploring it past what you can simply read on the wiki page - but it does not fit into the current iteration of OpenICE. Down the road when/if we find reason that the DDS piece does not fit into the OpenICE puzzle, we will change it. Again if you think that time is now, please please help us inform the initial prototype of OpenICE with requirements that are not being met.

Your initial question was about "DDS interoperability with commercial and open source based systems". We have a requirement that requires the middleware implementations be interoperable. If we find this to not be true with no obvious solution, this would be a strong piece of evidence for us to move away from DDS - but we haven't encountered this.

I hope this helps,
Jeff
As an addendum and for the community's record, the author of the benchmarking blog post linked in the previous post deprecated his code and results. He recently published a follow up post about "Benchmarking Responsibly" and advised that there are "different use cases for different systems". A worthwhile read: http://www.bravenewgeek.com/benchmark-responsibly/
Jeff,
Thank you informing all of us. The point I was trying to make specifically towards that Kafka, ZeroMQ and other technologies that have a lot of potential and are far easier to implement. It would be note worthy that we should be looked into these technologies that are completely open source, easier implementation on a large scale. I think it would be beneficial compare side by side technologies such as RTI-DDS, Kafka and ZeroMQ.
Thanks
Tim
Hello Jeff,
Below are some of my suggestions for the OR requirements that I would like MDPnP to consider.

Compatibility of Medical Devices connectivity
  • Plug N Play Recognition
  • On demand Hot swapping of medical devices
  • Auto-Recognition/Auto discovery of medical devices in each OR suite. (able to display both connected and unconnected devices)
  • Development of ubiquitous medical device driver interfaces.
  • Pluggable device specific interface library enabling easy support for varying device protocols and data formats
  • I/O communication over Wi-Fi, Bluetooth, USB, FTP, LAN, CANBUS and serial port based protocols
Patient Safety
  • Smart Alarms and messaging Alerts
  • Unique/novel alarms and alerts for Patient monitoring in OR setting
  • Versatile to allow clinician preferences
  • Compatibility of devices
  • Interlock equipment to ensure compatibility and reliability
  • Smart maintenance
  • Support for device enabled events (e.g. battery, supplies, etc.)
  • Monitor device connectivity per OR suite
  • Monitor overall system performance (network, congestion, latency etc.)
  • View status remotely of all connected devices
  • Rapid IU display of root cause of any issues
  • Alert devices status (device connection, online/offline etc.)
  • Automatic device detection, connection and data acquisition without user intervention.
  • Notification when device is unplugged or moved away from the radio range and automatic re-connection when device enters the range
Research Tools
  • Facilitate collection of desired data (numerical and waveform data)
  • Filter clinical raw data per user requirements and specifications
  • Close loop control system of medical devices
Improved methods of data display
  • Novel methods for displaying physiology data from devices and data from other sources
  • Real-time remote display via other platforms (handheld devices)
  • Automatic emergency paging system (Communication System)
  • A Comprehensive, accurate, reliable database with relevant data for clinical research purposes (retrospective analytics)
  • On demand real-time access to data (waveform/numerical) by remote multiple users
  • To Synchronize, integrate, and process medical data from all sources
  • Allow clinicians to index, search, and assemble data.
Software System Architecture
  • Ensure it is flexible, modular, scalable and evolvable
  • Time validation and synchronization with raw data
  • Enabling unified feed, frequency selection, parameter & unit mapping, unit conversion and data manipulation
  • Flexible clinical systems connectivity with system-specific data management interface library
Communication 
  • A method of communication between clinicians and care givers inside the OR and outside the OR vicinity
Hope this helps.

Thanks
Tim
Hello Jeff,
To answer to your previous statement. I'm content to see MDPnP group do whatever they want. However I would suggest to implement multiple solutions and compare the results to understand the true potential of technologies available at the prototype level. We should not be married to a single approach but be able to embrace alternative and improved technologies that should be considered and compared based on their functionalities, capabilities and implementation. Hence a true comparison on each technology should be reviewed before a final adoption of a particular technology has been decided. It's fine to embrace a single technology (such as DDS) for your particular use by the MDPnP group, but to endorse it for the entire community should necessitate a wider view. I think other communities such as the federal agencies should have been consulted.

To my knowledge most technology exists and enjoys widespread use before a standard is published. Standard is an end goal, not a starting point. Take for example Linux which has no standard's body behind it, nor does Java, C, C#, or x86. We should be more focussed on technologies that provide full availability of the technology details and ability to use, modify, and redistribute among the community.

I do not think it is possible to attain a goal of of 10ms between off the shelf commercially available ventilator and x-ray/CAT/MRI scanning machines. Because the ventilator itself as to my knowledge spits out a waveform with a latency of >100 msec+. May be you aware of any manufacturer that provides ventilator/s with waveforms at less than 10msec? However you might be willing to align the signals by artificially introducing latency into the x-ray signal as well. This would be a calibration procedure which I am not sure if the scanning manufacturers would be interested. Again this would not be so called true synchronization.

Thank you,
Kind regards
Tim

Yes! I'm glad you understand. We do need to implement multiple solutions of the different components of the system and compare them - what we are doing now is building the rest of the platform and deriving the metrics that we should them compare against! Exactly as you said: technologies "should be considered and compared based on their functionalities, capabilities and implementation". The middleware, or transport layer protocol plus API, is one single piece of a large puzzle. We need to understand the interactions and requirements amongst all these pieces by building a prototype before we can optimize any one given piece. As I said before, "if ... the DDS piece does not fit into the OpenICE puzzle, we will change it." We can't do so blindly.

You make a fantastic point (that I agree with) about standards-based technologies. I brought that point up previously to inform you about the original decision to use DDS. Companies that we would be dependent upon for adoption are asking for this even today. We cannot ignore that request. However, as you undoubtedly understand, we can build an intelligent case to justify an alternative approach.

With respect to x-ray / ventilator synchronization - your hesitation is exactly why we need to understand all of the use cases of the platform so we can inform the design decisions of future technologies. This is one of MD PnP's key goals! None of what we are building today will be possible without influencing design decisions of both medical devices and regulatory pathways. By the way, we have a simulated version of this use case that we have presented at conferences and prototyped in our demo-app on mdpnp.sourceforge.net. I don't know offhand how precise or accurate the synchronization is (in milliseconds) but it does work well with a simulated x-ray and Draeger V500.

Also thank you for your list of functionality you want to see incorporated into OpenICE! This is the beginnings of some fantastic input. Is it okay if I follow up with you to further refine these requests into use cases, requirements and specifications? I have some time allotted to this activity from several other sources and can add you to the list.

Again, I'm glad to see that you're on my page and that we are both focused on creating the best possible medical device interoperability platform possible. I look forward to discussing your functionality request with you further.
Thanks,
Jeff

Hi Jeff and Rado -


Interesting discussion. The push to standardization is due to large company desire for stability. But it is not only they, but the medical facility themselves also desire this stability. Medical facilities, barring research facilities, are pretty allergic to supporting N different communication schemes. While most infrastructures are capable of supporting the multitudes, the people running those infrastructures must provide support for them all, and trying to understand impacts of various communication schemes becomes every more complex. It's much the same stance as company IT departments who don't want to support non-standard equipment or SW for the vast majority of people (I *do* take exception for R&D personnel and machines, and believe that companies should be prepared to support R&D separately).


DDS is highly attractive from so many perspectives - the toolsuites for them are quite robust and attractive and cost no more than Visual Studio/MSDN subscriptions. So device companies will be better able to develop and deploy systems. Such tools often have multiple modes, allowing novice to expert developers to participate. Compare this to less mature solutions which often require advanced expertise, or heroic efforts to implement. So, this becomes extraordinarily expensive from a business perspective - you may only need one or two experts, but what if they leave (for any reason )? Now what?


Now, you may think that I'm a dyed-in-the-wool corporate weenie. I can tell you that I'm as much an early adopter as the next Linux geek - having been involved almost since its inception - and a huge open source proponent. I have architected an embedded Linux medical device using the yocto distribution builder. However, I learned after 25 years that playing the "I can write that program in N lines" game doesn't play well in most companies - barring startups. Learning how to navigate in a company is just as important as knowing how to write pristine code.


So, by all means, investigate, but be prepared to be *scrupulously honest* in representing your results, and cover far more than how a solution is less complex, faster, or how "anyone who knows anything about programming" can do the implementation or support it after you have moved on to the next project or company. Be sure to show how using the technology you're championing can be supported by a team comprised of the average engineer, plus one or two experts. And be sure to include a plan for sequestering and patching/updating the technology so that products dependent on it don't get stuck with unsupported versions and become vulnerable from a security perspective.


Of course, as you two have already mentioned, sometimes there's no choice but to venture off into the unknown to get a project delivered. And hell yes, that's fun! :)


Best Regards,

Paul

Hello Jeff,
Below is a recently published article that might be an interest to your group.
Using machine-to-machine/"Internet of Things" communication to simplify medical device information exchange
 by Christian Peterson from University of British Columbia, Vancouver, Canada.

using MQTT protocol instead of using DDS. 

Cheers
Tim

Hello Jeff, I`m new in the community,I`m from Mexico, i have many doubts about of DDS and your platform OpenICE. I`m working in your platform and I`m trying to understand how it works, but i don`t found how read the device data for the device-adapter and as the data is transmitted to the supervisor. I was reading about RTI Connext but I don`t know as this software is related with OpenICE. I already did the test with Hello-openICE but i didn`t saw as included the libraries of RTI. I want to do my own program but i don`t know how include RTI. I hope you help me. Best regards,

Arely Carballo