Your comments
Hi everyone,
To provide some further information. The only OMG standard protocol for DDS communication is RTPS/DDSI and the only "Platform Specific Module" for RTPS that has been ratified for RTPS is over UDP/IP. Some vendors, including RTI, do offer proprietary transports using shared memory. We disable this transport in our DDS build to maintain inter-vendor compatibility but if you'd like to create your own build and experiment with that transport then please do. More info here. I'd love to hear how it works out. Also I haven't had much success using the local loopback adapter but that can also be further explored if you'd like. We can contact RTI for further information. For the RTI implementation we are using all these settings are rolled up into the XML QoS configuration. There's reference material on configuring the RTPS (UDPv4) transport available here. In principle the loopback can be used but I haven't personally had a lot of success with it.
Regarding clocks we were talking just this morning about this in the lab. This is a rich and complex issue. DDS specifies a source_timestamp for each sample indicating when it was written into the Writer. In the past we have examined this value to gauge the time related to a sample but we are moving away from that in favor of specifying timestamps within our domain specific model. The reason being that these source_timestamp values should remain a detail of the middleware. In fact DDS can be configured to use a monotonic clock where the source_timestamp values are no longer populated from the realtime clock. More information on clock selection here.
Adding insult to injury in our QoS settings we are asking DDS to guarantee that samples are delivered in the order they were emitted at the source per the DesinationOrderQos. This means that DDS cannot allow samples to be written on the sending side out of time order. As a consequence a dramatic change in realtime clock values causes failures to "write" samples. This means we must at least 1.) ensure that ntp clock adjustments will not exceed source_timestamp_tolerance and at most 2.) consider switching to monotonic clocks across the board. In this way big shifts in the RTC won't interfere with the middleware layer. Alternatively we could change the system to use the default "BY_RECEPTION" DestinationOrderQos but we'll need to document it heavily that anyone consuming the data must cope with the possibility (even if remote as a practical matter) of data being delivered to a reader in a different order than it was written to the writer.
All that said to assign wall clock time to data from a medical device that emits no time markers (or too-low resolution time markers) requires an adapter with its clock set. For an embedded device (like a beaglebone) without connectivity to the internet at large you could try setting up a local host as NTP server. We're also going to experiment with battery-backed RTC add-ons for our beaglebones and we'll report back on our experiences. Another critical need for some absolute synchronized time is the need to correctly interleave data after the fact. To meet that requirement we demand that participants all have the means to attach meaningful realtime clock values to data. This as opposed to a potentially more complex solution in the spirit of Lamport.
I hope this information is helpful. I'm excited to be engaged in this discussion and welcome more feedback as we continue o evolve OpenICE.
Thank you
Jeff Plourde
To provide some further information. The only OMG standard protocol for DDS communication is RTPS/DDSI and the only "Platform Specific Module" for RTPS that has been ratified for RTPS is over UDP/IP. Some vendors, including RTI, do offer proprietary transports using shared memory. We disable this transport in our DDS build to maintain inter-vendor compatibility but if you'd like to create your own build and experiment with that transport then please do. More info here. I'd love to hear how it works out. Also I haven't had much success using the local loopback adapter but that can also be further explored if you'd like. We can contact RTI for further information. For the RTI implementation we are using all these settings are rolled up into the XML QoS configuration. There's reference material on configuring the RTPS (UDPv4) transport available here. In principle the loopback can be used but I haven't personally had a lot of success with it.
Regarding clocks we were talking just this morning about this in the lab. This is a rich and complex issue. DDS specifies a source_timestamp for each sample indicating when it was written into the Writer. In the past we have examined this value to gauge the time related to a sample but we are moving away from that in favor of specifying timestamps within our domain specific model. The reason being that these source_timestamp values should remain a detail of the middleware. In fact DDS can be configured to use a monotonic clock where the source_timestamp values are no longer populated from the realtime clock. More information on clock selection here.
Adding insult to injury in our QoS settings we are asking DDS to guarantee that samples are delivered in the order they were emitted at the source per the DesinationOrderQos. This means that DDS cannot allow samples to be written on the sending side out of time order. As a consequence a dramatic change in realtime clock values causes failures to "write" samples. This means we must at least 1.) ensure that ntp clock adjustments will not exceed source_timestamp_tolerance and at most 2.) consider switching to monotonic clocks across the board. In this way big shifts in the RTC won't interfere with the middleware layer. Alternatively we could change the system to use the default "BY_RECEPTION" DestinationOrderQos but we'll need to document it heavily that anyone consuming the data must cope with the possibility (even if remote as a practical matter) of data being delivered to a reader in a different order than it was written to the writer.
All that said to assign wall clock time to data from a medical device that emits no time markers (or too-low resolution time markers) requires an adapter with its clock set. For an embedded device (like a beaglebone) without connectivity to the internet at large you could try setting up a local host as NTP server. We're also going to experiment with battery-backed RTC add-ons for our beaglebones and we'll report back on our experiences. Another critical need for some absolute synchronized time is the need to correctly interleave data after the fact. To meet that requirement we demand that participants all have the means to attach meaningful realtime clock values to data. This as opposed to a potentially more complex solution in the spirit of Lamport.
I hope this information is helpful. I'm excited to be engaged in this discussion and welcome more feedback as we continue o evolve OpenICE.
Thank you
Jeff Plourde
Hi Nathan,
You're definitely not alone. I believe they used to call it "DLL Hell", if you'll pardon the expression. Unfortunately the community edition of RTI DDS does not include an installer that would ensure that these necessary redistributables get installed which I think would be the best practice deployment on windows. We'll try to document the need for that particular version more clearly. A pure java solution would be ideal.
Glad it's working now and please let us know if you have any more issues.
Cheers!
Jeff
You're definitely not alone. I believe they used to call it "DLL Hell", if you'll pardon the expression. Unfortunately the community edition of RTI DDS does not include an installer that would ensure that these necessary redistributables get installed which I think would be the best practice deployment on windows. We'll try to document the need for that particular version more clearly. A pure java solution would be ideal.
Glad it's working now and please let us know if you have any more issues.
Cheers!
Jeff
Nathan
You've already covered a lot of bases but there are a few more avenues you can explore to help debug the problem. It's true we no longer routinely test on Windows XP but I just downloaded and ran 0.5.0 on my WinXP VM so it ought to be possible.
Since the inception of the project a number of Windows users have had issues with linking in the DLLs due to the dependency on MSVCR80.DLL from the Visual C++ 2005 redistributable package. You say you've already got those libraries installed so the best way we've found to debug failure to link a DLL at runtime is the old reliable depends.exe. If you open nddsjava.dll in depends.exe it ought to tell you in a hurry what dependencies are not locatable on your system.
Which brings up the other point you mentioned. The default behavior of the app is to delete extracted libraries when the process ends as best it can to be 'polite' which may explain why you are not seeing the DLLs. If you have a JDK installed you can extract the libraries from lib/nddsjava-5.1-mdpnp-000006.jar with
Let me know what the dependency walk finds. I'd be surprised if it's not a failure to find MSVCR80.DLL but if all the dependencies are found we can start to look for other runtime irregularities.
Thank you
Jeff Plourde
You've already covered a lot of bases but there are a few more avenues you can explore to help debug the problem. It's true we no longer routinely test on Windows XP but I just downloaded and ran 0.5.0 on my WinXP VM so it ought to be possible.
Since the inception of the project a number of Windows users have had issues with linking in the DLLs due to the dependency on MSVCR80.DLL from the Visual C++ 2005 redistributable package. You say you've already got those libraries installed so the best way we've found to debug failure to link a DLL at runtime is the old reliable depends.exe. If you open nddsjava.dll in depends.exe it ought to tell you in a hurry what dependencies are not locatable on your system.
Which brings up the other point you mentioned. The default behavior of the app is to delete extracted libraries when the process ends as best it can to be 'polite' which may explain why you are not seeing the DLLs. If you have a JDK installed you can extract the libraries from lib/nddsjava-5.1-mdpnp-000006.jar with
jar xf lib/nddsjava-5.1-mdpnp-000006.jarand browse to
lib/i86Win32jdk/Alternatively I believe most file compression utilities (winzip and the like) ought to be able to extract the contents of the jar as well.
Let me know what the dependency walk finds. I'd be surprised if it's not a failure to find MSVCR80.DLL but if all the dependencies are found we can start to look for other runtime irregularities.
Thank you
Jeff Plourde
Hi Thomas
The warnings regarding bind errors ought to be non-fatal. In a more permanent deployment multiple participants on the same domain running on the same host can be explicitly set to bind to different ports. Alternatively what you're seeing is DDS 'discovering' that certain ports are already in use by another participant and moving on to use others harmlessly.
The subsequent unassignable data type warnings are more serious and probably are the result of the hello-openice project using our data structures as defined in version 0.4.3 to try communication with the 'demo-apps' bundle that you downloaded at version 0.5.0. Our data model is still subject to a lot of change as we experiment with different approaches. In version 0.5.0 we have marked all our types as 'MUTABLE' which should help going forward to increase compatibility between versions with changes to the data model.
I pushed changes to the hello-openice project to have it use version 0.5.0 of the data model (matching our current 'demo-apps' distribution) and I added a note about this to the README.md. If you 'git pull' the latest you ought to have more success.
While I was 'in there' I also added a second code path to the example showing the sometimes simpler case of receiving data availability signals directly from a middleware thread (instead of signaling your own thread via condition variables). In the lab we've also started experimenting with our own abstraction layers and I will try to include demonstrations of those soon.
Regards
Jeff Plourde
The warnings regarding bind errors ought to be non-fatal. In a more permanent deployment multiple participants on the same domain running on the same host can be explicitly set to bind to different ports. Alternatively what you're seeing is DDS 'discovering' that certain ports are already in use by another participant and moving on to use others harmlessly.
The subsequent unassignable data type warnings are more serious and probably are the result of the hello-openice project using our data structures as defined in version 0.4.3 to try communication with the 'demo-apps' bundle that you downloaded at version 0.5.0. Our data model is still subject to a lot of change as we experiment with different approaches. In version 0.5.0 we have marked all our types as 'MUTABLE' which should help going forward to increase compatibility between versions with changes to the data model.
I pushed changes to the hello-openice project to have it use version 0.5.0 of the data model (matching our current 'demo-apps' distribution) and I added a note about this to the README.md. If you 'git pull' the latest you ought to have more success.
While I was 'in there' I also added a second code path to the example showing the sometimes simpler case of receiving data availability signals directly from a middleware thread (instead of signaling your own thread via condition variables). In the lab we've also started experimenting with our own abstraction layers and I will try to include demonstrations of those soon.
Regards
Jeff Plourde
Hi Thomas,
This sounds like an exciting project!
RTI's DDS implementation allows for Quality of Service settings to be stored in XML 'profiles' (this is an extension to the DDS standard which allows QoS to be set only via API calls). By default the implementation will look for the USER_QOS_PROFILES.xml file (from which these profiles can be loaded) in the program's working directory. That is where the profiles for hello-openice are stored. Ensure that you are running the example program from the root of the project directory (which contains USER_QOS_PROFILES.xml) or move the xml file to wherever you'd like to run the program.
For deployment in the 'demo-apps' runnable package we've bundled these QoS profiles in the application jar which is an option we can discuss long-term as well for your project. I hope this helps. Let us know how your work progresses; and perhaps at some point as your plan comes together we can discuss your specific needs from the platform so we can incorporate them into our plans for OpenICE.
Thanks
Jeff Plourde
This sounds like an exciting project!
RTI's DDS implementation allows for Quality of Service settings to be stored in XML 'profiles' (this is an extension to the DDS standard which allows QoS to be set only via API calls). By default the implementation will look for the USER_QOS_PROFILES.xml file (from which these profiles can be loaded) in the program's working directory. That is where the profiles for hello-openice are stored. Ensure that you are running the example program from the root of the project directory (which contains USER_QOS_PROFILES.xml) or move the xml file to wherever you'd like to run the program.
For deployment in the 'demo-apps' runnable package we've bundled these QoS profiles in the application jar which is an option we can discuss long-term as well for your project. I hope this helps. Let us know how your work progresses; and perhaps at some point as your plan comes together we can discuss your specific needs from the platform so we can incorporate them into our plans for OpenICE.
Thanks
Jeff Plourde
Hi Vasanth,
Yes revision C will work. Our most recent purchases of beaglebones have also been revision C. Coincidentally in the lab last night we were working on a new disk image and some updates to device-adapter-setup.html to reflect the latest technology stack in use in the lab. We will push out those updates as soon as possible.
Thanks
Jeff
Yes revision C will work. Our most recent purchases of beaglebones have also been revision C. Coincidentally in the lab last night we were working on a new disk image and some updates to device-adapter-setup.html to reflect the latest technology stack in use in the lab. We will push out those updates as soon as possible.
Thanks
Jeff
That is great news Vasanth I'm glad the MP70 is working for you! In our lab we use an MP70 and an MX800 running revision J of the Philips software. We've had only limited opportunities to test the software on other models and software revisions. You are right that since they share the same data export protocol there is no explicit reason the MP5 shouldn't work. The protocol does allow for a lot of variation and it's possible our software is not handling particular protocol options used by the MP5. If you are using the default log4j settings the OpenICE 'demo-apps' package should be creating a 'demo-apps.log' in your working directory. If you'd like to send along that log after making an attempt to use the MP5 I could try to see if anything is obviously going wrong. If it is of value to you we can pursue the MP5 further.
Thank you
Jeff Plourde
Thank you
Jeff Plourde
Hi Rado,
We're currently using ubuntu 14.04.2 on the beagle bones in our lab successfully. Are you running on x86/x64 or ARM? With a UI or on the console?
Is there any message included with the Exception? A number of non-normal returns from the operating system "open" call will be thrown by PureJavaComm as a PortInUseException.
Some ideas that might help based on experiences in our lab:
Thanks
Jeff Plourde
We're currently using ubuntu 14.04.2 on the beagle bones in our lab successfully. Are you running on x86/x64 or ARM? With a UI or on the console?
Is there any message included with the Exception? A number of non-normal returns from the operating system "open" call will be thrown by PureJavaComm as a PortInUseException.
Some ideas that might help based on experiences in our lab:
- PureJavaComm expects ports to be specified without the "/dev/" portion. So if you are specifying an address on the command line do not include the "/dev/" portion ... for example /dev/ttyO0 would be specified simply as ttyO0
- we buy beaglebone RS-232 capes that are configured to route to the first UART which becomes /dev/ttyO0 in the OS, which, by default, in our ubuntu build has a console running on it. To disable the "serial" upstart service that provides the console we create a file called /etc/init/serial.override with the contents "manual". This overrides the service and configures it to not start on boot.
- When the OS creates the port it gives ownership to root in the dialout group. We usually place the user that runs the OpenICE software into the dialout group for this reason. Although if you're able to receive data via minicom this sounds unlikely.
Thanks
Jeff Plourde
Customer support service by UserEcho
We are trying to reproduce the problems you are having. With respect to the XML related error it would be helpful to know the classpath with which you are launching the program. Is it derived from the gradle dependency tree?
Thank you
Jeff