Improving AI with Simulation tecHnologies.
MIT Technology Review has included Reinforcement Learning as a top 10 breakthrough technologies for 2017 in its latest issue.
What is reinforcement learning? It is about to let machines learn by experimenting. As it is explained in the magazine, reinforcement learning copies a very simple principle from nature. The psychologist Edward Thorndike documented it more than 100 years ago. Thorndike placed cats inside boxes from which they could escape only by pressing a lever. After a considerable amount of pacing around and meowing, the animals would eventually step on the lever by chance. After they learned to associate this behavior with the desired outcome, they eventually escaped with increasing speed.
High fidelity simulation is going to be a key lever to reinforcement learning
Reinforcement learning algorithms can help, for example, to improve the "driving skills" of self-driving cars. Today’s driverless vehicles often falter in complex situations that involve interacting with human drivers, such as traffic circles or four-way stops. AI engineers can collaborate with simulation engineers to integrate a digital model of the driverless car in a simulated environment, replicating the most complex traffic situations without any risk for people and real traffic. In the virtual space, the control software can perform the maneuvers over and over altering its instructions a little in each attempt. Applying deep learning techniques, the system can extract the patterns from the best performed maneuvers, learning from experimentation.
We see then that high fidelity simulation is going to be a key lever to reinforcement learning but, what kind of simulation we need to do an effective learning? In a former post, we have already discussed about the many applications that simulation brings to the development of smart and connected devices, including the virtual experimentation and design of IoT's products. In that article, we were also discussing about the kind of simulation that it is needed to integrate in an effective way digital models of the new products in simulated environments: we need to work with Net-Centric and interoperable models and simulations. In the same way that the physical product evolves to the Internet of the Things, its digital model will need to evolve to an Internet of the Simulations or IoS, in which heterogeneous simulations of the diverse physical and cyber subsystems of the smart product can interoperate without restrictions.
The Internet of Simulations will allow to reinforce the learning of "smart" products by integrating a digital model of the smart product and its AI software in a complex virtual space.
But to evolve actual simulation products and solutions to the Internet of Simulations, several challenges needs to be addressed, especially in the technologies and architectures to integrate all kind of simulations and real system in a common virtual space. We are investing in new simulation technologies for the IoS and the result is our Simware platform. We are also collaborating with lead research groups as the Distributed Systems and Services Research group of the University of Leeds in the development of new simulation architectures and technologies for the Internet of Simulations. Staff and researchers in DSS group are doing very interesting projects related to the application of the Internet of Simulations to the virtual design and experimentation of new vehicles, included self-driving vehicles.
If you are interested in collaborate with us to improve the integrability of cyber-physical systems in simulated environments, send us an email to firstname.lastname@example.org. or write a comment here in this post.
One of the main challenges to solve before we can evolve real time distributed simulations to the network, fully connected in the Internet of Simulations, it is how to deal with in an effective way with multiple architectures and protocols. Lead users of distributed simulation, the military simulation & training community, has been working to solve this huge challenge for a long time and the main solutions has been mainly focused on the development of complex gateways solutions that connect the individual simulators' architectures. Gateways based solutions are hard to scale and introduce many restrictions in the interoperability, because the flow of information between the simulators is limited to the capabilities of the gateway.
This problem has been already a big one when trying to connect only a few man-in-the-loop (virtual) simulators in a common synthetic environment but it is becoming in a much bigger one now that simulation is also requested to integrate with real and live systems in the network. In this case the potential number of architectures and protocols involved is enormous when you compared with the few protocols and architectures in use in the military sim & training domain. For example, the Internet of the Things is opening new exciting opportunities for the real time simulations, as a type of Cyber Physical System or CPS connected to the IoT compliant systems. (to know more take a look to my former post in this blog).
To solve this challenge, in Simware, we are taking a more holistic approach to this problem and we are founding our technology in our Data-Centric and Layered Simulation Architecture : LSA. LSA is the first distributed simulation architecture designed to meet the specific simulation requirements of IoT systems dealing with multiple architectures and protocols.
LSA will not only allow to expand the applications of real time simulations to the IoT but also will enable new business models for the traditional markets of simulations as flight simulation as it is discussed in this blog.
Our Simware product-line leverages LSA to converge multiple standards and protocols in a common data-centric simulation platform. Simware, just out-of-the-box, is offering integration between DDS, HLA and Web applications, but it can be extended easily using its APIs to support other protocols and standards, as for example JAUS to connect with robotic systems (to know further about the capabilities provided by this kind of integration take a look to the Citius use case). You can find many resources in our website to know more about LSA & Simware but a good starting point is this post in the blog.
Are you having issues dealing with multi-architecture distributed simulations? Do you want to move toward the Internet of Simulations? Please contact us and we will help you to find the right solution to your pains.
General Manager Simware Solutions
I have read an interesting paper included in the volume 9848 of the SPIE proceedings , "Modeling and Simulation for Defense Systems and Applications XI". This paper (you can downloaded here proceedings.spiedigitallibrary.org/volume.aspx?volumeid=17674) titled "Internet of the Things, a possible change in the distributed modeling and simulation architecture paradigm" by Mark Riecken, Kurt Lessmann and David Schillero, proposes to consider LVC simulation as a type of Cyber Physical Systems or CPS in the Internet of the Things (IoT) as defined by NIST (know further about CPS concept at www.nist.gov/cps/ ).
Authors recognized many similarities between LVC simulation and IoT/CPS and propose a closer collaboration between both communities to improve both. LVC Simulation can benefit from IOT-/CPS to refresh and sustain its core technologies, because much of the technology employed at LVC simulation was developed prior to the proliferation of internet. IOT/CPS can leverage LVC simulation to experiment and test complex scenarios in synthetic playgrounds. Paper presents two specific use cases of IOT/CPS that would benefit from distributed simulation :
At a conclusion, authors are proposing to expand the collaboration between both communities using special sessions or forums at SPIE and similar venues that could evolve to permanent structures in which both communities could work together on common protocols, standardization processes, shared data models, LVC in CPS and modeling cybersecurity. This paper uses as references of the work to be done our study group at SISO for the Layered Simulation Architecture and the integration of different standards in our Simware platform.
We do support this proposal because we fully agree with the vision of the authors. Our R&D in Simware platform and our work at SISO and NATO CoIs related to LVC simulation are already pursuing this collaboration and a seamless interoperability between simulation and IOT technologies and processes. We called this vision the Internet of Simulations and it will enable the evolution of a niche technology, as it is distributed simulation nowadays to the mainstream, useful for exciting new applications as the use cases explored in the paper.
What do you think? Are you ready to collaborate with us on the future of simulation, the Internet of Simulations? If you are, please send me an email to jmlopez@simware.
Jose M Lopez
The market for flight training devices is growing, leveraged by the rising demand for air transportation and the increased request for virtual training in the military air forces. But it could grow at a much bigger pace if platforms were adopted. Training devices are still built as stovepipe and standalone products and therefore they are missing opportunities to deliver more functionality and capabilities if they were able to leverage the network.
Many industrial systems are already migrating to connected products, embracing the Internet of Things or IoT concept. As Michael Porter and James Heppelmann explained in the article "How smart connected products are transforming competition", published on Harvard Business Review on Nov 2014 : "Smart, connected industrial products offer exponencially expanding opportunities for new functionality and capabilities that trascent traditional product boundaries. The changing nature of products is disrupting value chains and forcing companies to rething nearly everything they do ...". Training devices are also industrial products and therefore they will have to evolve to the Internet of Things concept sooner than later. Market will demand not only connected training devices but also the new business models linked to IoT, as product-as-a-service or hybrid models between the extremes of product-as-a-service and conventional ownership, as for example a product sales bundles with a performance based contracts.
"Training devices are also industrial products and therefore they will have to evolve to the Internet of Things concept sooner than later"
The way to achieve this evolution in the flight simulation market is by evolving the current stovepipe training devices to connected products based on simulation platforms that allow to integrate the physical training device, composed by hardware and embedded software with software running on remote servers, owned by the simulator's provider or by external companies in the expanded supply chain that are providing specific software that increase the capabilities and functionalities of the training device.
Platforms would allow to build training devices as the integration of multiple simulation apps. In Simware Solutions, we have named this concept the Internet of Simulations or IoS. In the same way that Internet has transformed how we exchange and share information with others, and the Internet of the Things is promising to transform the way consumer and industrial devices are employed; the Internet of Simulations must unleash the real value of networked or distributed simulation.
In IoS, the stovepipe and standalone training device will evolve to a connected product that it is made up of multiple layers, some of them located in the training center of the customer and others running on remote servers. Conceptual architecture of a Connected Flight training device would be as the one shown in below picture.
"In the same way that Internet has transformed how we exchange and share information with others, and the Internet of the Things is promising to transform the way consumer and industrial devices are employed; the Internet of Simulations must unleash the real value of networked or distributed simulation"
IoS requires a new "technology infrastructure", that allows to provide training services with a hybrid deployment, combining some components on the customer facilities and many others on the cloud. This Cloud based deployment will allow not only a better maintenance and support of the training devices but also the capability to provide the same simulation functionality to different consumers located in different places and using different hardware. For example a simulator & training provider could serve the same high-fidelity simulation capabilities to a full flight simulator located in a dedicated training center in Florida and to a courseware running on a tablet that a pilot is using while is resting in his hotel after a flight in London.
Simware Solutions provides the technology and the architecture to make this evolution without any technical risk. Our Simware platform, as any IoT’s network infrastructure platform, provides the mechanisms to connect the simulators to the network and to share data between the publishers and subscribers or consumers of the data. Simulators can leverage Simware platform to evolve to smart devices that can connect to others in the network through the platform to improve their capabilities.
Simware's bedrock is its Layered Simulation Architecture or LSA. Below figure shows the layered architecture in Simware. These layers can be combined in many different ways to build almost any kind of simulation application, from simple training applications running on web or mobile platforms to complex full flight simulators. LSA is a network-oriented architecture, allowing to deploy a simulation "technology stack" as the one shown in above picture. Simware leverages one of the main data-exchange technologies in IoT, DDS to deliver real time and deterministic performance to the training device, even in a network deployment. Simware adds also compliance with HLA standard to connect with any simulation product already compliant with this distributed simulation technology.
The Internet of Simulations can bring many exciting opportunities to the flight simulation market. The technology to evolve the flight training devices to connected simulators is already in the market and Simware platform is an example. IoS will make easier the access to the flight simulation products to any type of customer, offering him new ways to experience the training. IoS will facilitate also the collaboration between the different stakeholders in the supply chain: the aircraft manufacturer, the simulation & training provider and also to many small and medium companies and research organizations that could provide its products and services embedded in the solutions of the large S&T providers. IoS would allow also to expand the applicability of flight simulations beyond training, for example to be used as test-sites for new equipment acting in this case as virtual prototyping laboratories, connected to the engineering departments of the aircraft or OEM manufacturers.
Are you ready to embrace IoS for your flight simulation solutions? If you are please contact us and we will help you to achieve it.
General Manager Simware Solutions
Our first post in this blog it is going to be about a topic of discussion in which the Simware team has been working for the last decade: simplifying the use of distributed simulation technologies.
During discussions with colleagues and friends within the Simulation and Training community, they bring up the many technical difficulties of implementing interoperability into simulation platforms.
Simulation&Training is associated with high end technology and, therefore, people translate concepts and applications that they use to manage other high-tech domains such as Business IT software, social media or mobile applications. In these high-tech domains, people are unfamiliar with the technical details of the devices and systems they are using and they are not concerned about how to connect them. For example how they connect their smartphone to a 4G antenna to place a call to a friend or how to connect to the google maps server and the GPS constellation in order to navigate to their business meeting on time or avoiding traffic jams.
The situation is very different when we are in the S&T domain and attempt to perform tasks much less complex than using a server in the cloud to navigate to a location. For example, even in the simplest case, as the federation of two virtual simulators in a LAN network, we can have several problems that prevent a seamless connection:
All above issues are solved with expensive services and specific pieces of software as the gateways. At the end, even the simplest case of federation requires a dedicated budget and at least a few weeks, if not months, of services to customize and integrate everything.
In conclusion, there is a gap between the marketing promise of the providers of distributed simulation solutions and the ability to deliver on that promise. Right now, only way to provide the Whole Product to the customer is by augmenting and complete the distributed simulations products with many other services and products, as the gateways, customization services for the simulators, etc. In fact the LVC Architectural Roadmap study reckoned that about 50% of the costs of interoperability are related to solve integration issues (to know more download LVCAR report at http://www.msco.mil)
Why we are living this situation? Why is so difficult to do distributed simulation and leverage the network in the simulators? LVCAR report also provides the answer: there are so many protocols, technologies and architectures and very few convergence points between all of them. Even when the features provided by all of them are very similar, their implementation and interfaces are very different. One way to solve the issues is by promoting a new architecture that replace all others, but a more pragmatic approach (in fact the approach already employed by many others industries, as Telcos or business IT) is to provide a common platform to all the different standards and protocols, providing a truly convergence between them. This is the approach we are using in our Simware platform and it is also the foundation for the Layered Simulation Architecture, that it is a nominating standard at SISO.
Do you agree with our analysis of the situation and our vision to move forward. Please share with us your thoughts commenting this post.
Posted by Jose-Maria Lopez. General Manager in Simware Solutions.
About the Blog
This Blog is supported by Simware Team. We are the team in charge of developing the product and supporting its users.