|
|
@@ -1,24 +1,175 @@
|
|
|
External Services
|
|
|
=================
|
|
|
|
|
|
-Explain the use of services.
|
|
|
+Up to now, everything was contained in the Modelverse.
|
|
|
+Nonetheless, this is not always possible, or desired: sometimes we want to run an operation through an external tool, or a service.
|
|
|
+There are two main reasons why we would like this: performance, and feasability.
|
|
|
+
|
|
|
+For performance, an external service is likely to be faster than a Modelverse-based implementation.
|
|
|
+Not only because the action language execution of the Modelverse is comparatively slow, but also because of the optimized implementation of the external tool.
|
|
|
+It is clear that a specialized tool will be much faster than a naive implementation, even if the naive implementation were to be done in an efficient language.
|
|
|
+For example, PythonPDEVS uses many non-trivial algorithms to speed up DEVS simulation, which are not easily mimicked in a new implementation.
|
|
|
+While not impossible, of course, it seems wasteful to spend a significant amount of time to reimplement a tool.
|
|
|
+
|
|
|
+For feasability, an external service can offer operations that the Modelverse would otherwise not be able to execute.
|
|
|
+This links back to performance, but also to the algorithms required for the operation.
|
|
|
+For example, reimplementing a differential equation solver is wasteful, knowing that specialized tools have spent many man-years to implement and optimize it.
|
|
|
+We can never dream to come even close to these implementations.
|
|
|
+
|
|
|
+Both go hand-in-hand: a naive implementation is often possible, but there is always the trade-of between spending more time in developing the algorithm, and more time in using the algorithm.
|
|
|
+The ideal solution would be to use an existing tool, thereby relying on the expertise of other developers, and merely making it available in the Modelverse.
|
|
|
+
|
|
|
+When making this available in the Modelverse, we strife to minimize the difference between an internal and external operation.
|
|
|
+This can be done using services: instead of modelling the operation, we model the communication with a tool that does the operation.
|
|
|
+While there remains a big difference, as we cannot alter the semantics of the external tool, we come fairly close, abstracting away from the tool that is actually used for the operation.
|
|
|
+
|
|
|
+In this section, we will illustrate this operation through the use of a *Fibonacci* service, which takes a single integer, and returns the Fibonacci number corresponding to it.
|
|
|
|
|
|
Modelverse-side
|
|
|
---------------
|
|
|
|
|
|
-Explain how services work in the Modelverse.
|
|
|
+On the Modelverse, users can use the *comm* operations to communicate with external tools.
|
|
|
+This is highly similar to communication for input and output to the designated user.
|
|
|
+Code on the Modelverse can set up a connection to a registered service as follows::
|
|
|
+
|
|
|
+ Integer function fibonacci(n : Integer):
|
|
|
+ String port
|
|
|
+
|
|
|
+ // Set up a dedicated communication port to the service
|
|
|
+ port = comm_connect("fibonacci")
|
|
|
+
|
|
|
+ // Send some data: the integer we got ourselves
|
|
|
+ comm_send(port, n)
|
|
|
+
|
|
|
+ // The external service computes the result
|
|
|
+
|
|
|
+ // The comm_get function is blocking
|
|
|
+ Integer result
|
|
|
+ result = comm_get(port)!
|
|
|
+
|
|
|
+ // Close the communication port from our side
|
|
|
+ comm_close(port)
|
|
|
+ return result!
|
|
|
+
|
|
|
+In this example, we send an integer parameter to the external service, and expect it to reply with another integer, containing the final result.
|
|
|
+When we got our result, we should always call *comm_close* to close the port at the Modelverse side.
|
|
|
+
|
|
|
+Multiple calls to *comm_send* and *comm_get* are possible, depending on the communication protocol devised between the Modelverse and the external service.
|
|
|
+For example, if we want to wait for the service to accept the result (e.g., the service can give an error on non-integer input)::
|
|
|
+
|
|
|
+ Integer function fibonacci(n : Integer):
|
|
|
+ String port
|
|
|
+ port = comm_connect("fibonacci")
|
|
|
+
|
|
|
+ comm_send(port, n)
|
|
|
+
|
|
|
+ String response
|
|
|
+ response = comm_get(port)
|
|
|
+ if (response == "OK"):
|
|
|
+ Integer result
|
|
|
+ result = comm_get(port)
|
|
|
+ comm_close(port)
|
|
|
+ return result!
|
|
|
+ elif (response == "Integer value expected!"):
|
|
|
+ // error out, as it is not an integer!
|
|
|
+ log("Non-integer input to fibonacci function!")
|
|
|
+ comm_close(port)
|
|
|
+ return -1!
|
|
|
+
|
|
|
+Sometimes, for example when coupled to Statecharts, we want to poll whether or not there is input from the service.
|
|
|
+This can be done using the *comm_poll* function, whose signature is identical to *comm_get*, though it returns a boolean and returns immediately.
|
|
|
+As expected, it does not consume the value itself, which can subsequently be read out using *comm_get*, which will now return immediately if *comm_poll* was True.
|
|
|
|
|
|
Service-side
|
|
|
------------
|
|
|
|
|
|
-Explain how services work in the Python wrapper.
|
|
|
+The external service itself should be slightly augmented with a minimal wrapper.
|
|
|
+This wrapper is responsible for the communication with the Modelverse, by importing the Modelverse wrapper, deserializing the input from the Modelverse, and serializing the output from the tool.
|
|
|
+For our fibonacci example, where the fibonacci code is written in Python as well, we can code this as follows::
|
|
|
+
|
|
|
+ >>> def fibonacci_service(port):
|
|
|
+ ... def fibonacci(n):
|
|
|
+ ... if n <= 2:
|
|
|
+ ... return 1
|
|
|
+ ... else:
|
|
|
+ ... return fibonacci(n - 1) + fibonacci(n - 2)
|
|
|
+ ... mv_input = service_get(port)
|
|
|
+ ... result = fibonacci(mv_input)
|
|
|
+ ... service_set(port, result)
|
|
|
+ >>> service_register("fibonacci", fibonacci_service)
|
|
|
+ >>> try:
|
|
|
+ ... while raw_input() != "STOP":
|
|
|
+ ... pass
|
|
|
+ ... finally:
|
|
|
+ ... service_stop()
|
|
|
|
|
|
-Example
|
|
|
--------
|
|
|
+In this simple piece of code, we define the actual fibonacci code in the function *fibonacci*, which is totally unrelated to the Modelverse and is just pure Python code.
|
|
|
+To wrap it, we define *fibonacci_service*, which is a function taking the communication port.
|
|
|
+This function is responsible for communication with the Modelverse, through the operations *service_get* and *service_set*.
|
|
|
+Both functions have the expected signature.
|
|
|
+The service function is invoked on a thread, and therefore runs concurrently with other invocations of this same service.
|
|
|
+Upon termination of the function, the thread ceases to exist.
|
|
|
+All threads are daemon threads, such that, if the service wants to exit, it can immediately exit without first finishing up the processing of tasks in the Modelverse.
|
|
|
+The Modelverse tasks should be resilient to service failure or termination anyway, possibly through the use of a statechart that contains timeouts.
|
|
|
|
|
|
-Give a full example of a service.
|
|
|
+When the service is registered, we have to specify the name of the service to be used in the Modelverse on a *comm_connect* call.
|
|
|
+The second parameter is the function to invoke when a connection is made, and must take one parameter: the communication port to be used.
|
|
|
+After the service is registered, the main thread simply continues.
|
|
|
+This makes it possible for the administrators of the service to maintain full control over their service.
|
|
|
+If the main thread terminates, all currently running services are terminated.
|
|
|
+Note that, after registration, this program is not able to do any other operations, until the *service_stop* operation is sent.
|
|
|
+
|
|
|
+When doing type checking first, we can do multiple *service_set* operations::
|
|
|
+
|
|
|
+ >>> def fibonacci_service(port):
|
|
|
+ ... def fibonacci(n):
|
|
|
+ ... if n <= 2:
|
|
|
+ ... return 1
|
|
|
+ ... else:
|
|
|
+ ... return fibonacci(n - 1) + fibonacci(n - 2)
|
|
|
+ ... mv_input = service_get(port)
|
|
|
+ ... if isinstance(mv_input, int):
|
|
|
+ ... service_set(port, "OK")
|
|
|
+ ... result = fibonacci(mv_input)
|
|
|
+ ... service_set(port, result)
|
|
|
+ ... else:
|
|
|
+ ... service_set(port, "Integer value expected!")
|
|
|
+ >>> service_register("fibonacci", fibonacci_service)
|
|
|
+ >>> try:
|
|
|
+ ... while raw_input() != "STOP":
|
|
|
+ ... pass
|
|
|
+ ... finally:
|
|
|
+ ... service_stop()
|
|
|
+
|
|
|
+The invoked function is in this case other Python code, though this does not need to be that way.
|
|
|
+It would also be possible to invoke another executable, such as a *fibonacci* program, as long as the program can be used in an automated way.
|
|
|
+Therefore, the external services are not limited to Python in any way, and we merely use Python as a convencience, since we already have a wrapper for it.
|
|
|
+Using an external program happens through the *subprocess* Python module::
|
|
|
+
|
|
|
+ >>> def fibonacci_service(port):
|
|
|
+ ... def fibonacci(n):
|
|
|
+ ... import subprocess
|
|
|
+ ... result = subprocess.check_output(["./fibonacci", str(n)])
|
|
|
+ ... return int(result)
|
|
|
+ ... mv_input = service_get(port)
|
|
|
+ ... result = fibonacci(mv_input)
|
|
|
+ ... service_set(port, result)
|
|
|
+ >>> service_register("fibonacci", fibonacci_service)
|
|
|
+ >>> try:
|
|
|
+ ... while raw_input() != "STOP":
|
|
|
+ ... pass
|
|
|
+ ... finally:
|
|
|
+ ... service_stop()
|
|
|
|
|
|
HUTN compiler
|
|
|
-------------
|
|
|
|
|
|
-Explain how the HUTN compiler (for the compile_code and compile_model operations) works using services
|
|
|
+While it might not be obvious, some of the internal Modelverse operations are implemented using external services already.
|
|
|
+This is the case for the *compile_code* and *compile_model* functions.
|
|
|
+These functions rely on a HUTN compiler running as a service: the string is sent there, and the service sends back the list of operations to execute on the Modelverse (i.e., the compiled data).
|
|
|
+This was done to prevent a complete implementation of a parser in the Modelverse.
|
|
|
+While this would certainly be possible, at the moment it is not of high priority, and it is easy to shift it externally.
|
|
|
+Also, by using external services, it becomes possible to rely on trusted parsers, such as ANTLR or Ply, instead of building one from scratch as well.
|
|
|
+
|
|
|
+Users of the *compile_code* and *compile_model* functions are never exposed to the use of an external service: it looks exactly like any other function call.
|
|
|
+Nonetheless, there is a small difference: when the external service encounters a problem, or is simply not running, the functions will not be able to operate, even though the input is correct.
|
