123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197 |
- External Services
- =================
- Up to now, everything was contained in the Modelverse.
- Nonetheless, this is not always possible, or desired: sometimes we want to run an activity using an external tool, or a service.
- There are two main reasons why we would like this: performance and feasability.
- For performance, an external service is likely to be faster than a Modelverse-based implementation.
- Not only because the action language execution of the Modelverse is comparatively slow, but also because of the optimized implementation of the external tool.
- It is clear that a specialized tool will be much faster than a naive implementation, even if the naive implementation were to be done in an efficient language.
- For example, PythonPDEVS uses many non-trivial algorithms to speed up DEVS simulation, which are not easily mimicked in a new implementation.
- While not impossible, of course, it seems wasteful to spend a significant amount of time to reimplement a tool.
- For feasability, an external service can offer operations that the Modelverse would otherwise not be able to execute.
- This links back to performance, but also to the algorithms required for the operation.
- For example, reimplementing a differential equation solver is wasteful, knowing that specialized tools have spent many man-years to implement and optimize it.
- We can never dream to come even close to these implementations.
- Both go hand-in-hand: a naive implementation is often possible, but there is always the tradeoff between spending more time in developing the algorithm, and more time in using the algorithm.
- The ideal solution would be to use an existing tool, thereby relying on the expertise of other developers, and merely making it available in the Modelverse.
- When making this available in the Modelverse, we strife to minimize the difference between an internal and external operation.
- This can be done using services: instead of modelling the activity, we model the communication with a tool that does the actual activity.
- While there remains a big difference, as we cannot alter the semantics of the external tool, we come fairly close, abstracting away from the tool that is actually used for the operation.
- In this section, we will illustrate this operation through the use of a *Fibonacci* service, which takes a single integer, and returns the Fibonacci number corresponding to it.
- Modelverse-side
- ---------------
- On the Modelverse, users can use the *comm* operations to communicate with external tools.
- This is highly similar to communication for input and output to the designated user.
- Code on the Modelverse can set up a connection to a registered service as follows::
- Integer function fibonacci(n : Integer):
- String port
- // Set up a dedicated communication port to the service
- port = comm_connect("fibonacci")
- // Send some data: the integer we got ourselves
- comm_send(port, n)
- // The external service computes the result
- // The comm_get function is blocking
- Integer result
- result = comm_get(port)!
- // Close the communication port from our side
- comm_close(port)
- return result!
- In this example, we send an integer parameter to the external service, and expect it to reply with another integer, containing the final result.
- When we got our result, we should always call *comm_close* to close the port at the Modelverse side.
- Multiple calls to *comm_send* and *comm_get* are possible, depending on the communication protocol devised between the Modelverse and the external service.
- For example, if we want to wait for the service to accept the result (e.g., the service can give an error on non-integer input)::
- Integer function fibonacci(n : Integer):
- String port
- port = comm_connect("fibonacci")
- comm_send(port, n)
- String response
- response = comm_get(port)
- if (response == "OK"):
- Integer result
- result = comm_get(port)
- comm_close(port)
- return result!
- elif (response == "Integer value expected!"):
- // error out, as it is not an integer!
- log("Non-integer input to fibonacci function!")
- comm_close(port)
- return -1!
- Sometimes, for example when coupled to Statecharts, we want to poll whether or not there is input from the service.
- This can be done using the *comm_poll* function, whose signature is identical to *comm_get*, though it returns a boolean and returns immediately.
- As expected, it does not consume the value itself, which can subsequently be read out using *comm_get*, which will now return immediately if *comm_poll* was True.
- Service-side
- ------------
- The external service itself should be slightly augmented with a minimal wrapper.
- This wrapper is responsible for the communication with the Modelverse, by importing the Modelverse wrapper, deserializing the input from the Modelverse, and serializing the output from the tool.
- For our fibonacci example, where the fibonacci code is written in Python as well, we can code this as follows::
- >>> def fibonacci_service(port):
- ... def fibonacci(n):
- ... if n <= 2:
- ... return 1
- ... else:
- ... return fibonacci(n - 1) + fibonacci(n - 2)
- ... mv_input = service_get(port)
- ... result = fibonacci(mv_input)
- ... service_set(port, result)
- >>> service_register("fibonacci", fibonacci_service)
- >>> try:
- ... while raw_input() != "STOP":
- ... pass
- ... finally:
- ... service_stop()
- In this simple piece of code, we define the actual fibonacci code in the function *fibonacci*, which is totally unrelated to the Modelverse and is just pure Python code.
- To wrap it, we define *fibonacci_service*, which is a function taking the communication port.
- This function is responsible for communication with the Modelverse, through the operations *service_get* and *service_set*.
- Both functions have the expected signature.
- The service function is invoked on a thread, and therefore runs concurrently with other invocations of this same service.
- Upon termination of the function, the thread ceases to exist.
- All threads are daemon threads, such that, if the service wants to exit, it can immediately exit without first finishing up the processing of tasks in the Modelverse.
- It is up to the Modelverse tasks to be resilient to such service failure or termination anyway, possibly through the use of a statechart that contains timeouts.
- When the service is registered, we have to specify the name of the service to be used in the Modelverse on a *comm_connect* call.
- The second parameter is the function to invoke when a connection is made, and must take one parameter: the communication port to be used.
- After the service is registered, the main thread simply continues.
- This makes it possible for the administrators of the service to maintain full control over their service.
- If the main thread terminates, all currently running services are terminated.
- Note that, after registration, this program is not able to do any other operations, until the *service_stop* operation is sent.
- When doing type checking first, we can do multiple *service_set* operations::
- >>> def fibonacci_service(port):
- ... def fibonacci(n):
- ... if n <= 2:
- ... return 1
- ... else:
- ... return fibonacci(n - 1) + fibonacci(n - 2)
- ... mv_input = service_get(port)
- ... if isinstance(mv_input, int):
- ... service_set(port, "OK")
- ... result = fibonacci(mv_input)
- ... service_set(port, result)
- ... else:
- ... service_set(port, "Integer value expected!")
- >>> service_register("fibonacci", fibonacci_service)
- >>> try:
- ... while raw_input() != "STOP":
- ... pass
- ... finally:
- ... service_stop()
- The invoked function is in this case other Python code, though this does not need to be that way.
- It would also be possible to invoke another executable, such as a *fibonacci* program, as long as the program can be used in an automated way.
- Therefore, the external services are not limited to Python in any way, and we merely use Python as a convencience, since we already have a wrapper for it.
- Using an external program happens through the *subprocess* Python module::
- >>> def fibonacci_service(port):
- ... def fibonacci(n):
- ... import subprocess
- ... result = subprocess.check_output(["./fibonacci", str(n)])
- ... return int(result)
- ... mv_input = service_get(port)
- ... result = fibonacci(mv_input)
- ... service_set(port, result)
- >>> service_register("fibonacci", fibonacci_service)
- >>> try:
- ... while raw_input() != "STOP":
- ... pass
- ... finally:
- ... service_stop()
- HUTN compiler
- -------------
- While it might not be obvious, some of the internal Modelverse operations are implemented using external services already.
- This is the case for the *compile_code* and *compile_model* functions.
- These functions rely on a HUTN compiler running as a service: the string is sent there, and the service sends back the list of operations to execute on the Modelverse (i.e., the compiled data).
- This was done to prevent a complete implementation of a parser in the Modelverse.
- While this would certainly be possible, at the moment it is not of high priority, and it is easy to shift it externally.
- Also, by using external services, it becomes possible to rely on trusted parsers, such as ANTLR or Ply, instead of building one from scratch as well.
- Users of the *compile_code* and *compile_model* functions are never exposed to the use of an external service: it looks exactly like any other function call.
- Nonetheless, there is a small difference: when the external service encounters a problem, or is simply not running, the functions will not be able to operate, even though the input is correct.
- In any case, the compile_code and compile_model operations return a string value if they encounter a problem.
- This problem could be anything, such as a syntax error or a semantical analysis error.
- Otherwise, the functions return the action language function and model, respectively.
- Before you can execute these functions, the HUTN compilation service must be running, which can be done by executing::
- >>> python scripts/HUTN_service.py
- The service will connect to the Modelverse specified in its configuration (i.e., the *init* call).
- It stays connected until the *STOP* input is given on *stdin*.
- Others
- ------
- Many other services have been defined and are automatically ran by the Modelverse.
- This includes a JSON service (serialization and deserialization), a file server (persistent storage to files), a DEVS simulation server (both interactive and batch execution), and a PetriNet analysis service.
- New services can easily be defined by putting a Python file in the *services* folder of the Modelverse.
- All its subfolders are traversed and for each the *main.py* file is executed upon starting the Modelverse.
- Upon starting the Modelverse, a log is printed to stdout of which services have started up.
- When the Modelverse terminates, all services also quit.
|