Fixed realtime + work on bokeh plotting

doc/examples/ContinuousTime.rst

@@ -56,7 +56,7 @@ changes throughout the simulation time. The clock-as-a-block structure allows th
 variation of the :code:`dt`, as is required for adaptive step size. This can be done
 manually by computing some simulation outputs, or via RK-preprocessing.
 
-.. attention::
+.. note::
     Runge-Kutta preprocessing is only available if there are one or more instances of
     :class:`CBD.lib.std.IntegratorBlock` in the original model. Also make sure not to
     use a flattened model to prevent errors.
@@ -83,7 +83,7 @@ adaptive step size of a CBD model called :code:`sinGen`, the following code can
    RKP = RKPreprocessor(tableau, atol=2e-5, hmin=0.1, safety=.84)
    newModel = RKP.preprocess(oldModel)
 
-.. attention::
+.. warning::
     Notice how the :code:`preprocess` method returns a new model that must be used in the simulation.
     Make sure to refer to this model when reading output traces or changing constants (see also
     :doc:`Dashboard`).

doc/examples/LivePlot.rst

@@ -178,18 +178,16 @@ Using Bokeh
 As an alternative for `MatPlotLib`, `Bokeh <https://docs.bokeh.org/en/latest/index.html>`_ kan be used. However, as
 you will see, this will require a little bit more "managing" code.
 
-.. attention::
+.. warning::
     While functional for the most part, live plotting using `Bokeh` is still in beta. Not all features will work
     as expected.
 
-.. warning::
+.. note::
     In order to get this plotting framework to show live plots, you need to start a `Bokeh` server via the command:
 
     .. code-block:: bash
 
-        bokeh serve
-
-    |
+        bokeh serve <experiment file>
 
 .. code-block:: python
 
@@ -228,7 +226,3 @@ you will see, this will require a little bit more "managing" code.
 
 .. figure:: ../_figures/sine-wave-bokeh.gif
     :width: 400
-
-.. note::
-    Currenly, there is a lot of "flickering" of the plot. There has not yet been found a solution
-    for this problem. It is presumed that this is a consequence of Bokeh being browser-based.

doc/examples/RealTime.rst

@@ -14,7 +14,7 @@ values.
     - When using progress bars, `tqdm <https://tqdm.github.io/>`_ must be installed.
     - In :doc:`LivePlot`, realtime simulation is used together with a variation of the :doc:`SinGen` example.
 
-.. attention::
+.. note::
     Unlike PyPDEVS_, interrupt events are not possible. However, as can be seen in the :doc:`Dashboard`
     example, the :class:`CBD.lib.std.ConstantBlock` allows for altering the internal value it outputs
     **during** the simulation. This mechanism may be manipulated to allow for external interrupts if
@@ -136,7 +136,7 @@ simulation can advance to the next timestep.
 
     print("FINISHED!")
 
-.. attention::
+.. warning::
     The simulation is still variable on the time constraints of your current system. Use the
     :class:`CBD.realtime.threadingGameLoopAlt.ThreadingGameLoopAlt` instead to fully control the time yourself.
     In this case, the :func:`CBD.simulator.Simulator.realtime_gameloop_call` requires the simulation time to be

examples/scripts/Dashboard/SinGen_experiment.py

@@ -5,10 +5,11 @@ import tkinter as tk
 from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
 
 fig = plt.figure(figsize=(15, 5), dpi=100)
-fig.tight_layout()
 ax = fig.add_subplot(111)
 ax.set_ylim((-1, 1))
 
+fig.tight_layout()
+
 cbd = SinGen("SinGen")
 
 root = tk.Tk()

examples/scripts/LivePlot/SinGen_bokeh_experiment.py

@@ -3,12 +3,12 @@ from CBD.realtime.plotting import PlotManager, Backend, LinePlot, follow
 from CBD.simulator import Simulator
 
 from bokeh.plotting import figure, curdoc
-from bokeh.client import push_session
 
 sinGen = SinGen("sin")
 
 fig = figure(plot_width=500, plot_height=500, y_range=(-1, 1))
-curdoc().add_root(fig)
+document = curdoc()
+document.add_root(fig)
 
 # Use the Bokeh Backend
 manager = PlotManager(Backend.BOKEH)
@@ -18,18 +18,17 @@ def set_xlim(limits):
 	lower, upper = limits
 	fig.x_range.start = lower
 	fig.x_range.end = upper
-manager.connect('sin', 'update', lambda d: set_xlim(follow(d[0], 10.0, lower_bound=0.0)))
 
-session = push_session(curdoc())
-session.show()
+def xupdate(limits):
+	global document
+	document.add_next_tick_callback(lambda lim=limits: set_xlim(lim))
+
+manager.connect('sin', 'update', lambda d: xupdate(follow(d[0], 10.0, lower_bound=0.0)))
+
+# session = push_session(curdoc())
+# session.show()
 
 sim = Simulator(sinGen)
 sim.setRealTime()
 sim.setDeltaT(0.1)
 sim.run(20.0)
-
-# NOTE: currently there can be 'flickering' of the plot
-import time
-while manager.is_opened():
-	session.push()
-	time.sleep(0.1)

src/CBD/Core.py

@@ -308,7 +308,7 @@ class BaseBlock:
         """
         Rewinds the CBD model to the previous iteration.
 
-        Danger:
+        Warning:
             Normally, this function should only be used by the internal mechanisms
             of the CBD simulator, **not** by a user. Using this function without a
             full understanding of the simulator may result in undefined behaviour.
@@ -535,7 +535,7 @@ class CBD(BaseBlock):
             Whenever this function is not called, upon simulation start a clock
             is added with the default values.
 
-        Danger:
+        Warning:
             **Clock Usage Assumption:** When adding a (custom) clock to your model(s),
             its outputs will always represent the (relative) simulated time and time-delta,
             independent of the simulation algorithm used. I.e., changing the delay of a

src/CBD/lib/ev3.py

@@ -461,7 +461,7 @@ class DifferentialDrive(BaseBlock):
 		- **IN1** -- The linear velocity in mm/s.
 		- **IN2** -- The angular velocity (top view) in deg/s. This is not the wheel rotation velocity.
 
-	Danger:
+	Warning:
 		You cannot manipulate the motors individually while this block is active.
 	"""
 	def __init__(self, block_name, left_port_name, right_port_name, wheel_diameter, axle_length,

src/CBD/lib/network.py

@@ -1,7 +1,7 @@
 """
 This file contains the CBD building blocks that can be used in networking communications.
 
-Danger:
+Warning:
     This code is currently in beta and not yet ready to be used in applications. Many parts
     may be subject to change.
 

src/CBD/lib/std.py

@@ -760,7 +760,7 @@ class DelayBlock(BaseBlock):
 		of the dependency graph, because of the difference at iteration 0 and
 		iteration 1.
 
-	Attention:
+	Warning:
 		The block delays based on an iteration, not on a specific time-delay.
 		When using adaptive stepsize simulation (either in the CBD simulator,
 		or when exported), this block may be the cause for errors.
@@ -982,7 +982,7 @@ class DeltaTBlock(BaseBlock):
 		in CBD models. In general, this block is therefore preferred over the
 		:code:`Clock` (unless you really know what you're doing).
 
-	Attention:
+	Warning:
 		For fixed step-size iterations, you may need to use this block for
 		*frame-independent logic* (as it is called in the videogame-world)
 		to ensure correct behaviour.
@@ -1019,7 +1019,7 @@ class Clock(CBD):
 		- **time** -- The current simulation time.
 		- **rel_time** -- The relative simulation time, ignoring the start time.
 
-	Danger:
+	Warning:
 		**Clock Usage Assumption:** When adding a (custom) clock to your model(s),
 		its outputs will always represent the (relative) simulated time and time-delta,
 		independent of the simulation algorithm used. I.e., changing the delay of a
@@ -1115,7 +1115,7 @@ class DummyClock(BaseBlock):
 		- **rel_time** -- The relative simulation time, ignoring the start time.
 		- **delta** -- The delay between two clock "ticks".
 
-	Danger:
+	Warning:
 		Only use this block in the tests!
 
 	See Also:
@@ -1171,8 +1171,8 @@ class SequenceBlock(BaseBlock):
 	:Output Ports:
 		**OUT1** -- The values from the sequence.
 
-	Danger:
-		Preferrably only use this block in the tests!
+	Warning:
+		Preferably only use this block in the tests!
 	"""
 	def __init__(self, block_name, sequence):
 		BaseBlock.__init__(self, block_name, [], ["OUT1"])

src/CBD/realtime/plotting.py

@@ -24,7 +24,9 @@ except ImportError:
 	_SNS_FOUND = False
 
 try:
-	from bokeh.plotting import curdoc
+	import bokeh.plotting as bkplt
+	import bokeh.io
+	import bokeh.models
 	_BOKEH_FOUND = True
 except ImportError:
 	_BOKEH_FOUND = False
@@ -58,11 +60,9 @@ class Backend:
 	
 			.. code-block:: bash
 	
-			   bokeh serve
-	
-			|
+			   bokeh serve <experiment file>
 			
-		Attention:
+		Warning:
 			Because of how Bokeh works, it is currently impossible to capture premature
 			close events on a system. The plots will always remain active once started.
 		"""
@@ -188,9 +188,9 @@ class PlotHandler:
 				                                     interval=interval, frames=frames)
 				figure[0].canvas.mpl_connect('close_event', lambda _: self.__close_event())
 			elif Backend.compare("BOKEH", backend):
-				curdoc().add_periodic_callback(lambda: self.update(), interval)
+				bokeh.io.curdoc().add_periodic_callback(lambda: self.update(), interval)
 				# TODO (is this even possible?):
-				curdoc().on_session_destroyed(lambda ctx: self.__close_event())
+				bokeh.io.curdoc().on_session_destroyed(lambda ctx: self.__close_event())
 
 	def signal(self, name, *args):
 		"""
@@ -219,10 +219,7 @@ class PlotHandler:
 		"""
 		if name not in self.__events:
 			raise ValueError("Invalid signal '%s' in PlotHandler." % name)
-		# print("SIGNAL:", name)
 		self.__event_bus.append((name, args))
-		# for evt in self.__events[name]:
-		# 	evt(*args)
 
 	def connect(self, name, function):
 		"""
@@ -447,7 +444,7 @@ def follow(data, size=None, lower_bound=float('-inf'), upper_bound=float('inf'),
 								the data will not be shown for a value of :code:`1.0`.
 								Defaults to :code:`0.5` (= the middle).
 
-	Danger:
+	Warning:
 		* The lower bound must be strictly smaller than the upper bound.
 		* The size is :code:`None` and the bounds are infinity and the limits are equal.
 		* The size cannot be larger than the distance between the bounds.
@@ -580,6 +577,9 @@ class LinePlot(PlotKind):
 			line, = figure[1].plot([], [], *self.args, **self.kwargs)
 			return line
 		elif self.is_backend("BOKEH"):
+			# source = bokeh.models.ColumnDataSource({'x': [], 'y': []})
+			# figure.line(source=source, *self.args, **self.kwargs)
+			# return source
 			return figure.line([], [], *self.args, **self.kwargs)
 		elif self.is_backend("SNS"):
 			# matplotlib: figure[1] is the axis

src/CBD/realtime/threadingGameLoop.py

@@ -13,7 +13,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from . import accurate_time as time
 from threading import Lock
 
 _GL_LOCK = Lock()
@@ -24,32 +23,24 @@ class ThreadingGameLoop(object):
     Time will only progress when a :func:`step` call is made.
     """
     def __init__(self):
-        """
-        Constructor
-        """
-        self.next_event = float('inf')
+        self.event_list = []
 
     def step(self):
         """
-        Perform a step in the simulation. Actual processing is done in a seperate thread.
+        Perform a step in the simulation.
         """
         with _GL_LOCK:
-            if time.time() >= self.next_event:
-                self.next_event = float('inf')
-                getattr(self, "func")()
+            now = time.time()
+            while now >= self.event_list[0][0]:
+                _, func = self.event_list.pop()
+                func()
         
     def wait(self, delay, func):
         """
         Wait for the specified time, or faster if interrupted
 
-        :param delay: time to wait
-        :param func: the function to call
-        """
-        self.func = func
-        self.next_event = time.time() + delay
-    
-    def interrupt(self):
-        """
-        Interrupt the waiting thread
+        Args:
+            delay (float):      Time to wait.
+            func (callable):    The function to call.
         """
-        self.next_event = 0
+        self.event_list.append((time.time() + delay, func))

src/CBD/realtime/threadingGameLoopAlt.py

@@ -31,10 +31,7 @@ class ThreadingGameLoopAlt(object):
         really know what you're doing.
     """
     def __init__(self):
-        """
-        Constructor
-        """
-        self.next_event = float('inf')
+        self.event_list = []
         self.time = 0.0
 
     def step(self, time):
@@ -47,22 +44,16 @@ class ThreadingGameLoopAlt(object):
         """
         with _GL_LOCK:
             self.time = time
-            if self.time >= self.next_event:
-                self.next_event = float('inf')
-                getattr(self, "func")()
+            while self.time >= self.event_list[0][0]:
+                _, func = self.event_list.pop()
+                func()
         
     def wait(self, delay, func):
         """
-        Wait for the specified time, or faster if interrupted
+        Wait for the specified time.
 
-        :param delay: time to wait
-        :param func: the function to call
-        """
-        self.func = func
-        self.next_event = self.time + delay
-    
-    def interrupt(self):
-        """
-        Interrupt the waiting thread
+        Args:
+            delay (float):      Time to wait.
+            func (callable):    The function to call.
         """
-        self.next_event = 0
+        self.event_list.append((self.time + delay, func))

src/CBD/realtime/threadingPython.py

@@ -13,9 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import time
-from threading import Event, Thread, Lock
-from . import accurate_time as time
+from threading import Thread
+from . import accurate_time
 
 class ThreadingPython(object):
     """
@@ -31,11 +30,11 @@ class ThreadingPython(object):
         """
         #NOTE this call has a granularity of 5ms in Python <= 2.7.x in the worst case, so beware!
         #     the granularity seems to be much better in Python >= 3.x
-        p = Thread(target=ThreadingPython.callFunc, args=[self, time, func])
+        p = Thread(target=self.run, args=[time, func])
         p.daemon = True
         p.start()
 
-    def callFunc(self, delay, func):
+    def run(self, delay, func):
         """
         Function to call on a seperate thread: will block for the
         specified time and call the function afterwards.
@@ -45,5 +44,5 @@ class ThreadingPython(object):
             func:           The function to call. No arguments can be
                             used and no return values are needed.
         """
-        time.sleep(delay)
+        accurate_time.sleep(delay)
         func()

src/CBD/simulator.py

@@ -147,11 +147,7 @@ class Simulator:
 			self.__realtime_start_time = time.time()
 			self.__lasttime = 0.0
 
-		# self.__event_thread.start()
-		# self.__threading_backend.wait(0, self.__event_thread_loop)
-		# self.__event_thread_loop()
-		self.__threading_backend_args[0].after(3000, lambda: print("hey"))
-		self.__threading_backend.wait(2, lambda: print("ho"))
+		self.__threading_backend.wait(0, self.__event_thread_loop)
 		self.signal("started")
 		if self.__realtime:
 			# Force execute the first iteration now. This way iteration n (n > 0) is
@@ -175,7 +171,6 @@ class Simulator:
 		self.__tracer.stopTracers()
 		self.signal("finished")
 		self.__finished = True
-		# self.__event_thread.join()
 
 	def __check(self):
 		"""
@@ -407,13 +402,6 @@ class Simulator:
 		self.__threading_backend_subsystem = subsystem
 		self.__threading_backend_args = args
 
-		# TODO: allow the user to also do things here?
-		# if subsystem == Platform.TKINTER:
-		# 	def cls(s, t):
-		# 		s.__finish()
-		# 		t.destroy()
-		# 	args[0].protocol("WM_DELETE_WINDOW", lambda a=self, b=args[0]: cls(a, b))
-
 	def setRealTimePlatformThreading(self):
 		"""
 		Wrapper around the :func:`setRealTimePlatform` call to automatically
@@ -487,7 +475,7 @@ class Simulator:
 		"""
 		Does a single simulation step.
 
-		Danger:
+		Warning:
 			Do **not** use this function to forcefully progress the simulation!
 			All functionalities for validly simulating and executing a system
 			that are provided through other parts of the interface should be
@@ -496,7 +484,7 @@ class Simulator:
 		"""
 		self.signal("prestep")
 		curIt = self.__sim_data[2]
-		# self.__threading_backend.wait(10, lambda: self.__tracer.traceNewIteration(curIt, self.getTime()))
+		self.__threading_backend.wait(0, lambda: self.__tracer.traceNewIteration(curIt, self.getTime()))
 
 		# Efficiency reasons: dep graph only changes at these times
 		#   in the given set of library blocks.
@@ -580,7 +568,7 @@ class Simulator:
 				block = component[0]  # the strongly connected component has a single element
 				if curIteration == 0 or self.__scheduler.mustCompute(block, self.getTime()):
 					block.compute(curIteration)
-					# self.__threading_backend.wait(10, lambda: self.__tracer.traceCompute(curIteration, block))
+					self.__threading_backend.wait(0, lambda: self.__tracer.traceCompute(curIteration, block))
 			else:
 				# Detected a strongly connected component
 				self.__solver.checkValidity(self.model.getPath(), component)
@@ -590,7 +578,7 @@ class Simulator:
 					if curIteration == 0 or self.__scheduler.mustCompute(block, self.getTime()):
 						blockIndex = component.index(block)
 						block.appendToSignal(solutionVector[blockIndex])
-						# self.__threading_backend.wait(10, lambda: self.__tracer.traceCompute(curIteration, block))
+						self.__threading_backend.wait(0, lambda: self.__tracer.traceCompute(curIteration, block))
 
 	def __hasCycle(self, component, depGraph):
 		"""
@@ -632,7 +620,12 @@ class Simulator:
 
 	def connect(self, name, function):
 		"""
-		Connect an event with an additional function.
+		Connect an event with an additional, user-defined function. These functions
+		will be executed on a separate thread and polled every 50 milliseconds.
+
+		Warning:
+			It is expected that the passed function terminates at a certain point
+			in time, to prevent an infinitely running process.
 
 		The functions will be called in the order they were connected to the
 		events, with the associated arguments. The accepted signals are:
@@ -650,10 +643,6 @@ class Simulator:
 			name (str):     The name of the signal to raise.
 			function:       A function that will be called with the optional arguments
 							whenever the event is raised.
-
-		Warning:
-			While these functions will be handled on a different thread, it heavy
-			computations will eventually result in a delay of all events.
 		"""
 		if name not in self.__events:
 			raise ValueError("Invalid signal '%s' in Simulator." % name)
@@ -689,14 +678,11 @@ class Simulator:
 		self.__event_bus.append((name, args))
 
 	def __event_thread_loop(self):
-		print("looping")
 		if not self.__finished:
 			while len(self.__event_bus) > 0:
 				name, args = self.__event_bus.pop()
 				for evt in self.__events[name]:
-					evt(*args)
-			# time.sleep(0.05)
-			print("hello there")
+					self.__threading_backend.wait(0, evt(*args))
 			self.__threading_backend.wait(0.05, self.__event_thread_loop)
 
 	def setCustomTracer(self, *tracer):
@@ -736,7 +722,7 @@ class Simulator:
 			to multiple files is possible, though more inefficient than simply (manually) copying
 			the file at the end.
 
-		Danger:
+		Warning:
 			Using multiple verbose tracers with the same filename will yield errors and undefined
 			behaviour.
 		"""

streaming.py

@@ -0,0 +1,27 @@
+# streaming/main.py
+from bokeh import models, plotting, io
+import pandas as pd
+from time import sleep
+from itertools import cycle
+data = pd.read_csv(
+    "https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-states.csv")
+data["date"] = pd.to_datetime(data["date"])
+data["new_cases"] = data.groupby("state")["cases"].diff()
+state = "California"
+california_covid_data = data[data["state"] == state].copy()
+source = models.ColumnDataSource(california_covid_data)
+p = plotting.figure(
+    x_axis_label="Date", y_axis_label="New Cases",
+    plot_width=800, plot_height=250, x_axis_type="datetime", tools=["hover", "wheel_zoom"]
+)
+p.line(x="date", y="new_cases",
+       source=source,
+       legend_label=state,
+       width=4,
+       )
+io.curdoc().add_root(p)
+index_generator = cycle(range(len(california_covid_data.index)))
+def stream():
+    index = next(index_generator)
+    source.data = california_covid_data.iloc[:index]
+io.curdoc().add_periodic_callback(stream, 10)