In the last section, we have seen how to use a very simple API of signals and vars for functional reactive programming. In this unit, we are going to take a look under the covers and see how that simple API could be implemented. We're going to develop, step by step, a simple implementation of signals and vars, which as you have seen form the basis of our approach to FRP. The two classes of signal and var are assumed to be in a package frp. We will start with summarizing the APIs, as we have seen them in the modules last week. So, here is the API of class signal. Signal takes as an argument the expression that constitutes the signal and it exposes a method, apply, which gives you the current value of the signal. For the moment, we have left out its implementation. There's also an object signal which has its own apply method. You'll recall that if objects have an apply method then they enable a syntax like this, that we can write signal and then some expression like that. And that would be the form that maps all the signals or creates constant signals. So, apply takes a binding parameter expression, which is this expression here that can be evaluated at multiple points in the future and it simply creates a signal with that expression. So, we've also used vars. So, a var is a subclass of signal and it adds one more method. And that's the update method, which takes the expression to be evaluated from now on. Its result type is unit and we have again left out the implementation. Like signals, var have an apply method, which allows us to create new vars with the same syntax as for signals like that. So now that we have the APIs, let's see how we could implement them. Here's the central implementation idea. Each signal maintains three data items. The first one is the current value of the signal. The second one is the expression defining the signal at the present time. So that will be used if you have to re-evaluate the signal. And the third one is a set of observers. The observers are all the other signals that would depend on the value of the current signal. The idea then is, if the signal changes, be it because it's updated or it depends on another signal that changes, so, if the signal changes, then all observers will be re-evaluated. So, that brings us to the question of dependency maintenance. How do we record dependencies in observers? When we evaluate a signal-valued expression, we need to know which signal caller gets defined or updated by the expression. If we know that, then we can add that caller to the observers of the signal when the signal is evaluated. That means when we take the current value of the signal. If we have added the caller to the observer set, then we can re-evaluate it each time the signal's value changes. So, each time a signal's value changes, all previously observing signals are re-evaluated. And what we also do then, we clear the set of sig observers because when we re-evaluate an observer, if the observer still depends on the value of the current signal, then it will automatically be re-entered into the observer set. But of course it might not depend on the current signal anymore because maybe another signal has changed, which is a Boolean condition, which makes the execution of the signal go into a different branch than before. So that's the central idea. There's the tricky issue; how do we find out on whose behalf a signal expression is evaluated? So, when we evaluate a signal expression, how do we know which other signal is currently updated or defined with a signal constructor? We will first look at one very simple way to do this - you might say a simplistic way to do this, and that is to maintain a global data structure that refers simply to the current caller and that we will update as we evaluate signals. We'll just come back to that later and discuss the wisdom of doing this and possible alternatives. The caller data structure is in fact accessed in a stack-like fashion because one evaluation of a signal might trigger the update or redefinition of other signals. So, the way we express this is in defining a class for stackable variables that can be accessed like this. So, here we would have a global variable, a value, caller, which is a stackable variable with some initial signal and then we could update the caller in a scoped fashion by saying caller with value, some other signal and then some expression that while the expression evaluates, the caller would have the value other signal and we could get the value out of the caller simply with the syntax caller dot value. So, those are the use cases. Here's the implementation. It's rather simple. So the class stack could be variable, it has a type parameter T and the value parameter which gives its initial value. Here's the stack maintained as a list of values. The current value is the head of the list and the with value operation takes a new value and an operation to perform that would have this new value as the top of the stack. So what it does is, it puts the new value on the top of the list, it executes the operation and when it's done with that, it tops the value of the list so the list becomes its own tail. So let's look at the set-up in Object Signal. You see here that in addition to the apply method, the object signal has now a value, caller, which is a stackable variable. The type parameter of the stackable variable is of type signal with an underscore as a parameter. That means that stackable variables can take signals of any value type. Let's have a look at the initial value. What should that be? Well, initially, there is no caller, so we have to express that somehow. And we choose to express it with a so-called sentinel object that we call NoSignal. So NoSignal is a special signal. It doesn't have a value at all. So it extends signal nothing and it doesn't have an implementation either. So we can pass the triple question marks, which essentially just mean unimplemented as the expression value for a NoSignal. So now that we have done with the set-up, let's have a look at the signal class. As we have mentioned before, the signal class has three items in its state. They are represented by three private variables. The first one is the current expression, which is called here, my expression. The second is the current value of the signal, which is called my value and the third is the set of observers. The initial values of my expression and my value are both uninitialized. The initial value of the set of observers is the empty set. So how do we initialize my expression and my value? Well, that will be done in the initial call to the update function that we are going to see next. So the update method gets called during the initialization of a signal and whenever somebody assigns a new value to the signal. It takes the expression to evaluate the new signal, assigns it to my expression and computes the current value of the signal. So we have to look at compute value next. Here it is. For now we use a simplified version of compute value - there will be more to add later - where compute value simply evaluates the current expression with the current signal as the caller and assigns the result into my value. So the last part of the Signal class is the apply method. Here is its implementation. Apply simply returns the current value of the signal, but before it does that, it adds the current caller to the set of observers. And the third part of apply is an assertion, which says that we should avoid defining cyclic signals that depend on themselves. So we have seen an example here. A signal like this is an error. So that error condition gets caught by this assert, which says that the observers of a signal may not contain the signal itself. So here, evaluating this expression here would be done with the current caller S. So that would mean that S gets added into the observers of the signal S itself. So this assert would be triggered and we would detect a cyclic signal definition. If the assert wasn't there, the error would still be detected, but it would be a stack overflow error. You would get an infinite recursion, which is much less informative than this assert. So if you've followed so far, then you should notice that there's one bit of the implementation of class signals still missing. Which is it? Is it error handling or re-evaluating the callers or constructing the observer set? Well, in fact, the missing piece is re-evaluating the callers. Let's have a look at that next. So we've seen that a signal's current value can change when somebody calls an update operation on a var or the value of a signal that the signal depends on changes. And in the second case, we need to detect these changes and propagate them to the observing signal. The propagation will be done in compute value. So now we'll find the implementation of that method. That's what it was before. We simply evaluate my expression with the current caller and assign the result to my value. And that's the new implementation of compute value. This part is as before, but now we store the result of evaluating my expression in a new value, new value. And if that new value is different from the current value, then we perform the following actions here. We assign the new value to my value. That's the same as before. But we also take the observers into a local value, obs, clear the set of observers and for each element of obs, we do a compute value. That will then in turn, by re-evaluating the signal, re-add the observer into the observer set. So because that will happen down here, we should clear the observer set before. So, that was the core of it. There's some little bit still to be done. One concerns NoSignal. In NoSignal, we have seen that NoSignal doesn't really have an expression associated with it, so we can't really compute its value. Because if we would try to evaluate the triple question mark, we would get an unimplemented exception. So, we override compute value in NoSignal to the empty expression that returns unit directly. So, the second bit of functionality still missing concerns vars. You'll recall that a var is a signal that can be updated by the client program and in fact, all the functionality we need for that is already present in class signal because in fact, signal does have already an update method. Here you see that the update method essentially does the right thing. It assigns the expression to be the current expression and it re-computes the value. But it is protected in class signal, which means only sub-classes can get at it whereas clients of signal cannot, which of course is intended because we want to make signal immutable. Clients of signals should not be able to call update. But of course clients of var should be able to call update. So, the way we achieve this is that we override the update method, make it public in var and the implementation of that update would simply be a call to the superclass, super dot update expression, so that way, essentially we use the same implementation, but we now expose it publicly. So this is it. I would argue that our implementation of functional reactive programming is quite stunning in its simplicity. But you might object that it's a little bit too simplistic. In particular, you might say, well, this makes use of the worst kind of state, namely, global state. Indeed, in the object signal, which is a global object, we have this value, caller, which is a stackable variable and that caller is accessed by all the code in an application using signals. One particular problem here is, what happens if we try to evaluate several signal expressions in parallel. Then multiple threads could access caller at the same time and could also update caller and stackable variable, so without protection in terms of synchronization or something we would get race conditions and unpredictable results. So one way to get around the problem of concurrent accesses to global state such as caller is to use synchronization. But synchronization has its own set of problems because it blocks threads. So threads might wait indefinitely long; it can be slow and it can lead to deadlocks. Another solution that is more appropriate here is to replace global state by thread-local state. So thread-local state means that each thread in an application accesses its own copy of the variable. So the variable is global for the thread but it's not shared between threads. Thread-local state is supported in Java through a Java util thread local and that's wrapped in Scala through a class Scala dot util dot dynamic variable. And we have engineered that in fact, the API of stackable variable matches exactly the API of dynamic variable. So we can simply swap out stackable variable, the global state that we want to avoid, for dynamic variable, which gives us the thread-local state. So the signal now will treat caller equals new dynamic variable with the same type parameter and initial value. So replacing stackable variable with dynamic variable is a definite improvement because we avoid the race conditions that the global variable would entail otherwise, if you have a multi-threaded application. But it comes with a number of other disadvantages. Well, the first one is actually shared with the global state and that's that essentially the state is imperative. We have an imperative state that's indirectly accessed by the whole application. So this gives essentially a high degree of entanglement of dependencies that are not made explicit in the types or input/output results of functions and that are therefore problematic. So that's essentially the problem of all imperative state. But the more global that state is, the worse the problem. The second shortcoming of thread-local state is that it's not terribly efficient because every access to a thread-local variable in fact involves a global hash table look-up. So the JDK stores essentially all thread-local variables in a hash table that's associated with the current thread and accessing that hash table is not as fast as if you just select that variable directly. And the third possible disadvantage is that if you have situations where you don't have a straightforward multi-threaded solution, but where, let's say, you multiplex threads over executors or tasks so your threads have become worker threads, then thread-local doesn't work so well because one task might run for a while on one thread and then it might switch to a different thread. And it would like to take its thread-local variable with it, but of course the thread-local variable gets locked to the current state. So in that case, it wouldn't be a solution. So to summarize, thread-local state is an improvement of unprotected global state, but it has its own set of problems. It's fragile and it plays well only with some approaches to concurrency. And it has still the problem that it is fundamentally a state that is shared by a large part of the application. There's actually another possible solution, which is much cleaner and that is to simply pass down the current caller into all the signal-valued expressions. Now if we do that explicitly, it would produce a lot of overhead. Essentially, every signal-valued expression has to have another parameter and these parameters have to be threaded through everything. But if you make the parameter implicit, then a lot of that burden can actually be avoided. So the idea is that instead of maintaining a thread-local variable, we pass its current value into a signal expression as an implicit parameter. And it's purely functional, but it currently requires still some more boiler plate than the thread-local solution because essentially, expressions have to close, have to take this implicit value as a parameter. Future versions of Scala might actually solve that problem, so we're currently tinkering with some ideas how that could be streamlined. So to conclude, we've given a quick tour of functional reactive programming with some usage examples and a full implementation. This is just a taste though, there's much more to be discovered. In particular, we've only covered one particular style of functional reactive programming, the style that treats discrete signals that are changed by events. There are other variants of functional reactive programming that treat continuous signals, sometimes just continuous and sometimes continuous and discrete together. So, a continuous signal cannot be broken up into a sequence of events because it changes its value at each time interval. What we do instead, typically in such systems, is use sampling instead of event propagation. So, that means we have signals that are continuous over the real time values. But what we need to do is not compute the signal at each possible point in time, but we can actually sample it as needed by the application with sufficient density. So, fascinating subjects for further exploration. What we will do in the next week and the week after is take a slightly different tack. Erik Meijer will take over and tell you about futures and observables. Observables are another form of event stream that is related to functional reactive programming, but it can be asynchronous and it can be non-deterministic.
