commit 6c3bf18c5ae463b684bf0407b4f47dd389f4abe4 Author: Max Rottenkolber Date: Sun Dec 4 22:50:17 2016 +0100 Add blog/erlangen-into. diff --git a/blog/erlangen-intro.meta b/blog/erlangen-intro.meta index bbbba5f..70afb25 100644 --- a/blog/erlangen-intro.meta +++ b/blog/erlangen-intro.meta @@ -1,4 +1,4 @@ :document (:title "Erlangen: Introduction" :author "Max Rottenkolber " :index-headers-p nil) -:publication-date nil +:publication-date "2016-12-04 22:48+0100" commit 640861fd074d4d5bab63f190c494c9315614c959 Author: Max Rottenkolber Date: Sun Dec 4 22:46:11 2016 +0100 blog/erlangen-input: version updated for new design. diff --git a/blog/erlangen-intro.mk2 b/blog/erlangen-intro.mk2 index 9cb39ac..f994d26 100644 --- a/blog/erlangen-intro.mk2 +++ b/blog/erlangen-intro.mk2 @@ -1,6 +1,6 @@ [Erlangen](https://github.com/eugeneia/erlangen) brings distributed, -asynchronous message passing to [Clozure CL](http://ccl.clozure.com/). It -orchestrates Clozure CL _processes_ (native threads) using [message passing](http://c2.com/cgi/wiki?MessagePassingConcurrency), +asynchronous message passing to [Clozure Common Lisp](http://ccl.clozure.com/). +It orchestrates Clozure CL _processes_ (native threads) using [message passing](http://c2.com/cgi/wiki?MessagePassingConcurrency), and encourages fault tolerant software architectures using [supervison trees](http://erlang.org/documentation/doc-4.9.1/doc/design_principles/sup_princ.html). It is also transparently distributed, all its features work seamlessly across IP networks. Thus, it can be used to build applications across multiple @@ -10,8 +10,8 @@ Clozure CL instances on different hosts. Erlangen borrows many ideas from Clozure CL processes are comparatively heavy weight, preemptively scheduled operating system threads, as opposed to Erlang’s cooperatively scheduled green threads. As such, processes are a scarce resource on Clozure CL. Erlangen is -based on the assumption that even with only a limited number of processes -available, message passing and supervision trees are still desireable features. +based on the assumption that even with only a limited capacity of concurrent +processes, message passing and supervision trees are still feasible features. While Erlang’s distribution features are sometimes overlooked, they are meant to be a main focus of Erlangen (but not of this article). Besides the overlap in features, terminology, and name, Erlangen and Erlang/OTP are completely @@ -23,13 +23,12 @@ erlangen-intro-send.svg Erlangen equips each process with a _mailbox_, a list of _links_, and a list of _monitors_. These upgraded processes are called _agents_. Agents can [send](http://mr.gy/software/erlangen/api.html#section-1-13) messages to the mailboxes of other agents, and [receive](http://mr.gy/software/erlangen/api.html#section-1-9) -the next message from in its own mailbox. When {send} fails to deliver a -message it signals an error. Message delivery can fail when the target agent -exited, the destination _mailbox_ is full (mailboxes are bounded FIFO queues), -or due to network failure. When {send} returns the message is guaranteed to be -delivered to the target agent’s mailbox. Agents can supply a timeout argument -to {receive}, which will then signal an error if no message is received until -the timeout is exceeded. +the next message from their own mailbox. Message delivery can fail when the +target agent exited, the destination mailbox is full (mailboxes are bounded +FIFO queues), or due to network failure. As such, delivery of any message to +the target agent’s mailbox is not guaranteed. Agents can supply a timeout +argument to {receive}, which will then signal a {timeout} error if no message +is received until the timeout is exceeded. #media# erlangen-intro-mailbox.svg @@ -37,34 +36,75 @@ erlangen-intro-mailbox.svg Agents can also _exit_ (using [exit](http://mr.gy/software/erlangen/api.html#section-1-6) or by returning normally), and send _kill messages_ to other agents. When an agent receives a kill message it exits immediately, and the remaining messages -in its mailbox are discarded. When {exit} fails to deliver the exit message it -signals an error. The links and monitors of an agent are references to other -agents. When an agent exits it attempts to send kill messages to all its links, +in its mailbox are discarded. The links and monitors of an agent are references +to other agents. When an agent exits it sends kill messages to all its links, and _exit notifications_ to all its monitors. Agents can use [link](http://mr.gy/software/erlangen/api.html#section-1-7) -and [unlink](http://mr.gy/software/erlangen/api.html#section-1-17) to manage -links and monitors. +and [unlink](http://mr.gy/software/erlangen/api.html#section-1-16) to manage +links and monitors. Unlike regular messages, kill messages and exit +notifications will never be lost in transit, and they are prioritized so that +they will be received before any pending regular message. Unless the target +agent exits before it receives the kill message or exit notification, they are +guaranteed to be delivered, eventually. #media# erlangen-intro-link.svg -Finally, agents can [spawn](http://mr.gy/software/erlangen/api.html#section-1-15) +Finally, agents can [spawn](http://mr.gy/software/erlangen/api.html#section-1-14) new agents. {spawn} takes a nullary function for the agent’s process to execute, and returns a reference to the newly created agent. This reference can -then be used to communicate or link with the agent. Additionally, {spawn} also -can also link the caller and the new agent before the new agent’s process -starts. This is useful to avoid a basic race condition where the child agent -exits before the parent could link with it. +then be used to communicate or link with the agent. Additionally, {spawn} can +also link the caller and the new agent before the new agent’s process starts. +This is useful to avoid a basic race condition where the child agent exits +before the parent could link with it. That covers pretty much all of the core functionality, but Erlangen is an -asynchronous framework for a synchronous programming language, and that poses -some problems. Our processes are preemtively scheduled, and it is unsafe to -stop them asynchronously. Er +asynchronous programming framework for a synchronous programming language, and +that poses some problems. Our processes are preemptively scheduled, and it is +unsafe to stop them asynchronously. Erlangen agents must call {receive} +periodically, or else they become unresponsive to kill messages. Specifically, +blocking calls to input and output routines, such as found in the Common Lisp +standard, can pose problems in some cases. For instance, imagine an agent that +listens for user input on an interactive input stream. - that does not call {receive} periodically +#code# +(spawn 'read-char) +# - messages can be lost even though they were delivered successfully +The resulting agent would block until a character is available in +{*standard-input*}, and not respond to incoming messages, potentially +indefinitely. This is a big no-go. One solution is to implement non-halting +agents as event loops. Instead of blocking while waiting for a single event, +they continuously poll for multiple events like pending input and messages. The +[select](http://mr.gy/software/erlangen/api.html#section-1-12) macro helps to +implement such an event loop. At its heart is a pacing poll algorithm that +takes naps when there is nothing to do, thereby trading a little bit of initial +latency for saved processor cycles in times of low load. Given alternative, +asynchronous input routines the pathological case from before can be avoided +using {select}: - exit notifications can be lost +#code# +(spawn (lambda () + (select ((read-char-no-hang) (char) + char)))) +# - [select](http://mr.gy/software/erlangen/api.html#section-1-12) +This agent polls for new input using the non-blocking {read-char-no-hang} while +also receiving pending messages. Since there is no {:receive} clause, all +incoming messages are discarded (but kill messages will still cause the agent +to exit). When {read-char-no-hang} returns a non-nil object as its first value, +the clause’s lambda list {(char)} is bound to the return values, the body +of the clause is evaluated, and its result returned. In this case, {char} is +returned, and the agent exits subsequently with that value. To manage +additional types of events, we would add more clauses to {select}. +To summarize, agents need to be programmed with concurrency in mind. Within the +boundaries of the Erlangen framework, this will come naturally by composing +programs using the provided message passing facilities. When faced with outside +input, use of blocking routines may not be acceptable. Sadly, this affects many +of the standard Common Lisp I/O routines, and Clozure CL’s socket stream +extension is only of limited help, too. Asynchronous I/O libraries like [IOLib](https://common-lisp.net/project/iolib/) +provide a way out. + +If you are interested in this framework, you are welcome to create an issue on +[GitHub](https://github.com/eugeneia/erlangen/issues), or shoot a mail to me at +[max@mr.gy](mailto:max@mr.gy). Let me know what you think! commit f331affc0e738f2cf970f59cbeca2b43a2a1f1b0 Author: Max Rottenkolber Date: Mon Oct 3 20:27:26 2016 +0200 blog/erlangen-intro sketch. diff --git a/blog/erlangen-intro.meta b/blog/erlangen-intro.meta new file mode 100644 index 0000000..bbbba5f --- /dev/null +++ b/blog/erlangen-intro.meta @@ -0,0 +1,4 @@ +:document (:title "Erlangen: Introduction" + :author "Max Rottenkolber " + :index-headers-p nil) +:publication-date nil diff --git a/blog/erlangen-intro.mk2 b/blog/erlangen-intro.mk2 new file mode 100644 index 0000000..9cb39ac --- /dev/null +++ b/blog/erlangen-intro.mk2 @@ -0,0 +1,70 @@ +[Erlangen](https://github.com/eugeneia/erlangen) brings distributed, +asynchronous message passing to [Clozure CL](http://ccl.clozure.com/). It +orchestrates Clozure CL _processes_ (native threads) using [message passing](http://c2.com/cgi/wiki?MessagePassingConcurrency), +and encourages fault tolerant software architectures using [supervison trees](http://erlang.org/documentation/doc-4.9.1/doc/design_principles/sup_princ.html). +It is also transparently distributed, all its features work seamlessly across +IP networks. Thus, it can be used to build applications across multiple +Clozure CL instances on different hosts. Erlangen borrows many ideas from +[Erlang/OTP](http://www.erlang.org/), hence the name. (Its a town!) + +Clozure CL processes are comparatively heavy weight, preemptively scheduled +operating system threads, as opposed to Erlang’s cooperatively scheduled green +threads. As such, processes are a scarce resource on Clozure CL. Erlangen is +based on the assumption that even with only a limited number of processes +available, message passing and supervision trees are still desireable features. +While Erlang’s distribution features are sometimes overlooked, they are meant +to be a main focus of Erlangen (but not of this article). Besides the overlap +in features, terminology, and name, Erlangen and Erlang/OTP are completely +unrelated, incompatible, and fundamentally different. + +#media# +erlangen-intro-send.svg + +Erlangen equips each process with a _mailbox_, a list of _links_, and a list of +_monitors_. These upgraded processes are called _agents_. Agents can [send](http://mr.gy/software/erlangen/api.html#section-1-13) +messages to the mailboxes of other agents, and [receive](http://mr.gy/software/erlangen/api.html#section-1-9) +the next message from in its own mailbox. When {send} fails to deliver a +message it signals an error. Message delivery can fail when the target agent +exited, the destination _mailbox_ is full (mailboxes are bounded FIFO queues), +or due to network failure. When {send} returns the message is guaranteed to be +delivered to the target agent’s mailbox. Agents can supply a timeout argument +to {receive}, which will then signal an error if no message is received until +the timeout is exceeded. + +#media# +erlangen-intro-mailbox.svg + +Agents can also _exit_ (using [exit](http://mr.gy/software/erlangen/api.html#section-1-6) +or by returning normally), and send _kill messages_ to other agents. When an +agent receives a kill message it exits immediately, and the remaining messages +in its mailbox are discarded. When {exit} fails to deliver the exit message it +signals an error. The links and monitors of an agent are references to other +agents. When an agent exits it attempts to send kill messages to all its links, +and _exit notifications_ to all its monitors. Agents can use [link](http://mr.gy/software/erlangen/api.html#section-1-7) +and [unlink](http://mr.gy/software/erlangen/api.html#section-1-17) to manage +links and monitors. + +#media# +erlangen-intro-link.svg + +Finally, agents can [spawn](http://mr.gy/software/erlangen/api.html#section-1-15) +new agents. {spawn} takes a nullary function for the agent’s process to +execute, and returns a reference to the newly created agent. This reference can +then be used to communicate or link with the agent. Additionally, {spawn} also +can also link the caller and the new agent before the new agent’s process +starts. This is useful to avoid a basic race condition where the child agent +exits before the parent could link with it. + +That covers pretty much all of the core functionality, but Erlangen is an +asynchronous framework for a synchronous programming language, and that poses +some problems. Our processes are preemtively scheduled, and it is unsafe to +stop them asynchronously. Er + + that does not call {receive} periodically + + messages can be lost even though they were delivered successfully + + exit notifications can be lost + + [select](http://mr.gy/software/erlangen/api.html#section-1-12) +