Programming with “Behaviors” in Erlang/OTP

P. Madanasekaran

Introduction

Joe Armstrong, co-creator of Erlang said   “ I didn’t set out to invent a new programming language. At the time we wanted to find a better way of writing the control software for a telephone exchange... …… So, I wrote a Prolog meta-interpreter that added parallel processes to Prolog, and then I added error handling mechanisms, and so on. After a while, this set of changes to Prolog acquired a name, Erlang”. Later Prologue interpreter was replaced with VM in C to improve performance. Even in syntax Prologue style code was replaced with functional style in later versions. Certain unforeseen events also helped. The team had to step in as one of the largest telecom projects using C++ within Ericsson collapsed. Then the Erlang project moved from the lab into production and was used to create AXD 301 ATM switch, one of the successful products of Ericson, which ran for ten years with nine 9s uptime(literally without any break).

 

Concurrent-Functional programming  language for Process-Oriented programming.

Massive fine-grained concurrency with light-weight processes that communicate through messages is a primary requirement of telephone systems, as a large number of simultaneous calls have to be taken care of. Armstrong started with a prototype in Prologue and added concurrency to it and for the purpose used constructs such as modules, process and process-communication from Concurrent languages such as Modula, Ada, Chill  etc. But the language also acquired a functional style, borrowing atoms, lists, guards, pattern-matching from functional languages such as, ML, Miranda etc. Combining the constructs from the two paradigms, it came out as a concurrent -functional language which created a new and unique paradigm- Process-Oriented Programming (POP). An Erlang app consists of a number of concurrently executing light-weight processes that were spawned from functions and communicate only through asynchronous messages. To create Erlang projects, you should know how to create useful functions, from which you can spawn the processes and link them for handling errors .Instead of handling an error in the process where the error occurs, Erlang lets the process die and correct the error by some other process in the link set, which makes building highly fault tolerant applications very easy. But knowing how to create and manage processes in this Process-oriented programming is not sufficient. You have to design carefully to avoid race conditions and deadlocks that may occur once in a while. Ericsson encountered these problems early when developing Erlang and created a set of libraries that eases them. OTP, the Open Telecom Platform, is useful for pretty much any large-scale project you want to do with Erlang, not just telecom work. It’s included with Erlang, and though it isn’t precisely part of the language. The boundaries of where Erlang ends and OTP begins aren’t always clear, but behaviors are an important subset of OTP that saves you from a lot of maintenance headaches. You can use processes built with behaviors and managed by supervisors in an OTP application and avoid the race conditions and deadlock that may otherwise beset your application.

OpenTelecom Platform(OTP)- A Poor Name for a Rich Framework.

As mentioned above, Open Telecom Platform (OTP) is a framework originally developed as part of Erlang for the telephone networks. It was initially used to build telephone exchanges and switches. But these devices have the same characteristics we want from any large online application, so OTP is now a general-purpose tool for developing and managing large systems. Erlang/OTP is actually a bundle that includes Erlang, a database- Mnesia, web-server, FTP server and an innumerable number of libraries like 1) compiler 2) run-time 3) Kernel 4) SASL(standard behaviors) 5) stdlib. OTP also defines a structure for our applications.

We will concentrate on “Standard Behaviors” a subset of OTP. What is a behavior? It is a library of components coded and arranged according to some well-known design patterns 1) gen-server(Client-Server) 2) gen_fsm(Finite State machine) 3) gen_event(Event-Handlers). Please see Figure-3.It is the normal structure of any Erlang OTP application. When you decide to use any of the above behaviors, you need not bother about spawning a process, as the behavior will run in its own process. The supervisor behaviors can be used to take care of error handling, in place of links and monitors, as we will see in the sample later. The Application behavior can be used to supervise the Supervisor behavior or as the root supervisor (For an example see Chapter 4.1.2 of Erlang  OTP in Action) – we will see some sample in the next article .

Programming model of  OTP behaviors:

Starting the server process, sending message to it and receiving reply from it, error handling, stopping the server are all functionalities found in any Erlang app. The content of the message or reply will differ according to our business logic. The first “Generic” functionalities are provided in libraries or behaviors. The second specific/callback functionalities have to be coded by the programmer for his specific needs. To help him templates that generate callback functions have been provided, he can override only those functions he actually uses. The programming model of Erlang OTP behaviors is somewhat similar to JEE model. Please have a look at the figure below. When the client calls the interface/ API functions, actually a message is sent to the server process. The first message is for starting the server. The library function “start_link”  starts the server process but you can tell in the init callback method how it should be initialized. When client makes a function call, the server process executes both the library/behavior function and the related Callback function. There is one–to-one mapping between a library function and the callback function. The response received by the client is the result of this combined execution of library and callback functions. The templates provide default callback implementations for all the exported interface functions. You can override any of them so that it can handle the custom logic. If you don’t provide any implementation for a function, the default implementation provided by the template will be used. When we see the sample it will become clear.



Sample

Our sample uses gen-server and supervisor behaviors. As shown in the earlier article, 
in Intelli J Idea create a New -> Erlang project. When you create the new Erlang file, you can change the Kind to OTP gen-server. Fill up the name.eg factorial_server

Click Ok the generated code includes a lot of comments and –spec functions. For the time being, you can delete them and have a look at the bare code.
The code may look as under:

-module(factorial_server).
      -author("admin").
-behaviour(gen_server). %----------1)
%% API -export([start_link/0]). %--------2)
%% gen_server callbacks -export([init/1,
handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).

-define(SERVER, ?MODULE)%----------3)
-record(state, {}).
%%% API
start_link() ->
gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). %---2)
%%% gen_server callbacks
init([]) ->
{ok, #state{}}.  % ---------4)
handle_call(_Request, _From, State) ->  %------5)
{reply, ok, State}.
handle_cast(_Request, State) ->   % ---------6)
{noreply, State}.
handle_info(_Info, State) ->
{noreply, State}.
terminate(_Reason, _State) ->
ok
.
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%% Internal functions
  • The declaration of behavior as “gen-server” enables the Template to generate necessary functions, for this design pattern.
  • The function start_link  API function called by the client, invokes the library function start_link, which starts the server process. This library function takes four arguments. 1) alias name of the process -local/global 2) Name of module, where the callback functions will be found. The other two arguments are empty. We can use them, if we want, to pass other arguments and options.
  • ? Server -Macro which will replace this with actual value- here “factorial_server”.
  • The init() will be called as soon as the server is started. The server is initialized by this method and only after the initialization is complete the start_link will return ok. In other words The start_link  is synchronous  and it waits for the response from init()  and only after getting the response, the start_link() returns {ok, Pid}.
  • The handle_call() is for synchronous calls. We will customize it.
  •  The handle_cast() is for asynchronous calls. Now we may not need this and other functions except terminate() which will be called when the server terminates.

Override some functions to customize them for your needs:

  • The init() may be overridden  to enable the supervisor to catch the error signal .
init([]) ->
 process_flag(trap_exit, true),
io:format(
"~p starting~n",[?MODULE]),
{ok,
0}.

Recall this process_flag was used when we created a custom supervisor process. We are using the same for initializing the server behavior. The flag  trap_exit true, will call terminate/2(default implementation) with the REASON as one of the arguments , when the server stops. The supervisor will catch the signal and take appropriate action.

We will override handle _call/3 to make synchronous calls to the server. ie if we want a reply from the sever. Otherwise we can override handle-cast/2.

Add the following API function below start_link function, to invoke handle_call/3.

  fact(N) ->
  gen_server:call(?
MODULE, {fact, N}).
 You can see once again an API function that makes a call to a library function.

You can see that the call() in gen_server library is invoked with module name and a tuple containing the message as arguments. The API functions that wrap messages or meant for sending messages are named the same thing as the message tag(fact). The message may be simple; but their actual format should not be leaked out of our module; they are wrapped by functions with the same name. That is another pattern and hallmark of well written Erlang/OTP code. The server process while executing gen_server:call/2 should  invoke handle_call/3 in the callback.

  • The handle_call/3 has to be overridden  as under:
handle_call({fact, N}, _From, State) -> {reply, factorial(N), State}.

You can see that the 1)  first parameter is the same  tuple that was passed to gen_server:call/2.That is how the two functions are mapped through pattern-matching. The second is Pid of the client process 3) every single GenServer function will reference the State which is used to pass updated data to recursive calls. Not significant in this sample as no State or recursion is used. In the tuple sent as response, the first element is a key word atom that tells that the next element is the response to be sent to the client -here the value returned by the private function factorial(N).The State mentioned above is the third element in the tuple.

  • The code for the internal function factorial (N), used above in the handle_call/3 is given below:
factorial(0) -> 1;
factorial(N) ->
  N * factorial(N-1).
We are leaving default implementations for other functions, as it is.

Supervisor

Erlang is a fault-tolerant language, in the sense that it does not try to prevent but cure errors without any loss of service. The processes with process_ flag trap_exit set to true can serve the purpose. When we use a behavior to build a process, you can use only the supervisor behavior to start and manage a server process and we cannot code and spawn the process on our own.

Create a New à Erlang File. You can fill up the wizard that appears as under.

In the generated code you can delete the comments and spec-functions as said earlier. You may be left with the following


-module(factorial_supervisor).
-author("admin").

-behaviour(supervisor). %% supervisor is also a behavior.
It is easy to add it other OTP behavior process . %%

%% API
-export([start_link/0,]).

%% Supervisor callbacks
-export([init/1]).

-define(SERVER, ?MODULE).


%%% API functions

start_link() ->
  supervisor:start_link({local, ?SERVER}, ?MODULE, []). %----1)

%%% Supervisor callbacks


init([]) ->    %-------2)
  RestartStrategy = one_for_one, %-------3)
  MaxRestarts = 1000,
  MaxSecondsBetweenRestarts = 3600,

  SupFlags = {RestartStrategy, MaxRestarts, MaxSecondsBetweenRestarts},  %-------3)

  Restart = permanent,
  Shutdown = 2000,
  Type = worker,
%% Child Specification contains details of the child processes it supervises
  AChild = {‘AChild’, {‘AChild’, start_link, []}, % -----4) 
    Restart, Shutdown, Type, [AChild’},
% return value of init
  {ok, {SupFlags, [AChild]}}. %------5)
%%% Internal functions
  1. The start_link() is similar to what we saw in gen_server behavior.
  2. Init(): This is where we tell the supervisor what to start and how to start it. In other words, the return value contains both Supervisor and Child specifications which tell the supervisor what process to supervise and how. The value returned by it to the behavior container is shown as 5.
  3. This is supervisor specification. This will be part of value returned by init.

       4)  This tuple is “Child Specification”. ie the description of the processes you want the Supervisor to supervise. The processes described in the Child Spec are started when the Supervisor starts and   exist for the life of the Supervisor itself. This will also be a part of the return value of init.  It’s just a tuple of six elements. We can modify it as under:

AChild = {factorial_server, {factorial_server, start_link, []},
 
Restart, Shutdown, Type, [factorial_server]},

In the generated code  the variable name “AChild” is left untouched.In three places we have replaced AName, AModule, AModule it with “factorial_server”.The first instance is the name by which the supervisor will reference it. The second instance is the module name, which contains the function start_link, which the supervisor will use to start the child. The third instance is the modules which have to be used when there is hot code upgrades. The other three words were defined earlier in the generated code marked as 3.

i) Restart- “permanent” – the Child process is not transient or temporary and will live for some-time,
ii)Shutdown – 2000- number indicates that this is a soft shutdown and the process will have that number of milliseconds to actually stop.
iii) Type- worker. It indicates the process is a worker, as supervisor can also be supervised.

5)  The tuple below is the return value of init().

The first value, the atom 'ok', tells the behavior container the initialization is ok.
The second is a two value tuple. The first value of this inner tuple is a supervisor specification which tells the supervisor how it should handle, if a child process supervised by it stops abnormally. See the value generated for SupFlags a) Restart Strategy.  one_for_one. That means that if a process dies only that process is to be restarted. There are more complex strategies available. b) MaxRestarts: the maximum number of restarts allowed. C) MaxSecondsBetweenRestarts:the timeframe for those restarts.

The second value in the inner tuple is the child specification mentioned above
“AChild” refers to the “AChild”-child specification defined earlier.

Only after seeing this return value from init ,the function start_link that starts supervisor behavior process is complete.

As we want to test everything in shell we will add the following function after start_link(), using the generated start_link. 

start_in_shell() ->
 {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = []),
 unlink(Pid).  
Build and Run You can choose Build -> Make Project.
When you get the message “Compilation succeeded successfully”
Choose Run -> Edit Configurations. The wizard will appear as Figure-1.
Expand ‘+’ and 

Choose Erlang Console and if you expand “use codePath and SDK of module” you can see “FactorialOTP”. Click on it and Wizard will appear as in Figure-2.click “OK”.

Choose Run -> Run UnNamed. You can see the Erlang shell in the IDE now.

1> factorial_supervisor:start_in_shell(). %%Execute
factorial_server starting   %% Result
true

 

factorial_server:fact(5).  %%Execute

120   %% Result

 

factorial_server:fact(five). %%Execute

factorial_server starting  %% server restarted by the supervisor as it crashed, and a part of the errors logged are given below:

 

** Generic server factorial_server terminating
** Last message in was {fact,five}
** When Server state == 0
** Reason for termination ==
** {badarith,
[{factorial_server,factorial,1,
[{file,"e:/lunaSpace/FactorialOTP/src/factorial_server.erl"},
{line,75}]},
{factorial_server,handle_call,3,
[{file,"e:/lunaSpace/FactorialOTP/src/factorial_server.erl"},
{line,54}]},
……….

Study the error message to understand what went wrong. Joe Amstrong the chief creator of Erlang says “Using gen_server, gen_supervisor, and so on, Erlang has been used to build systems with 99.9999999 percent reliability (that’s nine 9s). Used correctly, the error handling mechanisms can help make your program run forever (well, almost). The error logger described here has been run for years in live products”

If you execute
factorial_server:fact(4).
You can see the result.
24
This shows the supervisor has restarted the server.

Conclusion

When you create a Supervision Tree using Processes and Links and trap_exit flags, without using “Behaviors” you have to safe-guard against race-conditions, deadlocks and other errors, while designing. As “Behaviors” are carefully engineered and well tested Components and you can use them, without bothering about the pitfalls, and focus on your business logic. But there is a price. When majority of your processes are transient there will be some performance reduction when compared to the performance of “raw” processes and links, as you will be using blocking synchronous calls. But its advantages outweigh this disadvantage. When the Erlang team faced race-conditions and deadlocks in an app created with process trees with links and monitors, OTP behaviors and supervisors were developed to tackle them. In the words of Joe Anderson the chief creator of Erlang “The power of OTP comes from the fact that properties such as fault tolerance, scalability, dynamic-code upgrade, and so on, can be provided by the behavior itself”. The writer of the call-back does not have to worry about the race conditions and deadlocks. Above all it enables him to write sequential code coding some functions and making function calls, without bothering about the underlying concurrency constructs like spawn(),send and receive.

So if your app needs high concurrency, fault-tolerance and low latency, Erlang/OTP should be your first choice. Dave Thomas the author of Programming Elixir says “Elixir uses the OTP framework for constructing process trees, and OTP includes the concept of process supervision. An incredible amount of effort has been spent getting this right, so I recommend using it most of the time”. This applies more to Erlang/OTP, the original. The talks and discussions available on YouTube, especially the talk by Joe Amstrong, “26 years with Erlang or how I got my grey hairs” not only tells the ordeals and travails faced by genuine creators but also how the language evolved in their attempts to meet the challenges. As Eric Stanmen from Klarna, a company providing web-services using Erlang, said “the most important aspects is that it makes you to think of a problem in a new way. In a concurrent way”. It may take some time to absorb the language syntax, as it is very concise. Another option available is “Elixir ,another language on Erlang VM, but with a Ruby-like syntax”. Scala and Clojure created for JVM as Java was not functional(then) and it was also considered verbose with boiler-plates. Elixir was created for exactly the opposite reason. Erlang is considered terse and “Prologuish”. Elixir is “Ruby-like” in syntax  but Erlang compatible- everything  in Erlang has been implemented in Elixir but some features from Clojure –protocols , lisp-like macros and tooling support- have also been added  .We will cover it in future articles.








}