 |
Maestro Open Toolkit: Clients/Servers with State Transfer Protocol
|
This document is a part of online
Ensemble documentation, under
Maestro Open Toolkit.
Maestro_ClSv is a public subclass of
Maestro_GroupMember.
It implements the abstraction of clients/servers with a
state transfer protocol.
Index of Topics:
Creating client/server objects
Sending messages
Participating in membership changes
State transfer
Index of Downcalls/Callbacks:
cast
send
scast
lsend
askState
sendState
xferDone
clSv_ReceiveCast_Callback
clSv_ReceiveSend_Callback
clSv_ReceiveScast_Callback
clSv_ReceiveLsend_Callback
clSv_Block_Callback
clSv_ChangingView_Callback
clSv_ProposeView_Callback
clSv_AcceptedView_Callback
clSv_askState_Callback
clSv_rcvState_Callback
The Maestro_ClSv class provides the following constructor:
Maestro_ClSv(Maestro_ClSv_Options &ops);
Maestro_ClSv_Options is a public subclass of
Maestro_grpMemb_Options.
In addition to fields inherited from Maestro_grpMemb_Options,
Maestro_ClSv_Options has the following two public fields:
Maestro_MbrshipOption mbrshipType -- Specifies whether the member is created
as a server (mbrshipType = MAESTRO_SERVER) or as a client
(mbrshipType = MAESTRO_CLIENT, the default value).
Maestro_XferType xferType -- Specifies the protection level for state transfer.
The value of xferType is one of the following:
MAESTRO_NO_XFER -- State transfer is not required (the default value).
MAESTRO_FREE_XFER -- Group members can send all types of messages
during state transfer.
MAESTRO_PROTECTED_XFER -- Group members can only send "safe" (state-neutral)
and state-transfer messages during state transfer.
MAESTRO_ATOMIC_XFER -- Only state-transfer messages can be sent during
state transfer.
Note that clients do not receive multicasts addressed to servers and do not
participate in state transfer.
The Maestro_ClSv class defines downcall methods for sending point-to-point
messages (send), multicasts to the entire group (cast),
multicasts to servers only (scast), and multicasts to group subsets
(lsend). Message downcalls are declared in Maestro_ClSv
as follows:
void cast(Maestro_Message &msg);
void cast(Maestro_Message &msg,
Maestro_MsgSendView &sendView);
void cast(Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_ClSv_MsgOptions &msgOps);
void send(Maestro_EndpID &dest,
Maestro_Message &msg);
void send(Maestro_EndpID &dest,
Maestro_Message &msg,
Maestro_MsgSendView &sendView);
void send(Maestro_EndpID &dest,
Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_ClSv_MsgOptions &msgOps);
void scast(Maestro_Message &msg);
void scast(Maestro_Message &msg,
Maestro_MsgSendView &sendView);
void scast(Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_ClSv_MsgOptions &msgOps);
void lsend(Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_ClSv_MsgOptions &msgOps);
The dest argument specifies the
endpoint ID of the destination
of a point-to-point message.
The sendView argument has the same semantics as in
message downcalls of the Maestro_GroupMember class.
The msg parameter contains the
message to be sent.
The Maestro_ClSv_MsgOptions class (of the msgOps argument)
is defined in Maestro_ClSv with the following public fields:
Maestro_MsgXferSafety msgXferSafety -- Specifies the state-transfer
safety level of a message. The value of msgXferSafety can be one
of the following: MAESTRO_MSG_GENERIC (the default value) -- for
regular messages; MAESTRO_MSG_SAFE -- for "state-safe" messages;
MAESTRO_MSG_XFER -- for state-transfer messages.
During MAESTRO_FREE_XFER state transfers, all messages may be sent,
and it is left up to the application to guarantee consistency of the global state.
During MAESTRO_PROTECTED_XFER state transfers, only messages with the
MAESTRO_MSG_SAFE or MAESTRO_MSG_XFER safety levels may be sent.
Messages with MAESTRO_MSG_GENERIC safety level are delayed by Maestro
until state transfer completes.
During MAESTRO_ATOMIC_XFER state transfers, only state-transfer messages
(with MAESTRO_MSG_XFER safety level) may be sent.
All other messages are delayed by Maestro until state transfer completes.
Maestro_EndpList destList -- Specifies the list of client destinations
for an scast message; specifies the list of all destinations for an
lsend message. By default, destList is empty.
An scast downcall sends a message to all servers and to an (empty by default)
list of clients. The client destinations are specified with the destList option.
For example:
Maestro_Message msg;
msg << Maestro_String("hello");
memb.scast(msg); // Send msg to servers only.
....
HorsEndpID eid;
....
Maestro_ClSv_MsgOptions mops;
mops.destList += eid; // Assuming eid is a client member of the group.
memb.scast(msg, mops); // Send msg to servers and to eid.
The lsend downcall sends a message to the (empty by default) list of
destinations included in destList.
For example:
Maestro_Message msg;
msg << Maestro_String("world");
Maestro_EndpID endp1, endp2;
....
Maestro_ClSv_MsgOptions mops;
mops.destList += endp1;
mops.destList += endp2; // Assuming endp1 and endp2 are members of the group.
memb.lsend(msg, mops); // Send msg to endp1 and endp2.
When a message is received at a destination, Ensemble invokes the corresponding
message callback. The Maestro_ClSv class defines the following
message callbacks corresponding to cast, send, scast,
and lsend downcalls respectively:
void clSv_ReceiveCast_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
void clSv_ReceiveSend_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
void clSv_ReceiveScast_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
void clSv_ReceiveLsend_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
The message callbacks are defined in Maestro_ClSv as no-op functions.
They are intended to be overloaded in subclasses of the Maestro_ClSv
class to implement application-specific functionality.
The Maestro_ClSv class defines the following callback methods
invoked by Ensemble during membership changes:
void clSv_Block_Callback() {}
void clSv_ProposeView_Callback(Maestro_ClSv_ViewData &viewData,
Maestro_MessageList &stateMsgs,
Maestro_Message &viewMsg) {}
void clSv_AcceptedView_Callback(Maestro_ClSv_ViewData& viewData,
Maestro_Message &viewMsg) {}
void clSv_ChangingView_Callback(Maestro_ClSv_ViewData &viewData,
Maestro_Message &stateMsg) {}
The clSv_Block_Callback(),
clSv_ChangingView_Callback(), clSv_ProposeView_Callback,
and clSv_AcceptedView_Callback() methods have the same semantics as
corresponding
membership-change-protocol callbacks of the Maestro_GroupMember class.
The callbacks are defined in Maestro_ClSv as no-op functions, and are intended
to be overloaded in subclasses of the Maestro_ClSv class to implement
application-specific functionality.
The
Maestro_ClSv_ViewData structure adds the following
fields in addition to those inherited from the
Maestro_GrpMemb_ViewData class:
Maestro_EndpList servers -- The list of servers in the new view.
Maestro_EndpList newServers -- Servers added since the previous view.
Maestro_EndpList departedServers -- Servers removed from the previous view.
Maestro_EndpList xferServers -- Servers doing state transfer in the new view.
Maestro_EndpList newXferServers -- State-transfer servers added since the previous view.
Maestro_EndpList departedXferServers -- State-transfer servers removed from the previous view.
Maestro_EndpList clients -- The list of clients in the new view.
Maestro_EndpList newClients -- Clients added since the previous view.
Maestro_EndpList departedClients -- Clients removed from the previous view.
Maestro_EndpID serverCoordinator -- The server with the lowest rank in the view.
Maestro_EndpID oldServerCoordinator -- The server coordinator of the previous view.
Maestro_ViewType myXferType -- Same as the value passed to the member object's constructor.
Maestro_ViewType viewType -- The type of state transfer in progress (if any).
Maestro_ClSvState state -- The state of the member in the new view (one of
MAESTRO_CLSV_STATE_CLIENT_NORMAL,
MAESTRO_CLSV_STATE_BECOMING_SERVER,
MAESTRO_CLSV_STATE_SERVER_XFER,
MAESTRO_CLSV_STATE_SERVER_XFER_DONE,
MAESTRO_CLSV_STATE_SERVER_NORMAL).
Maestro_ClSvState oldState -- The state of the member in the previous view.
int myServerRank -- The rank of the member among servers in the new view.
int myOldServerRank -- The rank of the member among servers in the previous view.
Maestro_MbrshipOption myMbrshipType -- Same as the value passed to the member
object's constructor.
int startXfer -- The flag is set when state transfer has to be
(re)started in the new view (which happens when the state of the
group member changes to MAESTRO_CLSV_SERVER_XFER).
If state transfer is in progress in a view, the viewType field in the
corresponding viewData structure specifies the required safety level
(one of MAESTRO_FREE_XFER, MAESTRO_PROTECTED_XFER, MAESTRO_ATOMIC_XFER).
If the safety level of a message is lower than that required by state transfer,
then Maestro will delay sending the message until state transfer completes.
The Maestro_ClSv class implements a
State Transfer Protocol and provides the following
interface to it.
Maestro notifies a server member that state transfer has to be (re)started
by setting the startXfer flag in the
viewData structure.
To request a portion of the state, the member can invoke the askState downcall,
which is declared in Maestro_ClSv as follows:
void askState(Maestro_EndpID &svr, Maestro_Message &msg);
The svr argument specifies the server from which a portion of the state is being
requested. The msg argument contains the state request message which tells the
server which part of the state is being asked for.
When a normal server receives a state request (submitted with a call to askState),
it invokes the askState_Callback method, which is defined in Maestro_ClSv
as follows:
void askState_Callback(Maestro_EndpID& origin, Maestro_Message &msg) {}
The origin argument identifies the member receiving the state. The msg
argument contains the state-request message.
When a normal server receives a state request, it should eventually respond by sending the
requested portion of the state to the state-recepient member. To send a state message,
the server can invoke the sendState downcall, which is declared in
Maestro_ClSv as follows:
void sendState(Maestro_EndpID &xferSvr, Maestro_Message &msg);
The xferSvr argument specifies the endpoint ID of the state-recepient member
(it should be the same as the origin argument in the corresponding
invocation of askState_Callback). The msg argument contains the
reply message with the requested portion of the state in it.
After a state-recepient member receives a state message (sent by a normal server
with a call to sendState), the member invokes the
rcvState_Callback method, which is defined in Maestro_ClSv as
follows:
void rcvState_Callback(Maestro_EndpID& origin, Maestro_Message &msg) {}
The origin argument identifies the sender of the state message.
The message is passed with the msg argument.
The askState_Callback and rcvState_Callback methods are defined
in Maestro_ClSv as no-op functions. They are intended to be overloaded
in subclasses of Maestro_ClSv to implement application-specific functionality.
After a state-recepient member completes state transfer, it should invoke the
xferDone method. Following that, Ensemble will eventually install a new view
in which the member will be included in the list of normal servers.
The state transfer interface of the Maestro_ClSv class is at a rather low level,
albeit most general. The
Maestro_CSX class (a subclass of
Maestro_ClSv) implements a higher-level state transfer interface, which should
be adequate for many applications.
send mail to
alexey@cs.cornell.edu