[Monetdb-developers] mindmap

Fabian Fabian.Groffen at cwi.nl
Wed Apr 28 11:31:29 CEST 2004


I would certainly add at
Mguardian/Architecture/Implementation/Watchdog\ function/Connection\ 
server/"limits concurrent connections to a certain Mserver or the whole 
'cluster' (prevent going down the drain due to DoS attack)"

and

Mguardian/Architecture/Implementation/Watchdog\ function/Connection\ 
server/"reserves a (small number of) connection(s) for administrator access"

and

Mguardian/Architecture/Implementation/Management\ function/"kills 
connections on request of DBA"


Martin Kersten wrote:
> Updated version. (Should find a place in the MonetDB Repository for it)
> 
> Inclusion of Peter's comments.
> - Mserver becomes a watchdog for local Mserver Instances (other name 
> needed)
> 
> Mguardian itself is a process to manage a community of Mservers, who 
> have delegated some control.
> 
> Please add comments.
> 
> 
> ------------------------------------------------------------------------
> 
> <map version="0.7.1">
> <node TEXT="Mguardian">
> <node TEXT="Introduction" POSITION="right">
> <node TEXT="Automated crash recovery" FOLDED="true">
> <node TEXT="Single Mserver failure survivals"/>
> <node TEXT="Fail-over in case of a hardware crash"/>
> </node>
> <node TEXT="Resource management" FOLDED="true">
> <node TEXT="Dbfarm management" FOLDED="true">
> <node TEXT="Creation of necessary directories"/>
> <node TEXT="Assurance of sufficient space"/>
> </node>
> <node TEXT="Load distribution on multiple servers" FOLDED="true">
> <node TEXT="Forwarding query clients to slave"/>
> </node>
> <node TEXT="Automatic backup/recovery management" FOLDED="true">
> <node TEXT="Day/weekly schedules"/>
> <node TEXT="Highwatermark scheduling"/>
> </node>
> <node TEXT="Disk volume management" FOLDED="true">
> <node TEXT="Multi-hierarchical chkpoint stores"/>
> <node TEXT="log compression"/>
> </node>
> </node>
> <node TEXT="Monitoring" FOLDED="true">
> <node TEXT="Lifeness of all Mserver instances in the convoy"/>
> <node TEXT="Resource usage" FOLDED="true">
> <node TEXT="MonetDB client load"/>
> <node TEXT="Host resource load"/>
> </node>
> </node>
> <node TEXT="Authorization" FOLDED="true">
> <node TEXT="User administration"/>
> <node TEXT="Granting access to services"/>
> </node>
> </node>
> <node TEXT="Design goals" POSITION="right">
> <node TEXT="appearance">
> <node TEXT="Transparent to users">
> <node TEXT="Becoming a member of a guardian group &#xa;is an option in  the Mserver configuration file."/>
> </node>
> <node TEXT="Transparent to location">
> <node TEXT="Cluster machines are supported"/>
> <node TEXT="loosely coupled servers through HTTP"/>
> </node>
> <node TEXT="Transparent to hw/sw">
> <node TEXT="Platform independent code"/>
> <node TEXT="Python? Java?"/>
> </node>
> </node>
> <node TEXT="Convoys" FOLDED="true">
> <node TEXT="The guardian manages a group of database services"/>
> <node TEXT="The guardian process acts as a servant to those requiring it"/>
> <node TEXT="It is a free association, i.e. a Mserver can at any time&#xa;join/leave the group. Depending on the way this is&#xa;realised, Mguardian may initiate an action"/>
> </node>
> <node TEXT="State management">
> <node TEXT="Driven by XML configuration file and state file"/>
> <node TEXT="Ascii-based state management at transaction level"/>
> <node TEXT="Distributed transaction management"/>
> </node>
> <node TEXT="Multiple points of failure infrastructure" FOLDED="true">
> <node TEXT="Multiple servers in a cluster"/>
> <node TEXT="Mguardian lifes outside the area of Mservers?"/>
> </node>
> <node TEXT="Web-based status monitoring" FOLDED="true">
> <node TEXT="At the level of a single server"/>
> <node TEXT="At the level of managin multiple Mservers"/>
> </node>
> <node TEXT="Active warning system" FOLDED="true">
> <node TEXT="Email warning to DBA"/>
> </node>
> </node>
> <node TEXT="Architecture" POSITION="right">
> <node TEXT="Configuration files" FOLDED="true">
> <node TEXT="XML-based structure of Mserver Farms"/>
> <node TEXT="Database specific management conditions"/>
> <node TEXT="Disaster recovery rules"/>
> </node>
> <node TEXT="Implementation">
> <node TEXT="Watchdog function" FOLDED="true">
> <node TEXT="A lightweight independent process that keeps that&#xa;keeps a specific Mserver instance alive">
> <node TEXT="It 'behaves' as the main thread of Mserver"/>
> <node TEXT="It should maintain information for restart" FOLDED="true">
> <node TEXT="how to know this  ?"/>
> <node TEXT="By being part of Mserver thread group"/>
> </node>
> </node>
> <node TEXT="Recover policy is database specific">
> <node TEXT="Simply restart Mserver using known script"/>
> <node TEXT="Rejoin the Mguardian convoy"/>
> <node TEXT="Set-back the latest chkpoint instance"/>
> </node>
> <node TEXT="Monitoring" FOLDED="true">
> <node TEXT="Collect session information"/>
> <node TEXT="Obtain webbased state of a single Mserver"/>
> </node>
> <node TEXT="Connection server">
> <node TEXT="Should take over Internet thread listener"/>
> <node TEXT="Authenticates valid connections"/>
> <node TEXT="Forward request to convoy if needed"/>
> </node>
> </node>
> <node TEXT="Management function" FOLDED="true">
> <node TEXT="Independent MonetDB application" FOLDED="true">
> <node TEXT="Runs  on secure system and keeps overview"/>
> <node TEXT="Mknife access possible"/>
> </node>
> <node TEXT="Manages allocation of connection requests"/>
> <node TEXT="Revives the watchdogs or call for DBA help"/>
> <node TEXT="Monitors high-water marks and calls the DBA"/>
> </node>
> </node>
> <node TEXT="Mguardian GUI" FOLDED="true">
> <node TEXT="A system wide control centre ">
> <node TEXT="Gui-based"/>
> <node TEXT="Controls instances in cluster"/>
> <node TEXT="Can collect information from autonomous nodes"/>
> </node>
> <node TEXT="It receives reports from the watchdogs" FOLDED="true">
> <node TEXT="Incremental load levels"/>
> <node TEXT="Database farm size"/>
> <node TEXT="log sizes"/>
> <node TEXT="Number of concurrent clients"/>
> <node TEXT="Machine load level"/>
> </node>
> <node TEXT="supports remotely initiated actions" FOLDED="true">
> <node TEXT="Watchdog is the proxy for local system actions"/>
> </node>
> </node>
> </node>
> <node TEXT="Implementation strategy" POSITION="right">
> <node TEXT="Phase 1: Mserver becomes a watchdog">
> <node TEXT="Performs rudimentary guardian actions &#xa;before admitting a Mserver Instance"/>
> <node TEXT="Performs auto reboot of Mserver Instance"/>
> <node TEXT="Tell users about  the language/instance port to use"/>
> <node TEXT="Keeps log of user sessions"/>
> </node>
> <node TEXT="Phase 2: Mserver monitoring">
> <node TEXT="Mguardian database design"/>
> <node TEXT="Webservice to collect  data" FOLDED="true">
> <node TEXT="HW load"/>
> <node TEXT="Free log space"/>
> <node TEXT="Database space"/>
> </node>
> <node TEXT="Simple admin control to change properties"/>
> <node TEXT="Prototyping as Mknife scripts"/>
> </node>
> <node TEXT="Phase 3: Mguardian in control">
> <node TEXT="Primary portal for client connectivity"/>
> <node TEXT="start/stop Mservers"/>
> </node>
> </node>
> </node>
> </map>




More information about the developers-list mailing list