Download Sphinx-system-design

Main Sphinx Design Concepts
•
There are two primary design components which comprise Sphinx
• The Database Warehouse
• The Control Process
•
The Database Warehouse is the information depot for all of Sphinx
• All information is stored/retrieved via the database (or possibly multiple
databases)
• There is no direct communication between different internal Sphinx modules
• The API for all internal modules is SQL along (with the database schema)
•
The Control Process is the main point of activity
• It is a loop which calls different Sphinx modules
• All modules in the control process operate on stateful entities in the database,
changing the state of an entity from one to another state
• Sphinx operates as a Finite State Machine!
• Uses data mining principles in the design:
• The API for all modules is SQL along the database schema
• Modules mine the database for information and then take some action.
Version 1: Sphinx Server/Client
Single-threaded
Control Process
Data Warehouse
Message Interface
Sphinx Client
Message Interface
DAG Reducer
Prediction Engine
Job Tracker
Job Submitter
Site Tables
Condor-G/DAGMan
VDT Client
Job Execution Planner
Resource
Job Table
DAG Table
Replica Table
Information Gatherer
Monitoring Service
Message Tbl.
Sphinx Server
VDT Server Site
More Details
• The next slides show the message passing between
different Sphinx components
• The components are split (packaged) in the following
way:
• Sphinx Client
• Incomming Messages
• Outgoing Messages
• Sphinx Server
• Incomming Messages
• Control Process
• Outgoing Messages
• See http://ufgrid.phys.ufl.edu:8080 for documentation
• The next version of Sphinx (currently in progress) will
split/package the components into a more logically
distinct, modular scheme (yet to be decided)
Version 1: Sphinx Client (outgoing message)
Message ID
Msg Type
Msg Sender
Message
Clinet
ACK
Files
Message Type
Path to file(DAX)
Path to User ID file
Msg strings
Sender Info
XML RPC
XML message
Destination URL
ClinetOutModule
MsgID
Msg Type
Msg Sender
Message
XML message
MsgHandler
XMLDocUtils
MsgID
Msg Type
Msg Sender
Message
XML message
DOMparser
SendRequest
ACK
ACK
Version 1: Sphinx Client (incoming message)
JobWrapper
job ID
dag ID
Message string
SubmissionGenerator
Msg string
Work Dir
ACK
ClientIncModule
XML message
Msg type
Message string
XML message
MsgHandler
XML message
DOMparser
Msg ID
Msg Type
Msg Sender
Message
createSubmission
Super dag file
execute
condor_submit_dag
Version 1: Sphinx Server (incoming message)
DB Connector
Msg Type
Msg Sender Info
Message
XML message
InMessageWrapper
IncModule
ACK
(Msg ID)
Msg Id
Msg Type
Msg Sender Info
Message
XML message
MsgHandler
XML message
MessageWrapper
checkIn
Dag
message
Msg Id
Msg Type
Msg Sender Info
Message
checkIn
checkOut
remove(msg)
Job
DOMparser
AbstractDagParser
Dag Obj
Dag Info
Dagwrapper
Job Obj
Job Info
Jobwrapper
checkIn
Version 1: Control Process
Control Process
UNREDUCED
Dag Table
UNPREDICTED
DagReducer
UNPREDICTED
PredictionEngine
UNFINISHED
UNPREDICTED
Planner
UNPLANNED
Job
Table
UNPLANNED
Job, Strategy
execute
SchedulingModule
Sites Info
Exec. site info
Input file info
Output file info
UNSENT
Physical site info
Site
Tables
JobWrapper
Message
Table
Check In / SUB_REQ
Client ID
Msg tpye
Job info string
MessageWrapper
OutMessageWrapper
Physical file info
Replica
Table
Version 1: Sphinx Server (outgoing message)
Msg ID
Client Info
Msg type
message
XML message
Destination URL
OutModule
OutgoingInterface
ACK
Next msg ID
message
SendRequest
ACK
MsgID
Msg Type
Client Info
Message
XML message
MessageWrapper
MsgHandler
Check Out
MsgID
Msg Type
Client Info
Message
Message
DOMparser
SOAP RPC
XML message
ACK
Version 2 of Sphinx (current)
• The current development cycle is represented in the next three
slides
• Essentially:
• More planning tables have been added to the database warehouse
• More planning modules have been added to the control process
• A Data Management Component is being added to the Sphinx Server
• Replica Management (via RLS and GridFTP)
• The Monitoring Interface was upgraded to use MonALISA
• We also want to investigate the possibility of making a “Clarens
service” out of any arbitrary planning module in the control
process
• The Job Submitter, Job Tracking, and Condor-G/DAGMan (which
currently form the Sphinx Client) is an example.
Version 2:
Sphinx Server with Sphinx Submission Client
Multi-threaded
Control Process
Data Warehouse
Message Interface
Sphinx Client
Message Interface
DAG Reducer
Job Predictor
Policies
Accounting
Resource Prop.
Resource Hier.
Resource Dist.
Distance
Entity-Name
Entity Hier.
Job-Entity Prop.
Job
DAG
Rel. Data
Rel. Appl.
Job Tracking
Grid Weather
Schedule
Prediction
Message
DAG Predictor
Job Tracker
Job Submitter
Job Admission Control
DAG Admission Control
Condor-G/DAGMan
DAG Data Repl. Planner
VDT Client
Job Execution Planner
Resource
DAG Tracker
Data Management
Information Gatherer
Sphinx Server
Repl. Loc. Serv.
Monitoring Service
VDT Server Site
Version 2:
Sphinx Server with Sphinx UI Client
Multi-threaded
Control Process
Data Warehouse
Sphinx Client
Message Interface
Message Interface
DAG Reducer
Job Predictor
Entity Hier.
Job-Entity Prop.
Job
DAG
Rel. Data
Rel. Appl.
Job Tracking
Grid Weather
Schedule
Prediction
Message
DAG Predictor
Job Admission Control
DAG Admission Control
DAG Data Repl. Planner
Job Execution Planner
Job Submitter
Job Tracker
DAG Tracker
Data Management
Information Gatherer
Sphinx Server/VDT Client
Condor-G/DAGMan
Policies
Accounting
Resource Prop.
Resource Hier.
Resource Dist.
Distance
Entity-Name
UI Machine
Resource
Repl. Loc. Serv.
Monitoring Service
VDT Server Site
Version 2: Finite State Machine Control Process
DAG: unreduced
JOB: unpredicted
no
Await Mesg.
yes
DAG: unpredicted
JOB: unpredicted
no
Reduce DAG
yes
DAG: unpredicted
JOB: unaccepted
no
Predict Job
no
Predict DAG
DAG: unfinished
JOB: unsent
DAG: unfinished
JOB: remove
no
Admit Job
DAG: remove
JOB: remove
yes
Remove DAG
yes
no
Plan Job
no
Submit Job
yes
yes
DAG: unplanned
JOB: unplanned
Plan Dag
yes
yes
DAG: unaccepted
JOB: unplanned
no
yes
yes
DAG: unaccepted
JOB: unaccepted
DAG: unfinished
JOB: unplanned
no
Admit DAG
no
Remove Job