Reference/API

Basic framework module

This is the main module implementing the framework.

@author: skwok

class keckdrpframework.core.framework.Framework(pipeline_name, configFile)

This class implements the core of the framework.

The processing is event driven. An event can be defined as a change in set of items of interest, for example files or directories, or something in memory. Events are appended to a queue. Events are associated with arguments, such time, name of files, or new values of some variables. In a loop, an event is taken out from the queue and translated into an action.

An action is a call to a regular function, a method in the pipeline class or in a regular class.

If desired, a Data_set() can created to keep a list of files in memory.

config

Instance of ConfigClass that uses the configuration file to create a set of configuration parameters

Type:ConfigClass

logger

Type:log

pipeline

pipeline can be a string, a module, a class or an object of subclass base_pipeline The pipeline that will be used in the framework

Type:pipeline

context

Instance of Processing_context, which passed along to all processing steps.

pipeline_name: name of the pipeline class containing recipes

Creates the event_queue and the action queue

append_event(event_name, args, recurrent=False)

Appends low priority event to the end of the queue

end()

Releases the event_queue. Needed when a client ingest_data and then quits.

event_to_action(event, context)

Returns an Action() Passes event.args to action.args. Note that event.args comes from previous action.output.

This method is called in the action loop. The actual event_to_action method is defined in the pipeline and it depends on the incoming event and context.state.

execute(action, context)

Executes one action The input for the action is in action.args. The action returns action_output and it is passed to the next event if action is successful.

get_event()

Retrieves and returns an event from the queues. First it checks the high priority queue, if fails then checks the regular event queue.

If there are no more events, then it returns the no_event_event, which is defined in the configuration.

ingest_data(path=None, files=None, monitor=False)

Adds files to the data_set. The data_set resides in the framework context.

init_signal()

Captures keyboard interrupt

main_loop()

This is the main action loop.

This method can be called directly to run in the main thread. To run in a thread, use start_action_loop().

on_exit(status=0)

Hook fo exit Subclasses can override to continue in the main_loop or call exit(status)

on_state(state)

Hook to change context state. Default is to ignore state = ‘stop’. To terminate, override this method to return ‘stop’.

start_action_loop()

This is a thread running the action loop.

wait_for_ever()

Because the action loops runs in a thread, this methods waits until keep_going is false.

keckdrpframework.core.framework.create_context(event_queue=None, event_queue_hi=None, logger=None, config=None)

Convenient function to create a context for working withtout the framework. Useful in Jupyter notebooks for example.