flowcraft.generator.recipe module

class flowcraft.generator.recipe.InnuendoRecipe

Bases: object

Methods

build_downstream(self, process_descriptions, …)	Builds the downstream pipeline of the current process
build_pipeline_string(self, forks)	Parses, filters and merge all possible pipeline forks into the final pipeline string
build_upstream(self, process_descriptions, …)	Builds the upstream pipeline of the current process
define_pipeline_string(self, …)	Builds the possible forks and connections between the provided processes
run_auto_pipeline(self, tasks)	Main method to run the automatic pipeline creation
validate_pipeline(pipeline_string)	Validate pipeline string

count_forks = None

int : counts the total possible number of forks

forks = None

list : a list with all the possible forks

pipeline_string = None

str : the generated pipeline string

process_to_id = None

dict: key value between the process name and its identifier

static validate_pipeline(pipeline_string)

Validate pipeline string

Validates the pipeline string by searching for forbidden characters

Parameters:

pipeline_string : str

STring with the processes provided

build_upstream(self, process_descriptions, task, all_tasks, task_pipeline, count_forks, total_tasks, forks)

Builds the upstream pipeline of the current process

Checks for the upstream processes to the current process and adds them to the current pipeline fragment if they were provided in the process list.

Parameters:

process_descriptions : dict

Information of processes input, output and if is forkable

task : str

Current process

all_tasks : list

A list of all provided processes

task_pipeline : list

Current pipeline fragment

count_forks : int

Current number of forks

total_tasks : str

All space separated processes

forks : list

Current forks

Returns

——-

list : resulting pipeline fragment

build_downstream(self, process_descriptions, task, all_tasks, task_pipeline, count_forks, total_tasks, forks)

Builds the downstream pipeline of the current process

Checks for the downstream processes to the current process and adds them to the current pipeline fragment.

Parameters:

process_descriptions : dict

Information of processes input, output and if is forkable

task : str

Current process

all_tasks : list

A list of all provided processes

task_pipeline : list

Current pipeline fragment

count_forks : int

Current number of forks

total_tasks : str

All space separated processes

forks : list

Current forks

Returns

——-

list : resulting pipeline fragment

define_pipeline_string(self, process_descriptions, tasks, check_upstream, check_downstream, count_forks, total_tasks, forks)

Builds the possible forks and connections between the provided processes

This method loops through all the provided tasks and builds the upstream and downstream pipeline if required. It then returns all possible forks than need to be merged à posteriori`

Parameters:

process_descriptions : dict

Information of processes input, output and if is forkable

tasks : str

Space separated processes

check_upstream : bool

If is to build the upstream pipeline of the current task

check_downstream : bool

If is to build the downstream pipeline of the current task

count_forks : int

Number of current forks

total_tasks : str

All space separated processes

forks : list

Current forks

Returns:

list : List with all the possible pipeline forks

build_pipeline_string(self, forks)

Parses, filters and merge all possible pipeline forks into the final pipeline string

This method checks for shared start and end sections between forks and merges them according to the shared processes:

[[spades, ...], [skesa, ...], [...,[spades, skesa]]]
    -> [..., [[spades, ...], [skesa, ...]]]

Then it defines the pipeline string by replacing the arrays levels to the flowcraft fork format:

[..., [[spades, ...], [skesa, ...]]]
    -> ( ... ( spades ... | skesa ... ) )

Parameters:

forks : list

List with all the possible pipeline forks.

Returns:

str : String with the pipeline definition used as input for

parse_pipeline

run_auto_pipeline(self, tasks)

Main method to run the automatic pipeline creation

This method aggregates the functions required to build the pipeline string that can be used as input for the workflow generator.

Parameters:

tasks : str

A string with the space separated tasks to be included in the pipeline

Returns:

str : String with the pipeline definition used as input for

parse_pipeline

class flowcraft.generator.recipe.Innuendo(*args, **kwargs)

Bases: flowcraft.generator.recipe.InnuendoRecipe

Recipe class for the INNUENDO Project. It has all the available in the platform for quick use of the processes in the scope of the project.

Methods

build_downstream(self, process_descriptions, …)	Builds the downstream pipeline of the current process
build_pipeline_string(self, forks)	Parses, filters and merge all possible pipeline forks into the final pipeline string
build_upstream(self, process_descriptions, …)	Builds the upstream pipeline of the current process
define_pipeline_string(self, …)	Builds the possible forks and connections between the provided processes
run_auto_pipeline(self, tasks)	Main method to run the automatic pipeline creation
validate_pipeline(pipeline_string)	Validate pipeline string

flowcraft.generator.recipe.brew_innuendo(args)

Brews a given list of processes according to the recipe

Parameters:

args : argparse.Namespace

The arguments passed through argparser that will be used to check the the recipe, tasks and brew the process

Returns:

str

The final pipeline string, ready for the engine.

list

List of process strings.

class flowcraft.generator.recipe.Recipe

Bases: object

Methods

brew

pipeline_str = None

str: The raw pipeline string, with no attribute or directives, except for number indicators for when there are duplicate components.

e.g.: “fastqc trimmomatic spades” e.g.: “fastqc trimmomatic (spades#1 | spades#2)

directives = None

dict: Dictionary with the parameters and directives for each component in the pipeline_str attribute. Missing components will be left with the default parameters and directives.

brew(self)

flowcraft.generator.recipe.brew_recipe(recipe_name)

Returns a pipeline string from a recipe name.

Parameters:

recipe_name : str

Name of the recipe. Must match the name attribute in one of the classes defined in flowcraft.generator.recipes

Returns:

str

Pipeline string ready for parsing and processing by flowcraft engine

flowcraft.generator.recipe.list_recipes(full=False)

Method that iterates over all available recipes and prints their information to the standard output

Parameters:

full : bool

If true, it will provide the pipeline string along with the recipe name