flowcraft.generator.recipe module¶
-
class
flowcraft.generator.recipe.
InnuendoRecipe
[source]¶ Bases:
object
Methods
build_downstream
(process_descriptions, task, …)Builds the downstream pipeline of the current process build_pipeline_string
(forks)Parses, filters and merge all possible pipeline forks into the final pipeline string build_upstream
(process_descriptions, task, …)Builds the upstream pipeline of the current process define_pipeline_string
(process_descriptions, …)Builds the possible forks and connections between the provided processes run_auto_pipeline
(tasks)Main method to run the automatic pipeline creation validate_pipeline
(pipeline_string)Validate pipeline string -
count_forks
= None¶ int : counts the total possible number of forks
-
forks
= None¶ list : a list with all the possible forks
-
pipeline_string
= None¶ str : the generated pipeline string
-
process_to_id
= None¶ dict: key value between the process name and its identifier
-
static
validate_pipeline
(pipeline_string)[source]¶ Validate pipeline string
Validates the pipeline string by searching for forbidden characters
Parameters: - pipeline_string : str
STring with the processes provided
-
build_upstream
(process_descriptions, task, all_tasks, task_pipeline, count_forks, total_tasks, forks)[source]¶ Builds the upstream pipeline of the current process
Checks for the upstream processes to the current process and adds them to the current pipeline fragment if they were provided in the process list.
Parameters: - process_descriptions : dict
Information of processes input, output and if is forkable
- task : str
Current process
- all_tasks : list
A list of all provided processes
- task_pipeline : list
Current pipeline fragment
- count_forks : int
Current number of forks
- total_tasks : str
All space separated processes
- forks : list
Current forks
- Returns
- ——-
- list : resulting pipeline fragment
-
build_downstream
(process_descriptions, task, all_tasks, task_pipeline, count_forks, total_tasks, forks)[source]¶ Builds the downstream pipeline of the current process
Checks for the downstream processes to the current process and adds them to the current pipeline fragment.
Parameters: - process_descriptions : dict
Information of processes input, output and if is forkable
- task : str
Current process
- all_tasks : list
A list of all provided processes
- task_pipeline : list
Current pipeline fragment
- count_forks : int
Current number of forks
- total_tasks : str
All space separated processes
- forks : list
Current forks
- Returns
- ——-
- list : resulting pipeline fragment
-
define_pipeline_string
(process_descriptions, tasks, check_upstream, check_downstream, count_forks, total_tasks, forks)[source]¶ Builds the possible forks and connections between the provided processes
This method loops through all the provided tasks and builds the upstream and downstream pipeline if required. It then returns all possible forks than need to be merged à posteriori`
Parameters: - process_descriptions : dict
Information of processes input, output and if is forkable
- tasks : str
Space separated processes
- check_upstream : bool
If is to build the upstream pipeline of the current task
- check_downstream : bool
If is to build the downstream pipeline of the current task
- count_forks : int
Number of current forks
- total_tasks : str
All space separated processes
- forks : list
Current forks
Returns: - list : List with all the possible pipeline forks
-
build_pipeline_string
(forks)[source]¶ Parses, filters and merge all possible pipeline forks into the final pipeline string
This method checks for shared start and end sections between forks and merges them according to the shared processes:
[[spades, ...], [skesa, ...], [...,[spades, skesa]]] -> [..., [[spades, ...], [skesa, ...]]]
Then it defines the pipeline string by replacing the arrays levels to the flowcraft fork format:
[..., [[spades, ...], [skesa, ...]]] -> ( ... ( spades ... | skesa ... ) )
Parameters: - forks : list
List with all the possible pipeline forks.
Returns: - str : String with the pipeline definition used as input for
- parse_pipeline
-
run_auto_pipeline
(tasks)[source]¶ Main method to run the automatic pipeline creation
This method aggregates the functions required to build the pipeline string that can be used as input for the workflow generator.
Parameters: - tasks : str
A string with the space separated tasks to be included in the pipeline
Returns: - str : String with the pipeline definition used as input for
- parse_pipeline
-
-
class
flowcraft.generator.recipe.
Innuendo
(*args, **kwargs)[source]¶ Bases:
flowcraft.generator.recipe.InnuendoRecipe
Recipe class for the INNUENDO Project. It has all the available in the platform for quick use of the processes in the scope of the project.
Methods
build_downstream
(process_descriptions, task, …)Builds the downstream pipeline of the current process build_pipeline_string
(forks)Parses, filters and merge all possible pipeline forks into the final pipeline string build_upstream
(process_descriptions, task, …)Builds the upstream pipeline of the current process define_pipeline_string
(process_descriptions, …)Builds the possible forks and connections between the provided processes run_auto_pipeline
(tasks)Main method to run the automatic pipeline creation validate_pipeline
(pipeline_string)Validate pipeline string
-
flowcraft.generator.recipe.
brew_innuendo
(args)[source]¶ Brews a given list of processes according to the recipe
Parameters: - args : argparse.Namespace
The arguments passed through argparser that will be used to check the the recipe, tasks and brew the process
Returns: - str
The final pipeline string, ready for the engine.
- list
List of process strings.
-
class
flowcraft.generator.recipe.
Recipe
[source]¶ Bases:
object
Methods
brew -
pipeline_str
= None¶ str: The raw pipeline string, with no attribute or directives, except for number indicators for when there are duplicate components.
e.g.: “fastqc trimmomatic spades” e.g.: “fastqc trimmomatic (spades#1 | spades#2)
-
directives
= None¶ dict: Dictionary with the parameters and directives for each component in the pipeline_str attribute. Missing components will be left with the default parameters and directives.
-
-
flowcraft.generator.recipe.
brew_recipe
(recipe_name)[source]¶ Returns a pipeline string from a recipe name.
Parameters: - recipe_name : str
Name of the recipe. Must match the name attribute in one of the classes defined in
flowcraft.generator.recipes
Returns: - str
Pipeline string ready for parsing and processing by flowcraft engine