5i ldZddlmZmZmZmZddlZddlZddlm Z ddl m Z m Z m Z ddlmZddlmZd d ZGd d e e Zy) z DXWorkflow Handler ++++++++++++++++++ Workflows are data objects which contain metadata for a set of jobs to be run together. They can be run by calling the :func:`DXWorkflow.run` method. )print_functionunicode_literalsdivisionabsolute_importN)SystemRequirementsDict) DXDataObject DXExecutable DXAnalysis)DXError) basestringc Jt}|jd|||||d||S)am :param title: Workflow title (optional) :type title: string :param summary: Workflow summary (optional) :type summary: string :param description: Workflow description (optional) :type description: string :param output_folder: Default output folder of the workflow (optional) :type output_folder: string :param init_from: Another analysis workflow object handler or and analysis (string or handler) from which to initialize the metadata (optional) :type init_from: :class:`~dxpy.bindings.dxworkflow.DXWorkflow`, :class:`~dxpy.bindings.dxanalysis.DXAnalysis`, or string (for analysis IDs only) :rtype: :class:`DXWorkflow` Additional optional parameters not listed: all those under :func:`dxpy.bindings.DXDataObject.new`, except `details`. Creates a new remote workflow object with project set to *project* and returns the appropriate handler. Example: r = dxpy.new_dxworkflow(title="My Workflow", description="This workflow contains...") Note that this function is shorthand for:: dxworkflow = DXWorkflow() dxworkflow.new(**kwargs) )titlesummary description output_folder init_from) DXWorkflownew)rrrrrkwargs dxworkflows z/home/marpiech/ifpan-abm-pgxpred/analysis/marpiech-gwas-test/venv/lib/python3.12/site-packages/dxpy/bindings/dxworkflow.pynew_dxworkflowr(s?:JJNNF[XeqzFEF c<eZdZdZdZeejjZ eejjZ eejjZ eejjZeejj Zeejj$Zeejj(Zeejj,Zeejj0Zeejj4Zeejj8Zeejj<ZdZ ddZ!dZ" ddZ#dZ$ddZ%dd Z& dd Z' dd Z(d Z)dd Z*dZ+dZ,dZ-fdZ.xZ/S)rz Remote workflow object handler. This class is used for the workflow class data objects which produce an analysis when run. workflowc dd}d|vr|dt|dttfsEt|dtr't j dj |ds tdt|dtr d|di|d<nDd|dji|d<t|dtr|dj|dd<|d=|||d |||d |||d |||d d |||d|||dd|||ddtjj|fi|}|j|d|dy)a :param dx_hash: Standard hash populated in :func:`dxpy.bindings.DXDataObject.new()` containing attributes common to all data object classes. :type dx_hash: dict :param title: Workflow title (optional) :type title: string :param summary: Workflow summary (optional) :type summary: string :param description: Workflow description (optional) :type description: string :param output_folder: Default output folder of the workflow (optional) :type output_folder: string :param stages: Stages of the workflow (optional) :type stages: array of dictionaries :param workflow_inputs: Workflow-level input specification (optional) :type workflow_inputs: array of dictionaries :param workflow_outputs: Workflow-level output specification (optional) :type workflow_outputs: array of dictionaries :param init_from: Another analysis workflow object handler or and analysis (string or handler) from which to initialize the metadata (optional) :type init_from: :class:`~dxpy.bindings.dxworkflow.DXWorkflow`, :class:`~dxpy.bindings.dxanalysis.DXAnalysis`, or string (for analysis IDs only) Create a new remote workflow object. Nc:||n|}||vr||||||<||=yyNr)rdxhashkeynew_keys r _set_dx_hashz%DXWorkflow._new.._set_dx_hashvs7$_c'Gf}#;*&,SkF7O3Krrz^analysis-[0-9A-Za-z]{24}$z`Expected init_from to be an instance of DXWorkflow or DXAnalysis, or to be a string analysis ID.idinitializeFromprojectrrrr outputFolderstagesworkflow_inputsinputsworkflow_outputsoutputsr ) isinstancerr r recompilematchr get_id get_proj_iddxpyapi workflow_newset_ids)selfdx_hashrr$resps r_newzDXWorkflow._new^st0  & k"."6+#6Z8PQ#F;$7D$@AGG{H[\!#EFFf[1:>15vk7J0KG,-15vk7J7Q7Q7S0TG,-!&"5zB?Ek?R?^?^?` 01)<{#VWg.VWi0VWm4VWo~FVWh/VW&7BVW&8)Dxx$$W77 T$Z!34rc2||j|d<y||d<y)N editVersion)r=)r8 request_hash edit_versions r_add_edit_version_to_requestz'DXWorkflow._add_edit_version_to_requests"  *.*:*:L '*6L 'rc4t|tr t|}t|tst t|}|dks|t |j k\r-tdt t |j zdz|j |jdStjdj|?t|j Dcgc]}|jd|k(s|dc}}|r|S|j Dcgc]}|jd|k(s|d}}t |dk(rtd|zd zt |d kDrtd |zd z|dS#Y`xYw#tdxYwcc}wcc}w) a :param stage: A stage ID, name, or index (stage index is the number n for the nth stage, starting from 0; can be provided as an int or a string) :type stage: int or string :returns: The stage ID (this is a no-op if it was already a stage ID) :raises: :class:`~dxpy.exceptions.DXError` if *stage* could not be parsed, resolved to a stage ID, or it could not be found in the workflow zYDXWorkflow: the given stage identifier was neither a string stage ID nor an integer indexrz"DXWorkflow: the workflow contains zP stage(s), and the numerical value of the given stage identifier is out of ranger%z!^([a-zA-Z_]|stage-)[0-9a-zA-Z_]*$namez'DXWorkflow: the given stage identifier z5 could not be found as a stage ID nor as a stage namezEDXWorkflow: more than one workflow stage was found to have the name "") r.r intr lenr)strgetr/r0r1any)r8stage stage_indexstgstage_id_existsstage_ids_matching_names r _get_stage_idzDXWorkflow._get_stage_ids eZ ( E  %, {!%j Q+T[[1A"ABST[[IYEZZpqrr;;{+//5 5 ::9 : @ @ G S! "^swwt}X]G]3t9"^_O 9= "`swwvZ_G_3t9"`"` & '1 ,CeKOFFG G ( )A -adiiloop p*1- -;  {yzz#_ #as. E8 FF2FF,F8E= F c t|tr|} n,t|tr|j} n t dd| i} ||| d<||| d<||| d<||| d<|&t j |j| d<|j| | tjj|j| fi|} |j| dS#|jwxYw) a :param executable: string or a handler for an app or applet :type executable: string, DXApplet, or DXApp :param stage_id: id for the stage (optional) :type stage_id: string :param name: name for the stage (optional) :type name: string :param folder: default output folder for the stage; either a relative or absolute path (optional) :type folder: string :param stage_input: input fields to bind as default inputs for the executable (optional) :type stage_input: dict :param instance_type: Default instance type on which all jobs will be run for this stage, or a dict mapping function names to instance type requests :type instance_type: string or dict :param edit_version: if provided, the edit version of the workflow that should be modified; if not provided, the current edit version will be used (optional) :type edit_version: int :returns: ID of the added stage :rtype: string :raises: :class:`~dxpy.exceptions.DXError` if *executable* is not an expected type :class:`~dxpy.exceptions.DXAPIError` for errors thrown from the API call Adds the specified executable as a new stage in the workflow. zZdxpy.DXWorkflow.add_stage: executable must be a string or an instance of DXApplet or DXApp executabler%rBfolderinputsystemRequirementsrJ)r.r r r2r rfrom_instance_typeas_dictr@r4r5workflow_add_stage_dxiddescribe) r8rQstage_idrBrR stage_input instance_typer?rexec_idadd_stage_inputresults r add_stagezDXWorkflow.add_stages. j* - G  L 1 '')Gvw w'1  $,OD !  &*OF #  (.OH %  "'2OG $  $4J4]4]^k4l4t4t4vO0 1 ))/<H XX00_WPVWF MMOg MMOs +CC-c |j|tfd|jDd}|tdzdz|S)z :param stage: A number for the stage index (for the nth stage, starting from 0), or a string of the stage index, name, or ID :type stage: int or string :returns: Hash of stage descriptor in workflow c34K|]}|dk(s |yw)r%Nr).0rJrZs r z'DXWorkflow.get_stage..sQt9PuQs Nz The stage ID z could not be found)rOnextr)r )r8rJrr_rZs @r get_stagezDXWorkflow.get_stagesL %%e,Q$++QSWX >/H47LLM M rc |j|}d|i}|j|| tjj|j |fi||j |S#|j wxYw)a :param stage: A number for the stage index (for the nth stage, starting from 0), or a string of the stage index, name, or ID :type stage: int or string :param edit_version: if provided, the edit version of the workflow that should be modified; if not provided, the current edit version will be used (optional) :type edit_version: int :returns: Stage ID that was removed :rtype: string Removes the specified stage from the workflow rJ)rOr@r4r5workflow_remove_stagerXrY)r8rJr?rrZremove_stage_inputs r remove_stagezDXWorkflow.remove_stagesn%%e,%x0 ))*Fc||jjdduxr|jjddk(S)Nr+stateclosed)_descrH)r8s r is_lockedzDXWorkflow.is_lockeds2zz~~h't3[ w8OS[8[[rcfd|vr,|jdd\}}|j|dz|zS|S)a :param input_str: A string of one of the forms: "", "", ".", ".", "." :type input_str: string :returns: If the given form was one of those which uses the stage index or stage name, it is translated to the stage ID for use in the API call (stage name takes precedence) .rC)splitrO)r8 input_strregiondescribe_outputstage_identifier input_names r_get_input_namezDXWorkflow._get_input_namesD ) +4??3+B ( j%%&67#= J Jrcti}|D]0}|j|}||vrtd|zdz||||<2|S)NzDXWorkflow: the input for z was provided more than once)rr )r8workflow_inputeffective_inputr"rs r_get_effective_inputzDXWorkflow._get_effective_inputs\! >C--c2J_,:ZGJhhii*8*=OJ '  > rc |j|}tj|fi|}|jd_i|d<|dj D]D\}}|dk7r|j |}t j|j|d|<F|jd>i|d<|dj D]#\}}|dk7r|j |}||d|<%|jd,|dDcgc]}|dk(r|n|j |c}|d<|jdd rdg|d <|jd ,|d Dcgc]}|dk(r|n|j |c}|d <|Scc}wcc}w) Nstage_instance_typesstageSystemRequirements* stage_folders stageFolders rerun_stages rerunStages ignore_reuseF ignoreReuseignore_reuse_stages) rr _get_run_input_common_fieldsrHitemsrOrrUrV)r8rreffective_workflow_input run_inputrJvalue_stages r_get_run_inputzDXWorkflow._get_run_inputs#'#<#<^#L  ==>VaZ`a ::, - 935I/ 0 &'= > D D F y uC< ..u5E>T>g>ghm>n>v>v>x 34U; y ::o & 2(*In % & 7 = = ? 9 uC< ..u5E38 .)%0 9 ::n % 1%^4(!C-T-?-?-GG(Im $ ::ne ,(+uIm $ ::+ , 8%%:;(!C-T-?-?-GG(Im $ ((s ?E;Fc pttjj|j|fi|dS)Nr%)r r4r5 workflow_runrX)r8rrs r _run_implzDXWorkflow._run_impls-$((// IPPQUVWWrc2tt| |g|i|S)a8 :param workflow_input: Dictionary of the workflow's input arguments; see below for more details :type workflow_input: dict :param instance_type: Instance type on which all stages' jobs will be run, or a dict mapping function names to instance types. These may be overridden on a per-stage basis if stage_instance_types is specified. :type instance_type: string or dict :param stage_instance_types: A dict mapping stage IDs, names, or indices to either a string (representing an instance type to be used for all functions in that stage), or a dict mapping function names to instance types. :type stage_instance_types: dict :param stage_folders: A dict mapping stage IDs, names, indices, and/or the string "*" to folder values to be used for the stages' output folders (use "*" as the default for all unnamed stages) :type stage_folders: dict :param rerun_stages: A list of stage IDs, names, indices, and/or the string "*" to indicate which stages should be run even if there are cached executions available :type rerun_stages: list of strings :param ignore_reuse_stages: Stages of a workflow (IDs, names, or indices) or "*" for which job reuse should be disabled :type ignore_reuse_stages: list :returns: Object handler of the newly created analysis :rtype: :class:`~dxpy.bindings.dxanalysis.DXAnalysis` Run the associated workflow. See :meth:`dxpy.bindings.dxapplet.DXExecutable.run` for additional args. When providing input for the workflow, keys should be of one of the following forms: * "N.name" where *N* is the stage number, and *name* is the name of the input, e.g. "0.reads" if the first stage takes in an input called "reads" * "stagename.name" where *stagename* is the stage name, and *name* is the name of the input within the stage * "stageID.name" where *stageID* is the stage ID, and *name* is the name of the input within the stage * "name" where *name* is the name of a workflow level input (defined in inputs) or the name that has been exported for the workflow (this name will appear as a key in the "inputSpec" of this workflow's description if it has been exported for this purpose) )superrrun)r8rargsr __class__s rrzDXWorkflow.runs"LZ*>KDKFKKrr )NNNNNN) NFNNNFNFNFNN) NFNFNFNNN)NN)0__name__ __module__ __qualname____doc___class staticmethodr4r5workflow_describe _describeworkflow_add_types _add_typesworkflow_remove_types _remove_typesworkflow_get_details _get_detailsworkflow_set_details _set_detailsworkflow_set_visibility_set_visibilityworkflow_rename_renameworkflow_set_properties_set_propertiesworkflow_add_tags _add_tagsworkflow_remove_tags _remove_tagsworkflow_close_closeworkflow_list_projects_list_projectsr;r@rOr`rfrjrprxrrrrrrr __classcell__)rs@rrrIs FTXX778Idhh99:J !?!?@M = =>L = =>L"488#C#CDO488334G"488#C#CDOTXX778I = =>L $((11 2F!$(("A"ABN65p7 (.Tlp#-^ ((OS7<;@=B)- D L:?_c6:F P\ "HX&L&Lrr)NNNNN)r __future__rrrrr/r4system_requirementsrbindingsr r r exceptionsr compatr rrrrrrs@"SR 8==  BTL|TLr