5iSZdZddlmZmZmZmZddlZddlmZm Z ddl m Z ddl m Z dd lmZdd lmZdd lmZGd d ZgdZddgZddgZgdZdZGddeeZy)z DXApplet Handler ++++++++++++++++ Applets are data objects that store application logic, including specifications for executing it, and (optionally) input and output signatures. They can be run by calling the :func:`DXApplet.run` method. )print_functionunicode_literalsdivisionabsolute_importN) DXDataObjectDXJob)merge) is_project_id)SystemRequirementsDictDXError) basestringcneZdZdZdZedZedZdZdZ dZ dZ d Z d Z d d Zy ) DXExecutablezMethods in :class:`!DXExecutable` are used by :class:`~dxpy.bindings.dxapp.DXApp`, :class:`~dxpy.bindings.dxapplet.DXApplet`, :class:`~dxpy.bindings.dxworkflow.DXWorkflow`, and :class:`~dxpy.bindings.dxworkflow.DXGlobalWorkflow` ctd)Nz6This class is a mix-in. Use DXApp or DXApplet instead.NotImplementedError)selfargskwargss x/home/marpiech/ifpan-abm-pgxpred/analysis/marpiech-gwas-test/venv/lib/python3.12/site-packages/dxpy/bindings/dxapplet.py__init__zDXExecutable.__init__,s!"Z[[c jdxstj}d|i}dD]}j||||<tfddDrt j jd}t jd}t jd}t jd }||z|z|zj |d <jd jd |d <jd jd |d <jdg|d<tdtrdD]} t| tst| tr>| j td|dj| jat| tr|dj| tdn tdjdd|d<jdd|d<jdd|d<jdd|d<jdd|d<tj jddur|t!|r||d<jdt#|djdd|d<jdd|d<jd d |d <jd!d!|d"<jd#d#|d$<jd%d%|d&<jd'} | | d(k7r | dk(rin| |d)<|S)*ae Takes the same arguments as the run method. Creates an input hash for the /executable-xxxx/run method, translating ONLY the fields that can be handled uniformly across all executables: project, folder, name, tags, properties, details, depends_on, allow_ssh, debug, delay_workspace_destruction, ignore_reuse, and extra_args. projectinput)foldernametags propertiesdetailsc3DK|]}j|duywN)get).0keyrs r z.=sxsvzz#d*xs ) instance_type cluster_spec fpga_driver nvidia_driverr*r+r,r-systemRequirementssystem_requirements!system_requirements_by_executablesystemRequirementsByExecutable depends_on dependsOnz:A dxpy handler given in depends_on does not have an ID setz`Expected elements of depends_on to only be either instances of DXJob or DXDataObject, or stringsz&Expected depends_on field to be a listdelay_workspace_destructiondelayWorkspaceDestruction allow_sshallowSSHdebugpriority ignore_reuse ignoreReusedetachT extra_args cost_limit costLimitrankmax_tree_spot_wait_timemaxTreeSpotWaitTimemax_job_spot_wait_timemaxJobSpotWaitTimedetailed_job_metricsdetailedJobMetricspreserve_job_outputsFpreserveJobOutputs)r&dxpy WORKSPACE_IDanyr from_instance_typeas_dict isinstancelistr rget_idrappendrJOB_IDr r ) executable_inputrr run_inputarginstance_type_srdcluster_spec_srdfpga_driver_srdnvidia_driver_srditemrGs ` r_get_run_input_common_fieldsz)DXExecutable._get_run_input_common_fields/s**Y'<4+<+<./ F -Czz#*!' # - x6wx x 6 I I&**UdJe f 5fjj6PQ 4VZZ 5NOO 6vzz/7R S /@CS/SVe/ehy/y/C/C/EI* + ::+ , 8.4jj9N.OI* + ::9 : F:@**Eh:iI6 7 ::l # /%'Ik "&.5"<0JD!$.*T<2P;;=0")*f"gg!+.55dkkmD#D*5!+.55d;%'IJJJFGG ::3 4 @5;NYRXY ::* + 7+12F+GI' (rc td)a) Runs the executable with the specified input and returns a handler for the resulting execution object (:class:`~dxpy.bindings.dxjob.DXJob` or :class:`~dxpy.bindings.dxanalysis.DXAnalysis`). Any kwargs are passed on to :func:`~dxpy.DXHTTPRequest`. z_run_impl is not implementedrrrTrs r _run_implzDXExecutable._run_impls""@AArc td)z Takes the same arguments as the run method. Creates an input hash for the /executable-xxxx/run method. z!_get_run_input is not implementedrrrSrs r_get_run_inputzDXExecutable._get_run_inputs ""EFFrctd)N Abstract method used in executable_unbuilder.dump_executable z%_get_required_keys is not implementedrrs r_get_required_keyszDXExecutable._get_required_keys""IJJrctd)rlz%_get_optional_keys is not implementedrrms r_get_optional_keyszDXExecutable._get_optional_keysrorctd)rlz,_get_describe_output_keys is not implementedrrms r_get_describe_output_keysz&DXExecutable._get_describe_output_keyss""PQQrctd)rlz$_get_cleanup_keys is not implementedrrms r_get_cleanup_keyszDXExecutable._get_cleanup_keyss""HIIrNc! |j|fid|d|d|d|d|d|d|d| d | d | d | d | d |d|d|d|d|d|d|d|d|d|d|d|d|d|d|d|d|d|d| }"|j|"fi|!S) a :param executable_input: Hash of the executable's input arguments :type executable_input: dict :param project: Project ID of the project context :type project: string :param folder: Folder in which executable's outputs will be placed in *project* :type folder: string :param name: Name for the new job (default is "") :type name: string :param tags: Tags to associate with the job :type tags: list of strings :param properties: Properties to associate with the job :type properties: dict with string values :param details: Details to set for the job :type details: dict or list :param instance_type: Instance type on which the jobs will be run, or a dict mapping function names to instance type requests :type instance_type: string or dict :param depends_on: List of data objects or jobs to wait that need to enter the "closed" or "done" states, respectively, before the new job will be run; each element in the list can either be a dxpy handler or a string ID :type depends_on: list :param allow_ssh: List of hostname or IP masks to allow SSH connections from :type allow_ssh: list :param debug: Configuration options for job debugging :type debug: dict :param delay_workspace_destruction: Whether to keep the job's temporary workspace around for debugging purposes for 3 days after it succeeds or fails :type delay_workspace_destruction: boolean :param priority: Priority level to request for all jobs created in the execution tree, "low", "normal", or "high" :type priority: string :param head_job_on_demand: If true, the job will be run on a demand instance. :type head_job_on_demand: bool :param ignore_reuse: Disable job reuse for this execution :type ignore_reuse: boolean :param ignore_reuse_stages: Stages of a workflow (IDs, names, or indices) or "*" for which job reuse should be disabled :type ignore_reuse_stages: list :param detach: If provided, job will not start as subjob if run inside of a different job. :type detach: boolean :param cost_limit: Maximum cost of the job before termination. :type cost_limit: float :param rank: Rank of execution :type rank: int :param max_tree_spot_wait_time: Number of seconds allocated to each path in the root execution's tree to wait for Spot :type max_tree_spot_wait_time: int :param max_job_spot_wait_time: Number of seconds allocated to each job in the root execution's tree to wait for Spot :type max_job_spot_wait_time: int :param preserve_job_outputs: Copy cloneable outputs of every non-reused job entering "done" state in this root execution to a folder in the project. If value is True it will place job outputs into the "intermediateJobOutputs" subfolder under the output folder for the root execution. If the value is dict, it may contains "folder" key with desired folder path. If the folder path starts with '/' it refers to an absolute path within the project, otherwise, it refers to a subfolder under root execution's output folder. :type preserve_job_outputs: boolean or dict :param detailed_job_metrics: Enable detailed job metrics for this root execution :type detailed_job_metrics: boolean :param extra_args: If provided, a hash of options that will be merged into the underlying JSON given for the API call :type extra_args: dict :returns: Object handler of the newly created job :param fpga_driver: a dict mapping function names to fpga driver requests :type fpga_driver: dict :param system_requirements: System requirement single mapping :type system_requirements: dict :param system_requirements_by_executable: System requirement by executable double mapping :type system_requirements_by_executable: dict :param nvidia_driver: a dict mapping function names to nvidia driver requests :type nvidia_driver: dict :rtype: :class:`~dxpy.bindings.dxjob.DXJob` Creates a new job that executes the function "main" of this executable with the given input *executable_input*. rrr r!r"r#r*r]r^r_r+r2r6r:r`r8r4r9rar<r>r@rArCrGrEr=r,r/r0r-)rjrg)#rrSrrr r!r"r#r*r]r^r_r+r2r6r8r4r9rar:r`r<r>r@rArCrGrEr=r,r/r0r-rrTs# rrunzDXExecutable.runsT(D''(8E07E/5E.2E.2 E 4> E 18 E7DE>RE7DE6BE6BE4>E3<E6BE=PE /4!E"E`#E$2:%E&>P'E(06)E*4>+E,.2-E.AX/E0@V1E2>R3E4>R5E64>7E85@9E:=P;E<Kl=E>7D?E @t~~i2622r)NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN)__name__ __module__ __qualname____doc__r staticmethodr[rdrgrjrnrqrsrurwrrrr%s| \TTl" BGK K R J osos~Bz~jnnr j3rr) r titlesummarytypesr!httpsAppr"dxapi inputSpec outputSpecrunSpecaccessr#r;treeTurnaroundTimeThresholdr"r#) r r~rrr!r"rrr#cdfd }|S)Nc&tdzdz)Nz Wrapper for z does not existr) object_id input_params always_retryrmethods rnonexistentAPIWrapperz9_makeNonexistentAPIWrapper..nonexistentAPIWrapper;snv-0AABBr)NNr})rrs` r_makeNonexistentAPIWrapperr:sC  rcjeZdZdZdZeejjZ ee dZ ee dZ eejjZee dZee dZeejj"Zeejj&Zeejj*Zeejj.Zee dZeejj4ZdZd Zd Zd Zd Z d Z!dZ"dZ#fdZ$xZ%S)DXAppleta Remote applet object handler. .. py:attribute:: runSpec The applet's run specification (a dict indicating, among other things, how the code of the applet is to be interpreted). See `the API docs for Run Specification `_ for more information. .. py:attribute:: dxapi String containing the version of the DNAnexus API that the applet should run against. .. py:attribute:: access The applet's access requirements hash (a dict indicating any nonstandard permissions, such as requiring access to the internet, that are needed by the applet). See `the API docs for Access Requirements `_ for more information. .. py:attribute:: title String containing the (human-readable) title of the app .. py:attribute:: summary String containing a short, one-line summary of the applet's purpose .. py:attribute:: description String of free-form text (`Markdown `_ syntax is supported) containing a description of the applet. The description is presented to users to help them understand the purpose of the app and how to invoke it. .. py:attribute:: developerNotes String of free-form text (`Markdown `_ syntax is supported) containing information about the internals or implementation details of the applet, suitable for developers or advanced users. .. automethod:: _new appletz/applet-xxxx/addTypesz/applet-xxxx/removeTypesz/applet-xxxx/setDetailsz/applet-xxxx/setVisibilityz/applet-xxxx/closec dD]6}||vr%t|jjd|d||||<||=8dD]}||vs||||<||=tjj |fi|}|j |d|dy)a :param dx_hash: Standard hash populated in :func:`dxpy.bindings.DXDataObject.new()` containing attributes common to all data object classes. :type dx_hash: dict :param runSpec: Run specification :type runSpec: dict :param dxapi: API version string :type dxapi: string :param inputSpec: Input specification (optional) :type inputSpec: dict :param outputSpec: Output specification (optional) :type outputSpec: dict :param access: Access specification (optional) :type access: dict :param title: Title string (optional) :type title: string :param summary: Summary string (optional) :type summary: string :param description: Description string (optional) :type description: string .. note:: It is highly recommended that the higher-level module :mod:`dxpy.app_builder` or (preferably) its frontend `dx build `_ be used instead for applet creation. Creates an applet with the given parameters. See the API documentation for the `/applet/new `_ method for more info. The applet is not run until :meth:`run()` is called. )rrz: Keyword argument z is required)rrrr~r descriptionidrN)r __class__rxrIapi applet_newset_ids)rdx_hashrfieldresps r_newz DXApplet._new|sB( EF"t~~G^G^`efgg#E]GENu   \ "E!'5M " xx""75f5 T$Z!34rc Vtjj|jfi|S)a :returns: Full specification of the remote applet object :rtype: dict Returns the contents of the applet. The result includes the key-value pairs as specified in the API documentation for the `/applet-xxxx/get `_ method. )rIr applet_get_dxid)rrs rr&z DXApplet.gets"xx""4::888rc pttjj|j|fi|dS)Nr)r rIr applet_runrrfs rrgzDXApplet._run_impls,TXX((YI&I$OPPrc .tj|fi|Sr%)rrdris rrjzDXApplet._get_run_inputs<<=MXQWXXrctSr%)_applet_required_keysrms rrnzDXApplet._get_required_keys$$rctSr%)_applet_optional_keysrms rrqzDXApplet._get_optional_keysrrctSr%)_applet_describe_output_keysrms rrsz"DXApplet._get_describe_output_keyss++rctSr%)_applet_cleanup_keysrms rruzDXApplet._get_cleanup_keyss##rc2tt| |g|i|S)z Creates a new job that executes the function "main" of this applet with the given input *applet_input*. See :meth:`dxpy.bindings.dxapplet.DXExecutable.run` for the available args. )superrrw)r applet_inputrrrs rrwz DXApplet.runs!Xt(GGGGr)&rxryrzr{_classr|rIrapplet_describe _describer _add_types _remove_typesapplet_get_details _get_details _set_details_set_visibility applet_rename_renameapplet_set_properties_set_propertiesapplet_add_tags _add_tagsapplet_remove_tags _remove_tags_closeapplet_list_projects_list_projectsrr&rgrjrnrqrsrurw __classcell__)rs@rrr?s(+ZFTXX556I89PQRJ !;Z#[\O488112G"488#A#ABOTXX556I ; ;rsv"SR !*8 G3G3Z9'(EF ,i8F! TH|\THr