5i6DdZddlmZmZmZmZddlZddlZddlZddl m Z m Z m Z m Z ddlmZddlmZdd lmZdd lmZ dd ZGd d e Zy)af DXJob Handler +++++++++++++ Jobs are DNAnexus entities that capture an instantiation of a running app or applet. They can be created from either :func:`dxpy.bindings.dxapplet.DXApplet.run` or :func:`dxpy.bindings.dxapp.DXApp.run` if running an applet or app, or via :func:`new_dxjob` or :func:`DXJob.new` in the case of an existing job creating a subjob. )print_functionunicode_literalsdivisionabsolute_importN)DXObject DXDataObjectDXJobFailureErrorverify_string_dxid)DXError)SystemRequirementsDict)queue_entry_point) basestringc  Zt}|j||f|||||||| | | | d | |S)a :param fn_input: Function input :type fn_input: dict :param fn_name: Name of the function to be called :type fn_name: string :param name: Name for the new job (default is ":") :type name: string :param tags: Tags to associate with the job :type tags: list of strings :param properties: Properties to associate with the job :type properties: dict with string values :param details: Details to set for the job :type details: dict or list :param instance_type: Instance type on which the job will be run, or a dict mapping function names to instance type requests :type instance_type: string or dict :param depends_on: List of data objects or jobs to wait that need to enter the "closed" or "done" states, respectively, before the new job will be run; each element in the list can either be a dxpy handler or a string ID :type depends_on: list :param cluster_spec: a dict mapping function names to cluster spec requests :type cluster_spec: dict :param fpga_driver: a dict mapping function names to fpga driver requests :type fpga_driver: dict :param system_requirements: System requirement single mapping :type system_requirements: dict :param system_requirements_by_executable: System requirement by executable double mapping :type system_requirements_by_executable: dict :param nvidia_driver: a dict mapping function names to nvidia driver requests :type nvidia_driver: dict :rtype: :class:`~dxpy.bindings.dxjob.DXJob` Creates and enqueues a new job that will execute a particular function (from the same app or applet as the one the current job is running). Returns the :class:`~dxpy.bindings.dxjob.DXJob` handle for the job. Note that this function is shorthand for:: dxjob = DXJob() dxjob.new(fn_input, fn_name, **kwargs) .. note:: This method is intended for calls made from within already-executing jobs or apps. If it is called from outside of an Execution Environment, an exception will be thrown. To create new jobs from outside the Execution Environment, use :func:`dxpy.bindings.dxapplet.DXApplet.run` or :func:`dxpy.bindings.dxapp.DXApp.run`. .. note:: If the environment variable ``DX_JOB_ID`` is not set, this method assmes that it is running within the debug harness, executes the job in place, and provides a debug job handler object that does not have a corresponding remote API job object. ) nametags propertiesdetails instance_type depends_on cluster_spec fpga_driversystem_requirements!system_requirements_by_executable nvidia_driver)DXJobnew)fn_inputfn_namerrrrrrrrrrrkwargsdxjobs u/home/marpiech/ifpan-abm-pgxpred/analysis/marpiech-gwas-test/venv/lib/python3.12/site-packages/dxpy/bindings/dxjob.py new_dxjobr$/sPh GE EIIh5d*V])j|it"5Yz)5.45 LcleZdZdZdZddZ ddZdZddZdZ d Z d Z d Z dd Z d ZddZdZy)rz$ Remote job object handler. jobNcbd|_tj|||j|y)N)dxid)_test_harness_resultr__init__set_idselfr)s r#r+zDXJob.__init__rs&$(!$T* Dr%c vg}|t|tr|D]}t|tst|tr;|j t d|j |j ^t|tr|j |t dn t ddtjvri}||d<||d<|||d<|||d <|||d <td || | | fDrStj||}t| }t| }t| }||z|z|zj|d <| | |d <| | |d <|||d<|||d<tjj |fi|}|j#|dy|j#t%||||y)a :param fn_input: Function input :type fn_input: dict :param fn_name: Name of the function to be called :type fn_name: string :param name: Name for the new job (default is ":") :type name: string :param tags: Tags to associate with the job :type tags: list of strings :param properties: Properties to associate with the job :type properties: dict with string values :param details: Details to set for the job :type details: dict or list :param instance_type: Instance type on which the job will be run, or a dict mapping function names to instance type requests :type instance_type: string or dict :param depends_on: List of data objects or jobs to wait that need to enter the "closed" or "done" states, respectively, before the new job will be run; each element in the list can either be a dxpy handler or a string ID :type depends_on: list :param cluster_spec: a dict mapping function names to cluster spec requests :type cluster_spec: dict :param fpga_driver: a dict mapping function names to fpga driver requests :type fpga_driver: dict :param system_requirements: System requirement single mapping :type system_requirements: dict :param system_requirements_by_executable: System requirement by executable double mapping :type system_requirements_by_executable: dict :param nvidia_driver: a dict mapping function names to nvidia driver requests :type nvidia_driver: dict Creates and enqueues a new job that will execute a particular function (from the same app or applet as the one the current job is running). .. note:: This method is intended for calls made from within already-executing jobs or apps. If it is called from outside of an Execution Environment, an exception will be thrown. To create new jobs from outside the Execution Environment, use :func:`dxpy.bindings.dxapplet.DXApplet.run` or :func:`dxpy.bindings.dxapp.DXApp.run`. Nz:A dxpy handler given in depends_on does not have an ID setz`Expected elements of depends_on to only be either instances of DXJob or DXDataObject, or stringsz&Expected depends_on field to be a list DX_JOB_IDinputfunctionrrrc3$K|]}|du ywN).0 requirements r# zDXJob.new..sx{;d*xssystemRequirementssystemRequirementsByExecutable dependsOnrid)r2 input_hashrr) isinstancelistrr get_idr appendrosenvironanyrfrom_instance_typeas_dictdxpyapijob_newr,r)r.rr rrrrrrrrrrrr!final_depends_onitem req_inputinstance_type_srdcluster_spec_srdfpga_driver_srdnvidia_driver_srdresps r#rz DXJob.newwsV  !*d+&JD!$.*T<2P;;=0")*f"gg(// >#D*5(//5%'IJJJFGG "** $I!)Ig $+Ij !$( &!$( &!%*4 ,'x}l\giv>wxx$:$M$Mm]d$e!#9,#G "8"E$:=$I!3DGW3WZi3il}3}3G3G3I ./".2E ./0<>_ :;%)9 +&"'. )$88##I88D KKT # KK)7x5E/35 6r%c|7t|tr|jdst||j||_y)z :param dxid: New job ID to be associated with the handler (localjob IDs also accepted for local runs) :type dxid: string Discards the currently stored ID and associates the handler with *dxid* Nz localjob-)r>r startswithr _class_dxidr-s r#r,z DXJob.set_ids6  tZ0T__[5Q#45 r%c | | tdi}|||d<|||d<|||d<tjj|j|fi||_|j S)a :param fields: dict where the keys are field names that should be returned, and values should be set to True (by default, all fields are returned) :type fields: dict :param defaultFields: include default fields when fields is supplied :type defaultFields: bool :param io: Include input and output fields in description; cannot be provided with *fields*; default is True if *fields* is not provided (deprecated) :type io: bool :returns: Description of the job :rtype: dict Returns a hash with key-value pairs containing information about the job, including its state and (optionally) its inputs and outputs, as described in the API documentation for the `/job-xxxx/describe `_ method. zEDXJob.describe: cannot provide non-None values for both fields and iofields defaultFieldsio)r rGrH job_describerU_desc)r.rWrXrYr!describe_inputs r#describezDXJob.describesy.  ".ab b  '-N8 $  $.;N? + >#%N4 XX**4::~PP zzr%c ^tjj|jd|ifi|y)z :param tags: Tags to add to the job :type tags: list of strings Adds each of the specified tags to the job. Takes no action for tags that are already listed for the job. rN)rGrH job_add_tagsrUr.rr!s r#add_tagszDXJob.add_tagss% djj64.CFCr%c ^tjj|jd|ifi|y)z :param tags: Tags to remove from the job :type tags: list of strings Removes each of the specified tags from the job. Takes no action for tags that the job does not currently have. rN)rGrHjob_remove_tagsrUr`s r# remove_tagszDXJob.remove_tagss%   fd^FvFr%c ^tjj|jd|ifi|y)z :param allow_ssh: Allowable IP ranges to set for SSH access to the job :type allow_ssh: list of strings Updates a job's allowSSH field, overwrites existing values allowSSHN)rGrH job_updaterU)r. allow_sshr!s r#updatez DXJob.updates& DJJY(?J6Jr%c ^tjj|jd|ifi|y)a :param properties: Property names and values given as key-value pairs of strings :type properties: dict Given key-value pairs in *properties* for property names and values, the properties are set on the job for the given property names. Any property with a value of :const:`None` indicates the property will be deleted. .. note:: Any existing properties not mentioned in *properties* are not modified by this method. rN)rGrHjob_set_propertiesrU)r.rr!s r#set_propertieszDXJob.set_properties&s& ##DJJz0JUfUr%c d} |jd i|}|dk(ry|dk(rl|jd i|}djd i|}|jddk7r)|dd|dk7r|dj|dd z }t ||d k(r t d ||k\s|dkr t d t j |||z })a :param interval: Number of seconds between queries to the job's state :type interval: integer :param timeout: Maximum amount of time to wait, in seconds, until the job is done running :type timeout: integer :raises: :exc:`~dxpy.exceptions.DXError` if the timeout is reached before the job has finished running, or :exc:`dxpy.exceptions.DXJobFailureError` if the job fails Waits until the job has finished running. rdonefailedz;Job has failed because of {failureReason}: {failureMessage} failureFromNr<z (failure from {id}))r< terminatedzJob was terminated.z3Reached timeout while waiting for the job to finishr5) _get_stater]formatgetr timesleep)r.intervaltimeoutr!elapsedstatedescerr_msgs r# wait_on_donezDXJob.wait_on_done7s#DOO-f-E $t}}.v.^W^^faef88M*d2tM7J47PTXY]T^7^5<<]@STX@Y<ZZG'00 $'(=>>'!Wq['(]^^ JJx x G#r%c Xtjj|jfi|y)z0 Terminates the associated job. N)rGrH job_terminaterUr.r!s r# terminatezDXJob.terminateVs tzz4V4r%cPd|j|di}|||dd<|||dd<|S)a :param field: Output field name of this job :type field: string :param index: If the referenced field is an array, optionally specify an index (starting from 0) to indicate a particular member of the array :type index: int :param metadata: If the referenced field is of a data object class, a string indicating the metadata that should be read, e.g. "name", "properties.propkey", "details.refgenome" :type metadata: string Returns a dict containing a valid job-based object reference to refer to an output of this job. This can be used directly in place of a DNAnexus link when used as a job output value. For example, after creating a subjob, the following app snippet uses a reference to the new job's output as part of its own output:: mysubjob = dxpy.new_dxjob({}, "my_function") return { "myfileoutput": mysubjob.get_output_ref("output_field_name"), "myotherfileoutput": mysubjob.get_output_ref("output_array", index=1), "filename": mysubjob.get_output_ref("output_field_name", metadata="name") } z$dnanexus_link)r'fieldindexmetadata)rU)r.rrrlinks r#get_output_refzDXJob.get_output_ref\sH0!$**u"EF  .3D! "7 +  19D! ": . r%c D|jddtdi|dS)z :returns: State of the remote object :rtype: string Queries the API server for the job's state. Note that this function is shorthand for: dxjob.describe(io=False, **kwargs)["state"] rWT)rzrzr5)r]dictrs r#rrzDXJob._get_state{s't}}?Dt$4??HHr%r4 NNNNNNNNNNN)NNN)r i: )NN)__name__ __module__ __qualname____doc__rTr+rr,r]rardrirlr}rrrrr5r%r#rrks_F imVZBFW6r !F D G KV" >5 > Ir%rr)r __future__rrrrrBrurGrr r r exceptionsr rrutils.local_exec_utilsrcompatrr$rr5r%r#rsN" SR KK 86eiX\DH9x]IH]Ir%