5i"dZddlmZmZmZmZddlZddlZddlZddl m Z ddl m Z ddl mZddlZddlmZd d lmZd d lmZd d lmZeed ZdZdZej6rdZdZdZdZdZdZ GddeZ!y)zc DXDatabase Handler ************** This remote database handler is a Python database-like object. )print_functionunicode_literalsdivisionabsolute_importN)BytesIO)Lock) cpu_count) DXDataObject)object_exists_in_project)logger) basestringiiii@<c|jD]F\}}t|tstd|dt|tr6td|d|d|S)Nz Expected key z of headers to be a stringzExpected value z! of headers (associated with key z) to be a string)items isinstancer ValueError)headerskeyvalues z/home/marpiech/ifpan-abm-pgxpred/analysis/marpiech-gwas-test/venv/lib/python3.12/site-packages/dxpy/bindings/dxdatabase.py_validate_headersr2s[mmo- U#z*CQR R%, %s,- - - Nc|}td}t|dz}t|dz}t|dz}||krdj||dk7rdSdS||cxkr|krnndj||z S||cxkr|krnndj||z S||cxkr|krnndj||z S||krd j||z Sy ) z'Returns the file size in readable form.iz{0} {1}r bytesbytez {0:.2f} KiBz {0:.2f} MiBz {0:.2f} GiBz {0:.2f} TiBN)floatformat) num_bytesBKBMBGBTBs r_readable_part_sizer'<sA tB rDy B rDy B rDy B2va1f7AA&AA q2##AbD)) q2##AbD)) q2##AbD)) q##AbD)) rc.tj|y)N)rdebug)msgs rdo_debugr+Os LLrceZdZdZdZeejjZ e Z ejje ZdZddeddfdZd dZy) DXDatabasezRemote database object handler. :param dxid: Object ID :type dxid: string :param project: Project ID :type project: string database) max_workersNO_PROJECT_HINTNFctj|||t|_||_||_||_d\|_|_|_ t|_ d\|_ |_ t|_y)a> :param dxid: Object ID :type dxid: string :param project: Project ID :type project: string :param read_buffer_size: size of read buffer in bytes :type read_buffer_size: int :param expected_file_size: size of data that will be written, if known :type expected_file_size: int :param file_is_mmapd: True if input file is mmap'd (if so, the write buffer size will be constrained to be a multiple of the allocation granularity) :type file_is_mmapd: bool )dxidprojectNNN)NNN)r __init__r _read_buf _read_bufsize_expected_file_size_file_is_mmapd _download_url_download_url_headers_download_url_expiresr_url_download_mutex_request_iterator_response_iteratorset_http_threadpool_futures)selfr2r3read_buffer_sizeexpected_file_size file_is_mmapds rr5zDXDatabase.__init__esv$ dw? -#5 +VfRD68R$(6 :D7 7(+%rc i}|||d<|@dtjvr.|j}|rt|j |r|}||t j ur||d<dtjvrLttjdd5}|$|t j k7r|j|ddd|j5|j!|jtjkrd|vr t|d<tdj|t!j"j$|j&|fi|}td j||d |_ t)|j+d i|_d |_ |j} t/j.|j,} ddd| | fS#1swYxYw#1swY  fSxYw) a :param filename: desired filename of the local downloaded file :type filename: str :param project: ID of a project containing the file (the download URL will be associated with this project, and this may affect which billing account is billed for this download). If no project is specified, an attempt will be made to verify if the database is in the project from the DXDatabase handler (as specified by the user or the current project stored in dxpy.WORKSPACE_ID). Otherwise, no hint is supplied. This fall back behavior does not happen inside a job environment. :type project: str :returns: download URL and dict containing HTTP headers to be supplied with the request :rtype: tuple (str, dict) :raises: :exc:`~dxpy.exceptions.ResourceNotFound` if a project context was given and the database was not found in that project context. :raises: :exc:`~dxpy.exceptions.ResourceNotFound` if no project context was given and the database was not found in any projects. Obtains a URL that can be used to directly download files associated with the specified database. Nfilename DX_JOB_IDprojectContext_DX_DUMP_BILLED_PROJECTwtimeoutz5dxdatabase get_download_url - download_file args = {}z5dxdatabase get_download_url - download_file resp = {}urlrlL")osenviron get_proj_idr get_idr-r0openwriter=r:r<timeFILE_REQUEST_TIMEOUTr+r dxpyapidatabase_download_file_dxidrgetr;copy) rBrG src_filenamer3kwargsargsproject_from_handlerfdrespretval_download_urlretval_download_url_headerss rget_download_urlzDXDatabase.get_download_urls2  #+D  ?{"**<#'#3#3#5 #(@Pd(e.  7*2L2L#L%,D! " % 2bjj!:;SA &R&7j6P6P+PHHW% & % % P!!)T-G-G$))+-UF*(txx SU?V-W*-8* #'"4"4 *.))D4N4N*O '3 P6#$???? & & P6#$???s'GC6GGG*r4)__name__ __module__ __qualname____doc___class staticmethodrVrW file_describe _describeDXFILE_HTTP_THREADS_http_threadpool_sizeutilsget_futures_threadpool_http_threadpoolr0DEFAULT_BUFFER_SIZEr5rdrrr-r-Rs`FTXX334I/zz88EZ8['O $AT$(!.HQ@rr-)"rh __future__rrrrrNr[rTior threadingrmultiprocessingr rVr utils.resolverr rcompatrminrmMIN_BUFFER_SIZErrJOB_IDMD5_READ_CHUNK_SIZErUrr'r+r-rsrrrs"SR% 5)+q)";;'!*&H@H@r