a ܓiL@sddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZmZmZmZdd lmZeeeee d ZGd ddZdS)N)quote)load) InitSystem)JujuSSH)SSHControlPersist)LocalTransport) OCTransport)SaltStackMaster)CommandTimeoutExceptionConnectionExceptionUnsupportedHostExceptionInvalidTransportException)sos_parse_version)localZcontrol_persistocZ saltstackZjujuc@seZdZdjddZeddZddZd d Zd d Zed dZ eddZ ddZ ddZ ddZ ddZdkddZeddZddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zd.d/Zdld0d1Zd2d3Zd4d5Zd6d7Zd8d9Zdmdd?Z d@dAZ!dBdCZ"dDdEZ#dFdGZ$dHdIZ%dJdKZ&dLdMZ'dNdOZ(dPdQZ)dRdSZ*dTdUZ+dVdWZ,dXdYZ-dZd[Z.d\d]Z/d^d_Z0d`daZ1dbdcZ2dddeZ3dfdgZ4dhdiZ5dS)nSosNodeNTc Cs||_||_|d|_||d|_|d|_|d|_|d|_|jj |_ |jj |_ d|_ d|_ d|_ d|_i|_i|_|p|jj|_|jjs|jjs|j|j_|r||j_d|_d|_d|_g|_dgggg|dd|_d |_td |_td |_|||_ |j j!r(|j j!d k|_z|j "|jWn8t#yr}z|$d |WYd}~n d}~00|%|_ d|_&|j j|_|j r|jj'rd}|j(r|r|j s|j )dS|j r|*rd|j _+|j j+r|,|-dS)NZ cmdlineoptstmpdirhostlen need_sudo sos_optionsFsos_cmd)versionenableddisabledoptionspresetsr sosreportZsosZsos_uirootzUnable to open remote session: ).stripaddresscommonsopts_assign_config_optsrrrrnode_config_fileinherit_config_filerhostclusterhostname sos_env_vars _env_varspassword _passwordZ nopasswd_sudoZsudo_pwsos_path retrievedZhash_retrieved file_listsos_infosos_binloggingZ getLoggersoslogui_log_load_remote_transport _transportZ default_userZconnect Exception log_errordetermine_host_policy _sudo_binaryZno_local connected disconnectcheck_in_container containerizedcreate_sos_container_load_sos_info)selfrr r*Z local_sudoZ load_factserrrB9/usr/lib/python3.9/site-packages/sos/collector/sosnode.py__init__,sv                  zSosNode.__init__cCs|jr|jjSdS)NT)r5r:r@rBrBrCr:rszSosNode.connectedcCs|jdS)zDWrapper to close the remote session via our transport agent N)r5r;rErBrBrCr;yszSosNode.disconnectcCst|jdvrd|_t|j|S|jjtvr|d}|d|j|d<q>|jS)Nz env --nullstatusroutput=rK)r)rosenvironcopy run_commandsplit)r@retZln_valrBrBrCenv_varss   zSosNode.env_varscCsl|jsf|jjdd}|d|}rT|ddksTd|}||t||dkr`d}||_|jS) N/z command -v rPrz*Privilege escalation command not in PATH: sudozsudo -S)r9r! sudo_binaryrXrWr7r6)r@Z_binrYrArBrBrCr_s   zSosNode.sudo_binarycCs`||_|jd|j|jd|jj|jd|jd|jdd|jd|jjdS) z>Set the manifest section that this node will write to r'policyZ sos_versionrfinal_sos_commandrHN)manifest add_fieldrLr%os_release_namer/r5name)r@rcrBrBrCset_node_manifests zSosNode.set_node_manifestcCs<tjdr|ddStjddur8|ddSdS)zV Tries to identify if we are currently running in a container or not. z/run/.containerenvz/Found /run/.containerenv. Running in container.T containerNz/Found env var 'container'. Running in containerF)rTpathexists log_debugrUgetrErBrBrCr<s   zSosNode.check_in_containercCs|jjr|jj|jj||jjd}|j|dd}|ddvr|ddkrd|dvrf|d t d |dvr|d t |j|j dd}|dd kr| d |jj ddS|d|dt |d|dt dS)zJIf the host is containerized, create the container we'll be using )imageZauthZ force_pullT need_rootrP)r}rpzunable to retrieve auth tokenrQzHCould not pull image. Provide either a username and password or authfilezunknown: Not foundz%Specified image not found on registryrzTemporary container z createdz(Could not start container after create: z$Could not create container on host: F) r%r=r>r!rmget_container_authZforce_pull_imagerWr7r6Zrestart_sos_containerlog_infoZsos_container_name)r@cmdresrYrBrBrCr>sD        zSosNode.create_sos_containercCsB|jjr$|jjd|jj|jjS|jjd|jjp>|jjS)zrDetermine what the auth string should be to pull the image used to deploy our temporary container default) r!Z registry_userr%ZruntimesZfmt_registry_credentialsZregistry_passwordZfmt_registry_authfileZregistry_authfileZcontainer_authfilerErBrBrCrqs  zSosNode.get_container_authFcCs<z"|jd||d}|ddkWSty6YdS0dS)z3Checks for the presence of fname on the remote nodezstat rnrPrFN)rWr6)r@fnamerortrBrBrC file_existss  zSosNode.file_existscCs|jrd|jvr|jS|jS)NrF)r'rrErBrBrCrLszSosNode._hostnamecCsd}t|d|S)zUAttempts to obfuscate sensitive information in log messages such as passwordsz>(?P(pass|key|secret|PASS|KEY|SECRET).*?=)(?P.*?\s)z\g****** )resub)r@rNZregrBrBrC_sanitize_log_msg szSosNode._sanitize_log_msgcCs|j||dS)z:Format a ui message that includes host name and formattingN)r3inforOrMrBrBrCui_msgszSosNode.ui_msgcCs8tdd}d|jd|d|}|j|dS)z#Used to print and log info messagesrK[:] N)inspectstackrLr2r{r@rNcallerZlmsgrBrBrCrrszSosNode.log_infocCs8tdd}d|jd|d|}|j|dS)z$Used to print and log error messagesrKr}r~rrN)rrrLr2errorrrBrBrCr7szSosNode.log_errorcCsB||}tdd}d|jd|d|}|j|dS)z$Used to print and log debug messagesrKr}r~rrN)rzrrrLr2debug)r@rNrrBrBrCrks zSosNode.log_debugcCsX|jdtjddk}|jr(|r(|S|jjr>dt|S|jrT|j d|S|S)zIf we need to provide a sudo or root password to a command, then here we prefix the command with the correct bits ZUSERrzsu -c  ) r)rlrTrUrr! become_rootrrr_)r@rsZis_rootrBrBrC _format_cmd&s zSosNode._format_cmdc Csd}d}|jjdurN|jj|jj}|durd|d}|dr|d}n8|j|jjddd}|ddkr|d d \}}}|rt |dd kr|d 7}z|d |dd7}Wn4t y}z| d |WYd}~n d}~00||j d<|j dr$| d|j dn$|j|jjks>|dd|_dS|drZd|_|jd}|j|ddd}|ddkr||d |dr|dS)zLQueries the node for information about the installed version of sos N.rreleaseT use_containerrorPrrQ-z.0z#Unable to fully parse sos release: zsos version is z!sos is not installed on this nodeFz4.0z sos reportz -l3.6)r%Zcontainer_version_commandpackage_managerZ pkg_versionZ sos_pkg_namejoinrWrrXlenr6rkr/rrrr!Zprimaryr7r:check_sos_versionr0_load_sos_plugins_load_sos_presets) r@verrelpkgZpkgs_rArssosinforBrBrCr?3sH    &      zSosNode._load_sos_infocCsn|jd}|j|ddd}|ddkrj|dD]4}|dr4|dd}|jd |q4dS) Nz --list-presetsTrrPrrQzname:rKr)r0rW splitlinesr startswithrXr/append)r@rsrtlineZpnamerBrBrCrgs  zSosNode._load_sos_presetsc Csd}d}d}d}d}|d|}|d|}|d|} |dd} ||||jd<||||jd <|| ||jd <|| |d |jd <dS) Nz,The following plugins are currently enabled:z-The following plugins are currently disabled:z4The following options are available for ALL plugins:z+The following plugin options are available:z Profiles:z(.*?)z rrrTZprofiles)_regex_sos_helpr/) r@rZENABLEDZDISABLEDZ ALL_OPTIONSZOPTIONSZPROFILESZ enableregZdisregZoptregZproregrBrBrCrps    zSosNode._load_sos_pluginsc Csg}t||tjD]}|D]|}|s~z|}rD||dWqtyz}z|d|WYd}~qd}~00q |d} |dd| Dq q|S)NrzError parsing sos help: ,css|]}|r|VqdSN)r).0prBrBrC z*SosNode._regex_sos_help..) rxfindallSrrXrr6rkextend) r@ZregexrZis_listrtresultrZlsrArrBrBrCrs  ( zSosNode._regex_sos_helpc Cs`z|d||j|WStyZ}z$|d|d|WYd}~dSd}~00dS)z1Reads the specified file and returns the contentsz Reading file zException while reading : Nrb)rrr5 read_filer6r7)r@Zto_readrArBrBrCrs zSosNode.read_filec Csz|jr(|d|jdj|jdSti|jjtd|jj | dd}|rh|d|jd|S| dt d S) z[Attempts to identify the host installation against supported distributions zusing local policy r`Tz/etc/os-release)cachesysrootinitZ probe_runtimeZ remote_execZ remote_checkzloaded policy z for hostz4Unable to determine host installation. Ignoring nodeN) rrrr rerr!rrr5rWrr7r )r@r%rBrBrCr8s  zSosNode.determine_host_policyc CsXz|jd}t|t|kWStyR}z|d|WYd}~dSd}~00dS)aChecks to see if the sos installation on the node is AT LEAST the given ver. This means that if the installed version is greater than ver, this will still return True :param ver: Version number we are trying to verify is installed :type ver: ``str`` :returns: True if installed version is at least ``ver``, else False :rtype: ``bool`` rzError checking sos version: NF)r/rr6r7)r@rZ _node_verrArBrBrCrs  zSosNode.check_sos_versioncCs|js dS|jj|duS)z2Checks if a given package is installed on the nodeFN)r%rZ pkg_by_name)r@rrBrBrC is_installedszSosNode.is_installedrGc Cs|jsr|jsr|dz$|j|j}|s8|dtWn6typ}z|d|WYd}~n d}~00|r|jj r|j |}|r| |}|r|j } | |}|j|||||S)aRuns a given cmd, either via the SSH session or locally :param cmd: The full command to be run :type cmd: ``str`` :param timeout: Time in seconds to wait for `cmd` to complete :type timeout: ``int`` :param use_shell: If a shell is needed to run `cmd`, set to True :type use_shell: ``bool`` or ``auto`` for transport-determined :param use_container: Run this command in a container *IF* the host is a containerized host :type use_container: ``bool`` :param env: Pass environment variables to set for this `cmd` :type env: ``dict`` z-Node is disconnected, attempting to reconnectzFailed to reconnect to nodez!Error while trying to reconnect: N)r:rrkr5Z reconnectr+r r6r%r=Zformat_container_commandrr[updaterW) r@rstimeout use_shellrorenvZ reconnectedrAZ_cmd_envrBrBrCrWs(       zSosNode.run_commandc Csxz6|}|r||n |d|jr4||_Wn4tyj}z|d|WYd}~n d}~00|dS)z.Run an sos report on the node, then collect itz'Unable to determine path of sos archivezError during sos execution: N)execute_sos_commandfinalize_sos_pathr7r,retrieve_sosreportr-r6cleanup)r@rirArBrBrCrs  &zSosNode.sosreportcCs||jdvS)z/Verifies if the given preset exists on the noderr/)r@presetrBrBrC_preset_existsszSosNode._preset_existscs&tfdd|jd|jdfDS)z/Verifies if the given plugin exists on the nodec3s|]}|vVqdSrrB)rspluginrBrCrrz)SosNode._plugin_exists..rr)anyr/r@rrBrrC_plugin_existsszSosNode._plugin_existscCs||jdvS)z6Checks to see if the plugin is default enabled on noderrrrBrBrC_check_enabledszSosNode._check_enabledcCs||jdvS)z7Checks to see if the plugin is default disabled on noderrrrBrBrC_check_disabledszSosNode._check_disabledcCsR|dd}||sdS||r6||jjvr6dS||rN||jdvSdS)zAttempts to verify that the given option is available on the node. Note that we only get available options for enabled plugins, so if a plugin has been force-enabled we cannot validate if the plugin option is correct or notrrFrT)rXrrr!enable_pluginsrr/)r@optplugrBrBrC_plugin_option_exists s    zSosNode._plugin_option_existscsdfdd|DS)z^Returns a comma delimited list for sos plugins that are confirmed to exist on the noderc3s|]}|r|VqdSrrrorErBrCrrz,SosNode._fmt_sos_opt_list..)r)r@r!rBrErC_fmt_sos_opt_listszSosNode._fmt_sos_opt_listcCs ||_dS)zVExpose the node to the cluster profile determined for the environment N)r&)r@r&rBrBrC set_cluster szSosNode.set_clustercs^|jjr0|js|jj|_n|d|jjd|jjr\|jjD]}||jvr@|j|q@|jjr|jjD] |jvrl|jj|j<ql|jjr|jjD]>t fdd|j Dsd|jj}|j |q|j |r$|jj |j |Wdn1s0Yn6|jj |j|Wdn1sP0YdS)a+This is used to modify the sos report command run on the nodes. By default, sos report is run without any options, using this will allow the profile to specify what plugins to run or not and what options to use. This will NOT override user supplied options. zCluster specified preset z> but user has also defined a preset. Using user specification.c3s|]}|vVqdSrrBrrrBrCrBrz2SosNode.update_cmd_from_cluster..rSN)r&Z sos_presetrrrZ sos_pluginsrrrZsos_plugin_optionsrplugoptsZcheck_node_is_primarylockZset_primary_optionsZset_node_options)r@roptionrBrrCupdate_cmd_from_cluster&s0        . zSosNode.update_cmd_from_clustercCsJt|jj|_t|jj|_t|jj|_t|jj|_t|jj|_dS)zFrom the global opts configuration, assign those values locally to this node so that they may be acted on individually. N)listr! only_plugins skip_pluginsrrrrErBrBrCr"Ns zSosNode._assign_config_optscsjd}}|r*|dt|d}g}dr`jjdkr`|dttjjdrjjr|dttjjdr܈jj r|d jj r|d jj r|d tjj d r8jj r|d td jj jjr8|dtd jjdrjjjrj|dttjjdsdrˆjjdkr|djjjjr|djjdrjjr|djjjjr|d|dtj jjj}jD]0}j|}|d|d|rN|ndq*jrfddjD}|r|dtd d d!|Djrjr|d"tjnd#jd$j rfd%dj D}t!|t!j krsz,SosNode.finalize_sos_cmd..z-k css|] }|VqdSrrBrrBrBrCrrz+SosNode.finalize_sos_cmd..z --preset=zRequested to enable preset z" but preset does not exist on nodecsg|]}|r|qSrBrrrErBrCrrzRequested plugins z. were requested to be enabled but do not existz--only-plugins=zFinal sos command set to raNcsg|]}|r|qSrB)rrrErBrCrrzRequested to skip plugins z%, but plugins are already not enabledz--skip-plugins=cs.g|]&}|jvr|r|r|qSrB)rrrrrErBrCrs zRequested to enable plugins z1, but plugins are already enabled or do not existz--enable-plugins=)+r/determine_sos_labelrrr!threadsrstrZplugin_timeoutZallow_system_changesZ no_env_varsZsinceZ skip_commandsrZ skip_filesZ cmd_timeoutZcontainer_runtimeZ namespacesZ journal_sizeZ low_priorityrreplacerTrir%Z sos_bin_pathr0rrrrrkrrrsetrrrrrcrdrr)r@rlabelZsos_optsrrZr!ZplugsZnot_onlyZonlyskipZnot_skipZskiplnZnot_onenablerBrErCfinalize_sos_cmdYs                     $$   zSosNode.finalize_sos_cmdcCsd}||j|7}|jjr<||s,|jjn d|jj7}|sDdS|d||drdd}nd}|jdd d|}|d |S) z?Determine what, if any, label should be added to the sos reportrbrNzLabel for sos report set to rz--labelz--namerrrS)r&Zget_node_labelr!rrkrrrX)r@rZlcmdrBrBrCrs  zSosNode.determine_sos_labelcCs^|jj}|r||d}|d}|d|||_|dd|_|jd|jdS)z\Use host facts to determine if we need to change the sos path we are retrieving fromrbrzFinal sos path: r\r]Zcollected_archiveN) r%Zsos_path_striprrXrrr,archivercrd)r@riZpstriprBrBrCrs  zSosNode.finalize_sos_pathcCsT|dkr dS|dkr d|vr dS|dkr,dSt|dkrJ|d ddSd |S) Nr]z+sos report process received SIGKILL on noderKr^zsudo attempt failedz4sos report terminated unexpectedly. Check disk spacer zsos exited with code )rrX)r@rcstdoutrBrBrCdetermine_sos_errors zSosNode.determine_sos_errorc Cs8|dzd}d}d}|jjr2d|jj}nR|jjr|jsvd|jddd}|j|jj |d|}nd|jj }|r|j d ||_ |j |j |jj d d d |j d }|d d kr|dD]J}t|dr|}|dr|dd}q|dr|d}q|r~|jd|t|dkrP|jddn,t|dkrn|jddn|jddn|jddn:||d |d}|d|d d|dt||WSty|dYnHty2}z.|d||d |WYd!}~n d!}~00d!S)"z2Run sos report and capture the resulting file pathzGenerating sos report...Frbz--config-file=z/tmp/r\r]z.confrT)rrrorrrPrrQz*sosreport-*tar*)z sha256 z md5  rKzThe checksum is: r}checksum Z checksum_typeZmd5@Zsha256unknownzError running sos report. rc = z msg = zTimeout exceededz'Exception during sos report execution: zError running sos report: N)r|r!r#r$rrrXr5Zcopy_file_to_remoteZ config_filerrWrr(rfnmatchrrrcrdrrrkr6r r7rr)r@rirZconfig_file_argZ remote_configrtrrArBrBrCr sr       zSosNode.execute_sos_commandc Cs|jd}tj||dd}zH||rT|d|d||j||WS| d|dWdSt y}z$| d|d |WYd }~dSd }~00d S) z7Copies the specified file from the host to our temp dirr\r]zCopying remote z to local zAttempting to copy remote file %, but it does not exist on filesystemFzFailed to retrieve rN) rrTrirrXrwrrr5 retrieve_filerkr6)r@riZdestdirdestrArBrBrCrEs  zSosNode.retrieve_filec Csd|}zt|ddkr:|d|dWdS||rz|d|d|}|j|d d }|d d kWS|d |dWdSty}z$|d|d|WYd}~dSd}~00dS)z|Removes the spciefied file from the host. This should only be used after we have retrieved the file already rbr\rzRefusing to remove path z0: appears to be incorrect and possibly dangerousFzRemoving file zrm -f TrnrPrz!Attempting to remove remote file rzFailed to remove rN)rrXrrkrwrrrWr6)r@rirsrterBrBrC remove_fileTs  zSosNode.remove_filec Cs|js|jjr>z||jWnty<|dYdS0|d|j| dz| |j}Wn0ty}z||WYd}~dSd}~00|r| d|j |j ddd S| d dS) z,Collect the sos report archive from the nodezFailed to make archive readableFzRetrieving sos report from zRetrieving sos report...Nz!Successfully collected sos reportr\r]TzFailed to retrieve sos report)rr!rmake_archive_readabler,r6r7rrrr|rr.rrX)r@rYrArBrBrCrjs&      zSosNode.retrieve_sosreportcCsT|jdus|jrdSd|jvr6|d|jddS||j}|sP|ddS)zvRemove the sos report archive from the node, since we have collected it and it would be wasted space otherwiseNrzNode sos report path z/ looks incorrect. Not attempting to remove pathzFailed to remove sos report)r,rrkrr7)r@ZremovedrBrBrCremove_sos_archives  zSosNode.remove_sos_archivecCsL||jr,dD]}||j|rq,q|j}|rH|j|dddS)zrqrwrLrzr|rrr7rkrr?rrrrr8rrrWrrrrrrrrrr"rrrrrrrrrrrrrBrBrBrCr*sv F     (   4    ((   < r)rrr1rTrxZshlexrZ sos.policiesrZsos.policies.init_systemsrZsos.collector.transports.jujurZ(sos.collector.transports.control_persistrZsos.collector.transports.localrZsos.collector.transports.ocrZ"sos.collector.transports.saltstackr Zsos.collector.exceptionsr r r r Z sos.utilitiesrrIrrBrBrBrC s*