CasperSecurity
o
쑛hN7 � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m
Z
ddl m Z m
Z
mZmZmZmZmZ ddlmZ ddlmZ ddlmZmZ ddlmZ dd l m Z m Z m Z eej e�!e"��Z#G d
d � d e�Z$e$d d
�e$dd�e$dd�gZ%e$dd�e$dd�e$dd�e$dd�e$dd�e$d d �e$d d �e$d d!�e$d"d#�e$d$d%�e$d&d%�e$d'd(�g Z& dcd)eej' d*ej'fd+d,�Z(d-ee) d.ej*d/e)d*ee) fd0d1�Z+d-ee) d.ej*d/e)d*dfd2d3�Z,d4ej*d5ej*d6e-d*dfd7d8�Z.d9ej*d6e-d*dfd:d;�Z/d9ej*d*dfd<d=�Z0d9ej*d6e-d*dfd>d?�Z1d@e
e)e f d*eej* fdAdB�Z2e�*dC�fdDej*d*eej* fdEdF�Z3dGej*d*eej* fdHdI�Z4dJej*d*eej* fdKdL�Z5d9ej*d@e
e)e f dJej*dGej*d6e-d*df dMdN�Z6e�*dO�e�*dP�dQfdRe)d@e
e)e f dJej*dGej*d6e-d*df dSdT�Z7dUe8d*dfdVdW�Z9 Q XdddRe)dUe8dYe-dZe-d*df
d[d\�Z:d]e)d^ej;d*e8fd_d`�Z<e"dak�r�e�=e<dbe(� �>� �� dS dS )ezGDefine 'collect-logs' utility and handler to include in cloud-init cmd.� N)�datetime�timezone)�Any�Dict�Iterator�List�
NamedTuple�Optional�cast)�loggers)�Init)�ProcessExecutionError�subp)�tempdir)�copy�get_config_logfiles�
write_filec @ s e Zd ZU eed< eed< dS )�
ApportFile�path�labelN)�__name__�
__module__� __qualname__�str�__annotations__� r r �:/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.pyr s
r z(/var/log/installer/autoinstall-user-data�AutoInstallUserDataz/autoinstall.yaml�AutoInstallYAMLz'/etc/cloud/cloud.cfg.d/99-installer.cfg�InstallerCloudCfgz//var/log/installer/ubuntu_desktop_installer.log�UdiLogz-/var/log/installer/subiquity-server-debug.log�SubiquityServerDebugz-/var/log/installer/subiquity-client-debug.log�SubiquityClientDebugz%/var/log/installer/curtin-install.log� CurtinLogz0/var/log/installer/subiquity-curtin-install.conf�CurtinInstallConfigz8/var/log/installer/curtin-install/subiquity-initial.conf�CurtinConfigInitialz:/var/log/installer/curtin-install/subiquity-curthooks.conf�CurtinConfigCurtHooksz8/var/log/installer/curtin-install/subiquity-extract.conf�CurtinConfigExtractz=/var/log/installer/curtin-install/subiquity-partitioning.conf�CurtinConfigPartitioningz(/var/log/installer/curtin-error-logs.tar� CurtinErrorz$/var/log/installer/curtin-errors.tarz(/var/log/installer/block/probe-data.json� ProbeData�parser�returnc C sh | s t jddd�} | jdddddd d
� | jd d d
dd� | jddddddd� | jdddddd� | S )a2 Build or extend and arg parser for collect-logs utility.
@param parser: Optional existing ArgumentParser instance representing the
collect-logs subcommand which will be extended to support the args of
this utility.
@returns: ArgumentParser with proper argument configuration.
z collect-logsz)Collect and tar all cloud-init debug info)�prog� descriptionz --verbosez-v�countr � verbosityzBe more verbose.)�action�default�dest�helpz --tarfilez-tzcloud-init.tar.gzzOThe tarfile to create containing all collected logs. Default: cloud-init.tar.gz)r2 r4 z--include-userdataz-uF�
store_true�userdataz?DEPRECATED: This is default behavior and this flag does nothing)r2 r1 r3 r4 z--redact-sensitivez-rz�Redact potentially sensitive data from logs. Sensitive data may include passwords or keys in user data and root read-only files.)r2 r1 r4 )�argparse�ArgumentParser� add_argument)r+ r r r �
get_parserO sF ��� �
� r: �cmd� file_path�msgc
C s� |j jddd� zt| �j}W n" ty1 } zt|t|�� t�d|� d}W Y d}~|S d}~ww t||� t�d||j � |S )zCHelper which runs a command and writes output or error to filename.T��parents�exist_ok�collecting %s failed.N�collected %s to file '%s')
�parent�mkdirr �stdoutr
r r �LOG�debug�stem)r; r< r= �output�er r r � _write_command_output_to_file� s ��
rK c
C s� |j jddd� z |�d��}tj| ||d� W d � n1 s!w Y W n tyG } zt|t|�� t� d|� W Y d}~dS d}~ww t� d||j
� dS )z�Helper which runs a command and writes output or error to filename.
`subprocess.call` is invoked directly here to stream output to the file.
Otherwise memory usage can be high for large outputs.
Tr> �w)rE �stderrNrA rB ) rC rD �open�
subprocess�call�OSErrorr r rF rG rH )r; r< r= �frJ r r r � _stream_command_output_to_file� s ����rS r �out_dir�include_sensitivec C sd | � � r*|s| �� jtj@ r"|jddd� t| |� t�d| � dS t�d| � dS t�d| � dS )z-Collect a file into what will be the tarball.Tr> zcollected file: %sz#sensitive file %s was not collectedzfile %s did not existN) �is_file�stat�st_mode�S_IROTHrD r rF rG �trace)r rT rU r r r �
_collect_file� s
r[ �log_dirc C s| t D ]}t�| |jdd� �j}tt�|j�|dd� q|r:tD ]}t�| |jdd� �j}tt�|j�|dd� q dS dS )z'Obtain subiquity logs and config files.� NT)rU )�INSTALLER_APPORT_FILES�pathlib�Pathr rC r[ � INSTALLER_APPORT_SENSITIVE_FILES)r\ rU �src_file�destination_dirr r r �_collect_installer_logs� s"
�
��rd c C s@ t ddg| d dd�}t g d�| d dd�}|s |pd }d
S d
S ) z8Include cloud-init version and dpkg version in the logs.�
cloud-initz --version�versionzcloud-init --version�r; r< r= )z
dpkg-queryz--showz-f=${Version}
re z dpkg-versionz dpkg versionz
not-availableN)rK )r\ rf �dpkg_verr r r �_collect_version_info� s �� �ri c C sL |r t dg| d dd� t g d�| d dd� t g d�t�| d �d
d� d S ) z0Include dmesg and journalctl output in the logs.�dmesgz dmesg.txtz dmesg outputrg )�
journalctlz--boot=0�-o�
short-precisez journal.txtz systemd journal of current boot)rk z --boot=-1rl rm zjournal-previous.txtz systemd journal of previous bootN)rS r_ r` )r\ rU r r r �_collect_system_logs� s ��
�rn �log_cfgc c s � t | �D ]}t�|�V qdS )z7Get paths for cloud-init.log and cloud-init-output.log.N)r r_ r` )ro r r r r �_get_cloudinit_logs� s � �rp z
/etc/cloud�
etc_cloud_dirc # s6 � | d | d dg� � fdd�| � d�D �E dH dS )z�Get paths for all files in /etc/cloud.
Excludes:
/etc/cloud/keys because it may contain non-useful sensitive data.
/etc/cloud/templates because we already know its contents
�keys� templatesz99-installer.cfgc 3 s( � | ]}|j � vr|j� vr|V qd S �N)�namerC )�.0r ��ignorer r � <genexpr> s � ��z!_get_etc_cloud.<locals>.<genexpr>z**/*N��glob)rq r rw r �_get_etc_cloud s �
�
�r| � cloud_dirc C s: t �| �d�| �d�| �d�dd� | �d�D �| �d��S )zkGet paths for files in /var/lib/cloud.
Skip user-provided scripts, semaphores, and old instances.
zdata/*z
handlers/*zseed/*c s s � | ] }|� � r|V qd S rt )rV )rv �pr r r ry s � z%_get_var_lib_cloud.<locals>.<genexpr>z
instance/*zinstance/handlers)� itertools�chainr{ �r} r r r �_get_var_lib_cloud s �r� �run_dirc C s
| � d�S )z�Get all paths under /run/cloud-init except for hook-hotplug-cmd.
Note that this only globs the top-level directory as there are currently
no relevant files within subdirectories.
�*rz �r� r r r � _get_run_dir$ s
r� c C s� t | � t| |� t| |� t|�D ]}t|| t�|�j�d� d� qt �
t � t |d�t
|d��D ]}t|| t�|�j�d� |� q2dS )z8Collect all cloud-init logs into the provided directory.�/Tr� r� N)ri rn rd rp r[ r_ r` rC � relative_tor r� r| r� r� )r\ ro r� r} rU �logfiler r r �_collect_logs_into_tmp_dir- s(
����r� z/run/cloud-initz/var/lib/cloudT�tarfilec C s� t j�| �} t�tj��� �d�}t |d��*}t
� ||�}t |||||d� t
dd| d|t|��|� d�d�g� W d � n1 sCw Y t�d
| � d S ) a� Collect all cloud-init logs and tar them up into the provided tarfile.
:param tarfile: The path of the tar-gzipped file to create.
:param log_cfg: The cloud-init base configuration containing logging cfg.
:param run_dir: The path to the cloud-init run directory.
:param cloud_dir: The path to the cloud-init cloud directory.
:param include_sensitive: Boolean, true means include sensitive data.
zcloud-init-logs-%Y-%m-%d)�dir)r\ ro r� r} rU �tar�czfz-Cr� � NzWrote %s)�osr �abspathr �nowr �utc�date�strftimer r_ r` r� r r �replacerF �info)r� ro r� r} rU �dir_name�tmp_dirr\ r r r � collect_logsN s. � ��� �r� r0 c C s^ t �� | dkr tj}n | dkrtj}nt j}t�|� t�� }|� t�
d�� t� |� dS )z�Set up the logger for CLI use.
The verbosity controls which level gets printed to stderr. By default,
DEBUG and TRACE are hidden.
r r] z %(message)sN) r �
reset_logging�logging�INFO�DEBUG�TRACErF �setLevel�
StreamHandler� setFormatter� Formatter�
addHandler)r0 �level�handlerr r r �
_setup_loggerw s
r� F�redact_sensitive�include_userdatac C s| t |� t�� dkrtd��|rt�d� tg d�}|�� t| |j t
� |j j
�t
� |j j�| d� |s<t�d� dS dS )z:Handle calls to 'cloud-init collect-logs' as a subcommand.r z!This command must be run as root.z;The --include-userdata flag is deprecated and does nothing.)�ds_deps)r� ro r� r} rU z�WARNING:
Sensitive data may have been included in the collected logs.
Please review the contents of the tarball before sharing or
rerun with --redact-sensitive to redact sensitive data.N)r� r� �getuid� RuntimeErrorrF �warningr �read_cfgr� �cfgr_ r` �pathsr� r} )r� r0 r� r� �initr r r �collect_logs_cli� s* �
���r� �_name�argsc
C s^ t � �� }zt|j|j|j|jd� W dS ty. } zt|t j
d� W Y d}~dS d}~ww )z}Handle the CLI interface to the module.
Parse CLI args, redirect all exceptions to stderr, and return an exit code.
)r0 r� r� r� r )�fileNr] ) r: �
parse_argsr� r0 r� r� r6 � Exception�print�sysrM )r� r� rJ r r r �handle_collect_logs_args� s
���r� �__main__r� rt )r TF)?�__doc__r7 r r� r� r_ rW rO r� r r �typingr r r r r r r
�
cloudinit.logr �cloudinit.stagesr �cloudinit.subpr
r �cloudinit.temp_utilsr �cloudinit.utilr r r �CustomLoggerType� getLoggerr rF r ra r^ r8 r: r r` rK rS �boolr[ rd ri rn rp r| r� r� r� r� �intr� r� � Namespacer� �exitr� r r r r �<module> sJ $ �� ��������&��
�9���
����
����
���
���
�
�
� ��
� �
����
�$��
����
�)�����
�!
�