a !q ^a„ã@sêdZddlZddlZddlZddlmZddlm Z ddl m Z m Z m Z mZmZddlmZddlmZmZddl mZdd lmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#d Z$Gd d „d ej%j&ƒZ'de(e(e e)e(dœdd„Z*dS)z>html2text: Turn HTML into equivalent Markdown-structured text.éN)Úwrap)ÚDictÚListÚOptionalÚTupleÚUnioné)Úconfig)Ú AnchorElementÚ ListElement)Ú OutCallback) Údumb_css_parserÚ element_styleÚ escape_mdÚescape_md_sectionÚgoogle_fixed_width_fontÚgoogle_has_heightÚgoogle_list_styleÚgoogle_text_emphasisÚhnÚlist_numbering_startÚpad_tables_in_textÚskipwrapÚ unifiable_n)iärécsÒeZdZddejfeeeeddœ‡fdd„ Z eddœ‡fdd„ Z eedœd d „Z edd œd d „Z edœdd„Z eddœdd„Zeddœdd„Zeeeeeefddœdd„Zeddœdd„Zeeeefeedœdd„Zeeeefeeefddœd d!„Zeeeeefedd"œd#d$„Zddœd%d&„Zddœd'd(„Zddœd)d*„Zd=eeeeefdd,œd-d.„Zd>eedd/œd0d1„Zeed2œd3d4„Zeedœd5d6„Z eeefed7œd8d9„Z!eed:œd;d<„Z"‡Z#S)?Ú HTML2TextNÚ)ÚoutÚbaseurlÚ bodywidthÚreturncs tƒjdd�d|_d|_d|_tj|_tj|_ tj |_ ||_ tj |_tj|_tj|_tj|_tj|_tj|_tj|_tj|_tj|_tj|_ tj!|_"tj#|_$d|_%d|_&d|_'d|_(tj)|_*tj+|_,d|_-tj.|_/tj0|_1tj2|_3tj4|_5tj6|_7d|_8tj9|_:tj;|_<|du�r|j=|_>n||_>g|_?d|_@d|_Ad|_Bd|_Cd|_Dg|_Eg|_Fd|_Gd|_HtI Jd ¡|_Kd|_Lg|_Md|_Nd|_Od|_Pd|_Qd|_Rd |_Sd|_Td|_Ud|_Vi|_Wg|_Xd|_Yd|_Zd|_[d|_\d|_]i|_^||__d|_`d|_ad |_bd |_cd tjdd <dS) zË Input parameters: out: possible custom replacement for self.outtextf (which appends lines of text). baseurl: base URL of the document we process F)Zconvert_charrefsrÚ*Ú_z**NTz^[a-zA-Z+]+://rú _place_holder;Únbsp)eÚsuperÚ__init__Ú split_next_tdÚtd_countÚ table_startr Z UNICODE_SNOBÚ unicode_snobZ ESCAPE_SNOBÚ escape_snobZLINKS_EACH_PARAGRAPHÚlinks_each_paragraphÚ body_widthZSKIP_INTERNAL_LINKSÚskip_internal_linksZ INLINE_LINKSÚ inline_linksZ PROTECT_LINKSÚ protect_linksZGOOGLE_LIST_INDENTÚgoogle_list_indentZIGNORE_ANCHORSÚ ignore_linksZ IGNORE_IMAGESÚ ignore_imagesZIMAGES_AS_HTMLÚimages_as_htmlZ IMAGES_TO_ALTÚ images_to_altZIMAGES_WITH_SIZEÚimages_with_sizeZIGNORE_EMPHASISÚignore_emphasisZ BYPASS_TABLESÚ bypass_tablesZ IGNORE_TABLESÚ ignore_tablesÚ google_docÚ ul_item_markÚ emphasis_markÚ strong_markZSINGLE_LINE_BREAKÚsingle_line_breakZUSE_AUTOMATIC_LINKSÚuse_automatic_linksÚhide_strikethroughZ MARK_CODEÚ mark_codeZWRAP_LIST_ITEMSÚwrap_list_itemsZ WRAP_LINKSÚ wrap_linksZ PAD_TABLESÚ pad_tablesZDEFAULT_IMAGE_ALTÚdefault_image_altÚ tag_callbackZ OPEN_QUOTEÚ open_quoteZ CLOSE_QUOTEÚ close_quoteÚouttextfrÚ outtextlistÚquietÚp_pÚoutcountÚstartÚspaceÚaÚastackÚmaybe_automatic_linkÚ empty_linkÚreÚcompileÚabsolute_url_matcherÚacountÚlistÚ blockquoteÚpreÚstartpreÚcodeÚquoteÚ br_toggleÚ lastWasNLÚ lastWasListÚstyleÚ style_defÚ tag_stackÚemphasisÚdrop_white_spaceÚinheaderÚ abbr_titleÚ abbr_dataÚ abbr_listrÚstressedÚpreceding_stressedÚpreceding_dataÚ current_tagÚ UNIFIABLE)Úselfrrr©Ú __class__©ú6/usr/lib/python3.9/site-packages/html2text/__init__.pyr&%s–    ÿzHTML2Text.__init__)Údatar cs| dd¡}tƒ |¡dS)Nzz )Úreplacer%Úfeed)rortrprrrsrv‰s zHTML2Text.feedcCs8| |¡| d¡| | ¡¡}|jr0t|ƒS|SdS)Nr)rvÚoptwrapÚfinishrDr)rortZmarkdownrrrrrsÚhandle�s   zHTML2Text.handle)Úsr cCs"|j |¡|r|ddk|_dS)NéÿÿÿÿÚ )rJÚappendr_)rorzrrrrrsrI–s zHTML2Text.outtextf)r cCsX| ¡| ¡|jddd�d |j¡}|jr>tjjd}nd}|  d|¡}g|_|S)NrÚend©Úforceznbsp;ú r#) ÚcloseÚpbrÚoÚjoinrJr*ÚhtmlÚentitiesÚhtml5ru)roZouttextr$rrrrrsrx›s  zHTML2Text.finish)Úcr cCs| | |¡d¡dS©NT)Ú handle_dataÚcharref)ror‰rrrrrsÚhandle_charref¯szHTML2Text.handle_charrefcCs| |¡}|r| |d¡dSrŠ)Ú entityrefr‹)ror‰ÚrefrrrrrsÚhandle_entityref²s zHTML2Text.handle_entityref)ÚtagÚattrsr cCs|j|t|ƒdd�dS)NT©rN)Ú handle_tagÚdict)ror‘r’rrrrrsÚhandle_starttag¾szHTML2Text.handle_starttag)r‘r cCs|j|idd�dS)NFr“)r”)ror‘rrrrrsÚ handle_endtagÁszHTML2Text.handle_endtag)r’r cCs�d|vr dSd}t|jƒD]p\}}d|jvr~|jd|dkr~d|jvsPd|vrzd|jvr~d|vr~|jd|dkr~d}nd}|r|SqdS)zÍ :type attrs: dict :returns: The index of certain set of attributes (of a link) in the self.a list. If the set of attributes is not found, returns None :rtype: int ÚhrefNFÚtitleT)Ú enumeraterPr’)ror’ÚmatchÚirPrrrrrsÚ previousIndexÄs"ÿþý zHTML2Text.previousIndex)rNÚ tag_styleÚ parent_styler c Csât|ƒ}t|ƒ}d|vo|j}d}tjD]}||vo:||v}|r(qFq(d|voTd|v} t|ƒont|ƒ on|j } |�r|s‚| s‚| r�|jd7_|r¢|jd7_| rÀ| |j ¡|j d7_ |rÞ| |j ¡|j d7_ | �rÞ| d¡|j d7_ d|_ nÚ|�s| �s| �r*|jd8_d|_ | �rX|j �rH|j d8_ n | d¡d|_ |�r‚|j �rv|j d8_ n | |j ¡| �r¬|j �r |j d8_ n | |j ¡|�s¸| �rÊ|j�sÊ| d¡|�rÞ|jd8_dS) z/ Handles various text emphases z line-throughFÚitalicrú`Tr�N)rr@r ZBOLD_TEXT_STYLE_VALUESrrZrdrKr„r<rer=r\rO) rorNržrŸZ tag_emphasisZparent_emphasisZ strikethroughZboldZ bold_markerr ÚfixedrrrrrsÚhandle_emphasisàsb ÿý        zHTML2Text.handle_emphasis)r‘r’rNr cCsä ||_|jdur(| ||||¡dur(dS|rb|jdurb|dvrb|dksL|jrb| d¡d|_d|_|jrÚi}|r¦|jr„|jdd}t||j |ƒ}|j  |||f¡n4|jr¶|j  ¡ndiif\}}}|jrÚ|jdd}t |ƒ�r|  ¡|�rd|_| t |ƒdd ¡n d|_dS|d v�rl|j�rP|�rFt|ƒ�rF|  ¡n| ¡n|j�rd|d k�rdn|  ¡|d k�rž|�rž|jd k�r”| d¡n | d¡|dk�rÈ|�rÈ|  ¡| d¡|  ¡|dv�rö|�rè|jd7_n|jd8_|dk�r$|�r|jd7_n|jd8_|dv�r4d |_|dk�r†|�rp|  ¡|jddd�d|_|jd7_n|jd8_|  ¡ttdœdd„}|dv�rà|j�sà|�rÄ||ƒ�rÄd |j}n|j}| |¡|�ràd|_|dv�r*|j�s*|�r||ƒ�rd |j} n|j} | | ¡|�r*d|_|dv�rd|�rJ||ƒ�rJd} nd } | | ¡|�rdd|_|j�r‚|j�s‚| |||¡|d!v�r¨|j�s¨| d"¡|j |_|d#k�r|�rÚd|_d$|_ d%|v�r|d%|_n6|jdu�r |j du�söJ‚|j|j!|j <d|_d|_ |d&k�rF|j"�s0| |j#¡n | |j$¡|j" |_"dVtt%t%dd'œd(d)„} |d*k�rà|j&�sà|�rìd+|v�rÞ|d+du�rÞ|j'�r¤|d+ (d¡�sÞ|j  |¡|d+|_d|_|j)�rêd,|d+d-|d+<n |j  d¡nô|j�rà|j  ¡} |j�r|j�sd|_nÊ| �rà| d+du�s.J‚|j�rL| d¡d|_d|_|j*�r~|  +d%¡�pbd$} t,| ƒ} | || d+| ƒnb| -| ¡}|du�rž|j.|}n*|j/d7_/t0| |j/|j1ƒ}|j.  |¡| d.t%|j2ƒd/¡|dk�r4|�r4|j�s4d0|v�r4|d0du�sJ‚|j3�s(|d0|d+<| +d1¡�p8|j4}|j5�s^|j6�rþd2|v�s^d3|v�rþ| d4|d0d5¡d2|v�r¦|d2du�s�J‚| d6|d2d5¡d3|v�rØ|d3du�sÂJ‚| d7|d3d5¡|�rð| d8|d5¡| d9¡dS|jdu�rj|j}|j3�rTt,|ƒ|k�rT|j7 8|¡�rT| d,t,|ƒd-¡d|_dS| d¡d|_d|_|j3�r‚| t,|ƒ¡n²| d:t,|ƒd/¡|j*�rÒ| +d+¡�p®d$}| d;t,t9 :|j;|¡ƒd<¡nb| -|¡}|du�rò|j.|}n*|j/d7_/t0||j/|j1ƒ}|j.  |¡| dt%|j2ƒd/¡|d=k�rL|�rL|  ¡|d>k�rd|�sd| <¡|d?k�r~|�r~| d@¡|d?k�r–|�s–| <¡|dAv� r$|j=�s¸|j>�s¸|  ¡|�rð|j�rÐt?|ƒ}n|}t@|ƒ}|j=  tA||ƒ¡n,|j=� r|j=  ¡|j� s|j=� s| dB¡d|_>nd|_>|dCk� rÞ| <¡|� rÞ|j=� rV|j=d}n tAdDd ƒ}|j� rt| B|¡}n tC|j=ƒ}| dE|¡|jDdDk� rª| |jEd ¡n.|jDdFk� rØ|jFd7_F| t%|jFƒdG¡d|_|dHv� r¢|jG� r|dIk� r¢|� rn| ¡n�n’|jH� r†|� r&| ¡|dJv� rZ|� rH| dK I|¡¡n| dL I|¡¡n(|� rr| dM I|¡¡n| dN I|¡¡�n|dOk� rê|� rÄd|_J|jK� rê| d,tLjMd-¡| d¡n&|jK� rê| dPtLjMd-¡| d¡|dJv� r|� r|jN� r | dQ¡d|_N|dIk� r(|� r(d |_O|dIk� rF|� sFd|_N| ¡|dIk� r„|� s„|jJ� r„| dR PdSg|jO¡¡| ¡d|_J|dJv� r¢|� r¢|jOd7_O|dTk� rà|� rÀd|_Qd|_nd|_|jR� rØ| SdU¡|  ¡dS)WNT)ÚpÚdivraÚdlÚdtZimgú[Fr{éú#r�)r¤r¥r¥Úbrrz > ú Úhrz* * *)ÚheadraZscriptrra)ÚbodyrYú> r)ror cSst|jot d|jd¡ƒS)Nz[^\s]r{)ÚboolrlrTr›©rorrrrrsÚno_preceding_spaceˆsÿz0HTML2Text.handle_tag..no_preceding_space)ZemrœÚu)ÚstrongÚb)ÚdelÚstrikerzz ~~z~~)Zkbdr\Úttr¡Úabbrrr™Úq)roÚlinkr™r cSs@t |j|¡}| ¡r d |¡nd}| djt|ƒ|d�¡dS)Nz "{}"rz]({url}{title}))Úurlr™)ÚurlparseÚurljoinrÚstripÚformatr„r)ror¼r™r½rrrrrsÚlink_urlÈsz&HTML2Text.handle_tag..link_urlrPr˜ú<ú>z][ú]ÚsrcÚaltÚwidthZheightz rLr²rrrrrsr¤§sz HTML2Text.pcCs| ¡d|_dS)z Soft breaksrÐN)rƒr^r²rrrrrsr׫szHTML2Text.soft_brF)rtÚpuredatar€r c Cs2|jdur|j|7_|j�s.|jrR| ¡}|jrD|jsD|jsD|}|dkrRd|_|rŒ|jsŒt dd|¡}|rŒ|ddkrŒd|_ |dd…}|s˜|s˜dS|j rÐ|  d¡sº|  d ¡sºd|}|j rÐ|  d ¡d|_d |j}|rî|rî|dd ksü|jrü|d7}|j�r6|j�s|d 7}|d t|jƒ7}| dd|¡}|j �rVd |_ |j�rV| d¡}|j�rpd |_ d|_d |_|dk�r�d|_|  d¡d |_ |j�r¾|  |jd||j¡d |_ d|_|j �rÞ|j�sØ|  d¡d |_ |j�rÎ|jdk�rú|j�s|dk�rÎ|dk�r|  d¡g}|jD]Ž}|j|jk�r¤|  dt|jƒdt |j|jd¡¡d|jv�r˜|jddu�s€J‚|  d|jdd¡|  d¡n | |¡�q"|j|k�rÈ|  d¡||_|j �r|dk�r|j  !¡D]$\}} |  d|d| d¡�qêd|_|  |¡|jd7_dS)z6 Deal with indentation and whitespace Nrrz\s+r�Trr|z z [code]rÄrÌFr~r©z [z]: r˜r™z (rÊz *[)"rhrKr:ÚlstriprerZr\rTÚsubrOr[rÙrArrLrYrXrÝrurNr^r_rPr,rMrØrÛr¾r¿rr’r}riÚitems) rortràr€Zlstripped_dataZbqZnewar¼rºZ definitionrrrrrsr„°sª         ÿÿÿ   ÿþýÿ     z HTML2Text.o)rtÚ entity_charr cCs|sdS|jr$| ¡}d|_d|_n8|jr\t d|d¡rVt|jƒsV|jdvrVd|}d|_|jrr|j  t |ƒ¡|j durÎ|j }||kr¸|j  |¡r¸|j r¸| d|d¡d|_dS| d ¡d|_ d|_|jsì|jsì|sìt||jd �}||_|j|dd �dS) NFTz[^\s.!?]r)rPr\rZr�rÃrÄr¨)Zsnob)rà)rjrÀrkrTr›rrmrarbÚupdater rRrVr?r„rSr\rZrr+rl)rorträr˜rrrrrsr‹!sFÿþý ÿ þý zHTML2Text.handle_data)rÞr cCsb|ddvr t|dd…dƒ}nt|ƒ}|js>|tvr>t|Sz t|ƒWSty\YdS0dS)Nr)ÚxÚXrrr)Úintr*rÚchrÚ ValueError)rorÞr‰rrrrrsrŒLs   zHTML2Text.charrefcCsd|js|tjvrtj|Sztjj|d}WntyLd|dYS0|dkr`tj|S|S)Nú;ú&r$)r*r rnr†r‡rˆÚKeyError)ror‰ZchrrrrrsrŽZs  zHTML2Text.entityref)rar cCs*d}d|vr&t|ddd…ƒ|j}|S)zq Calculate the nesting count of google doc lists :type style: dict :rtype: int rz margin-leftNéþÿÿÿ)rèr1)rorarßrrrrrsrÜcszHTML2Text.google_nest_count)Útextr cCs|js |Sd}d}|jsd|_| d¡D]Ö}t|ƒdkræt||j|jƒsÈd}| d|j¡rbd}n| d¡rpd}t ||jd|d�}|d  |¡7}|  d¡r¨|d 7}d }qä|rº|d7}d }qä|d 7}d }qþt j  |¡sþ||d7}d }q(|d kr(|d7}|d 7}q(|S) zi Wrap all paragraphs in the provided text. :type text: str :rtype: str rrFr|rÐrÌr°)Zbreak_long_wordsÚsubsequent_indentr¬rz r©)r-rCr/ÚsplitrÝrrBrÙr;rr…Úendswithr ZRE_SPACEr›)rorïÚresultÚnewlinesZparaÚindentÚwrappedrrrrrsrwqsH  ü    zHTML2Text.optwrap)FF)F)$Ú__name__Ú __module__Ú __qualname__r Ú BODY_WIDTHrr rØrèr&rvryrIrxr�r�rrr–r—rr�r±r£r”rƒr¤r×rr„r‹rŒrŽrÜrwÚ __classcell__rrrrrprsr$sNüûd  " þ Jþ {ÿþ q+ rr)r†rrr cCs$|durtj}t||d�}| |¡S)N)rr)r rúrry)r†rrÚhrrrrrsÚ html2text®s rý)rN)+Ú__doc__Z html.entitiesr†Z html.parserrTÚ urllib.parseÚparser¾ÚtextwraprÚtypingrrrrrrr Úelementsr r r Zutilsr rrrrrrrrrrrrÚ __version__ÚparserZ HTMLParserrrØrèrýrrrrrrrsÚs(    <