Z j S r SSKrSSKJr SSKJrJrJr SSKJ r J
r
SSK J r J
r
" S S \S
S 9r\
R " \5 r\ " S S
\5 5 rS
/rg)z
Processor class for Donut.
N )
ImageInput)ProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput)auto_docstringloggingc \ rS rSr0 rSrg)DonutProcessorKwargs N)__name__
__module__ __qualname____firstlineno__ _defaults__static_attributes__r {/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/transformers/models/donut/processing_donut.pyr
r
s Ir r
F)totalc ^ \ rS rSrS U 4S jjr\ S S\S- S\\\ - \ - \
- S- S\ \ 4S jj5 r
\S 5 rS S jrS
rU =r$ )
DonutProcessor! Nc $ > [ TU ] X5 g )N)super__init__)selfimage_processor tokenizerkwargs __class__s r r DonutProcessor.__init__# s
4r imagestextr" c J Uc Uc [ S5 eU R " [ 4SU R R 0UD6nUb U R
" U40 US D6nUb. Ub US R
SS5 U R " U40 US D6nUc W$ Uc W$ WS WS' US US' U$ ) NzBYou need to specify either an `images` or `text` input to process.tokenizer_init_kwargs
images_kwargs text_kwargsadd_special_tokensF input_idslabels)
ValueError
_merge_kwargsr
r! init_kwargsr
setdefault)r r% r& r"
output_kwargsinputs encodingss r __call__DonutProcessor.__call__& s >dl ab b **
"&.."<"<
))&SM/4RSF ! m,778LeT tL}]/KLI <M
^ ( 5F8 "+K"8F; Mr c N U R R n[ USS/- 5 $ )Nr, r- )r model_input_nameslist)r image_processor_input_namess r r8 DonutProcessor.model_input_namesF s* &*&:&:&L&L#/;2IIJJr c n Uc U R R 5 n0 nU( Gao [ R " SU[ R 5 nUc GODXR 5 S nSU; a GO*USUR
S5 S- nU[ S5 [ S5 * n[ R " U5 n[ R " SU S3U[ R 5 n U c UR US5 nGOU R 5 n [ R " U5 n
[ R " U 5 n [ R " U
SU 3U[ R [ R - 5 n U b U R S5 R 5 n SU ; a7 SU ; a1 U R U SUS 9n
U
( a [ U
5 S:X a U
S
n
XU' Ou/ XG' U R S 5 HB nUR 5 nX; a US
S :X a US
S S:X a USS
nXG R U5 MD [ XG 5 S:X a XG S
XG' XR! U 5 [ U 5 - S R 5 nUSS S :X a U/U R USS SUS 9- $ U( a GMo U( a U( a U/$ U$ U( a / $ SU0$ )zC
Convert a (generated) token sequence into an ordered JSON format.
Nz z
text_sequence)r! get_added_vocabresearch
IGNORECASEstartindexlenescapereplacegroupDOTALLstrip
token2jsonsplitappendfind)r tokensr@ rA outputpotential_start start_tokenkey key_escaped end_tokenstart_token_escapedend_token_escapedcontentvalueleafs r rR DonutProcessor.token2jsonL s ..88:K ii
FO & !6!6!8!:;K+%%&B (9(9#(>(BCK c%jCH95C ))C.K T+ a"8&"--PI R8 %OO - &(ii &<#$&IIi$8! ))*+51B0CDfbmm^`^g^gNg & %mmA . 4 4 6G (W-? $Ze f "5zQ(-a*/3K&( $+MM)$ K K
8G 8Gr r )__doc__rG image_utilsr processing_utilsr r r tokenization_utils_baser r utilsr
r r
get_loggerr loggerr __all__r r r rn sk
% H H C ,+5
H % bG^ bG bGJ
r