ó R–joãó.•%SrSSKrSSKJr SSKJrJr \R"\5r "SS\5r \S\ S-4r \Sr 0q \\\ 4\S 'Sq\\\ 4S-\S 'S \\ -S \ S S4SjrS \\ -S S4SjrS \\4SjrS \S-4SjrSS\S S4Sjjrg)zÏRegistry for flash attention implementations. This module contains the registration system for flash attention implementations. It has no torch dependencies to avoid circular imports during initialization. éN)ÚCallable)ÚLiteralÚProtocolcó•\rSrSrSSjrSrg)ÚFlashAttentionHandleéNcó•g)N©)Úselfs Úm/root/GenerationalWealth/GenerationalWealth/venv/lib/python3.13/site-packages/torch/nn/attention/_registry.pyÚremoveÚFlashAttentionHandle.removes€˜cór )ÚreturnN)Ú__name__Ú __module__Ú __qualname__Ú__firstlineno__r Ú__static_attributes__r rr rrs†ß!rr.)ÚFA3ÚFA4Ú_FLASH_ATTENTION_IMPLSÚ_FLASH_ATTENTION_ACTIVEÚimplÚ register_fnrcó•U[U'g)aÓ Register the callable that activates a flash attention impl. .. note:: This function is intended for SDPA backend providers to register their implementations. End users should use :func:`activate_flash_attention_impl` to activate a registered implementation. Args: impl: Implementation identifier (e.g., ``"FA4"``). register_fn: Callable that performs the actual dispatcher registration. This function will be invoked by :func:`activate_flash_attention_impl` and should register custom kernels with the PyTorch dispatcher. It may optionally return a handle implementing :class:`FlashAttentionHandle` to keep any necessary state alive. Example: >>> def my_impl_register(module_path: str = "my_flash_impl"): ... # Register custom kernels with torch dispatcher ... pass # doctest: +SKIP >>> register_flash_attention_impl( ... "MyImpl", register_fn=my_impl_register ... ) # doctest: +SKIP N)r)rrs r Úregister_flash_attention_implrs€ð<$/Ô˜4Ò rcó”•[SS9 [RU5nUc[SUS[ 535eU"5nUbX4qgg)a# Activate into the dispatcher a previously registered flash attention impl. .. note:: Backend providers should NOT automatically activate their implementation on import. Users should explicitly opt-in by calling this function or via environment variables to ensure multiple provider libraries can coexist. Args: impl: Implementation identifier to activate. See :func:`~torch.nn.attention.list_flash_attention_impls` for available implementations. If the backend's :func:`register_flash_attention_impl` callable returns a :class:`FlashAttentionHandle`, the registry keeps that handle alive for the lifetime of the process (until explicit uninstall support exists). Example: >>> activate_flash_attention_impl("FA4") # doctest: +SKIP F)Ú _raise_warnNzUnknown flash attention impl 'z'. Available implementations: )Úrestore_flash_attention_implrÚgetÚ ValueErrorÚlist_flash_attention_implsr)rrÚhandles r Úactivate_flash_attention_implr%=sk€ô2!Øòô)×,Ñ,¨TÓ2€KØÑÜØ,¨T¨Fð3*Ü*DÓ*FÐ)Gð Ió ð ñ ‹]€FØ ÑØ#' .Ñðrcó<•[[R55$)zBReturn the names of all available flash attention implementations.)ÚsortedrÚkeysr rr r#r#fs€ä Ô(×-Ñ-Ó/Ó 0Ð0rcó.•[b [S$[$)z Return the currently activated flash attention impl name, if any. ``None`` indicates that no custom impl has been activated. r)rr rr Úcurrent_flash_attention_implr*ks#€ô #Ñ .ô  Ñ"ðô%ðrrcó•Sn[b [SnUbUR5 SqgU(a[RS5 Sqg)z( Restore the default FA2 implementation NézDTrying to restore default FA2 impl when no custom impl was activated)rr ÚloggerÚwarning)rr$s r r r xsL€ð €FÜÑ*Ü(¨Ñ+ˆà ÑØ ‰ Œð #Ñö Ü‰Ø Rô ð#Ñr)T)Ú__doc__ÚloggingÚcollections.abcrÚtypingrrÚ getLoggerrr-rÚ _RegisterFnÚ_FlashAttentionImplrÚdictÚstrÚ__annotations__rÚtuplerr%Úlistr#r*Úboolr r rr Úr<süðòó Ý$ß$ð × Ò ˜8Ó $€ô"˜8ô"ðsÐ0°4Ñ7Ð7Ñ8€ ؘlÑ+Ðà13И˜S +Ð-Ñ.Ó3àCGИ˜sÐ$8Ð8Ñ9¸DÑ@ÓGð/Ø Ð#Ñ #ð/ðð/ð ô /ðB&1Ø Ð#Ñ #ð&1à ô&1ðR1 D¨¡Iô1ð  c¨D¡jô ñ#¨dð#¸dö#r