파일 시스템 API

HfFileSystem 클래스는 fsspec을 기반으로 Hugging Face Hub에 Python 파일 인터페이스를 제공합니다.

HfFileSystem

HfFileSystem은 fsspec을 기반으로 하므로 제공되는 대부분의 API와 호환됩니다. 자세한 내용은 가이드 및 fsspec의 API 레퍼런스를 확인하세요.

class huggingface_hub.HfFileSystem

< source >

( *args **kwargs )

Parameters

endpoint (str, optional) — Endpoint of the Hub. Defaults to https://huggingface.co.
token (bool or str, optional) — A valid user access token (string). Defaults to the locally saved token, which is the recommended method for authentication (see https://huggingface.co/docs/huggingface_hub/quick-start#authentication). To disable authentication, pass False.
block_size (int, optional) — Block size for reading and writing files.
expand_info (bool, optional) — Whether to expand the information of the files.
**storage_options (dict, optional) — Additional options for the filesystem. See fsspec documentation.

Access a remote Hugging Face Hub repository as if were a local file system.

[!WARNING][HfFileSystem](/docs/huggingface_hub/v1.2.1/ko/package_reference/hf_file_system#huggingface_hub.HfFileSystem) provides fsspec compatibility, which is useful for libraries that require it (e.g., reading Hugging Face datasets directly with pandas). However, it introduces additional overhead due to this compatibility layer. For better performance and reliability, it’s recommended to use HfApi methods when possible.

Usage:

>>> from huggingface_hub import hffs

>>> # List files
>>> hffs.glob("my-username/my-model/*.bin")
['my-username/my-model/pytorch_model.bin']
>>> hffs.ls("datasets/my-username/my-dataset", detail=False)
['datasets/my-username/my-dataset/.gitattributes', 'datasets/my-username/my-dataset/README.md', 'datasets/my-username/my-dataset/data.json']

>>> # Read/write files
>>> with hffs.open("my-username/my-model/pytorch_model.bin") as f:
...     data = f.read()
>>> with hffs.open("my-username/my-model/pytorch_model.bin", "wb") as f:
...     f.write(data)

Specify a token for authentication:

>>> from huggingface_hub import HfFileSystem
>>> hffs = HfFileSystem(token=token)

init

< source >

( *args endpoint: typing.Optional[str] = None token: typing.Union[bool, str, NoneType] = None block_size: typing.Optional[int] = None expand_info: typing.Optional[bool] = None **storage_options )

resolve_path

< source >

( path: str revision: typing.Optional[str] = None ) → HfFileSystemResolvedPath

Parameters

path (str) — Path to resolve.
revision (str, optional) — The revision of the repo to resolve. Defaults to the revision specified in the path.

Returns

HfFileSystemResolvedPath

Resolved path information containing repo_type, repo_id, revision and path_in_repo.

Raises

ValueError or NotImplementedError

ValueError — If path contains conflicting revision information.
NotImplementedError — If trying to list repositories.

Resolve a Hugging Face file system path into its components.

ls

< source >

( path: str detail: bool = True refresh: bool = False revision: typing.Optional[str] = None **kwargs ) → list[Union[str, dict[str, Any]]]

Parameters

path (str) — Path to the directory.
detail (bool, optional) — If True, returns a list of dictionaries containing file information. If False, returns a list of file paths. Defaults to True.
refresh (bool, optional) — If True, bypass the cache and fetch the latest data. Defaults to False.
revision (str, optional) — The git revision to list from.

Returns

list[Union[str, dict[str, Any]]]

List of file paths (if detail=False) or list of file information dictionaries (if detail=True).

List the contents of a directory.

For more details, refer to fsspec documentation.

Note: When possible, use HfApi.list_repo_tree() for better performance.

Update on GitHub

Hub Python Library

파일 시스템 API

HfFileSystem

class huggingface_hub.HfFileSystem

__init__

resolve_path

ls

init