diff --git a/contributing/BACKENDS.md b/contributing/BACKENDS.md index 0c7ac6a15..90b56a55a 100644 --- a/contributing/BACKENDS.md +++ b/contributing/BACKENDS.md @@ -31,14 +31,23 @@ git clone https://github.com/dstackai/gpuhunt.git - **Online providers** offer dynamic machine configurations that are available at the very moment when you fetch configurations (e.g., GPU marketplaces). `gpuhunt` collects online providers' instance offers each time a `dstack` user provisions a new instance. - Examples: `tensordock`, `vastai`, etc. + Examples: `vastai`, `hotaisle`, etc. ### 1.3. Create the provider class Create the provider class file under `src/gpuhunt/providers`. -Make sure your class extends the [`AbstractProvider`](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/__init__.py) -base class. See its docstrings for descriptions of the methods that your class should implement. +Make sure your class extends either `OnlineProvider` or `OfflineProvider` from +[base.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/base.py), +matching the choice you made above. + +Both kinds implement `get`, returning `CatalogItem`s with `provider` set to your provider's `NAME`. +Additionally: + +- Online providers implement the `from_env` classmethod, which reads credentials from the + environment with `get_creds_env` and raises `MissingCredsError` if one is missing. Providers that + raise it are skipped by `default_catalog()` rather than failing the whole catalog. +- Offline providers may override `filter` to omit some offers from the published catalog. Refer to examples: - Offline providers: @@ -47,8 +56,8 @@ Refer to examples: [azure.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/azure.py), [lambdalabs.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/lambdalabs.py). - Online providers: - [vultr.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vultr.py) - [tensordock.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/tensordock.py), + [vultr.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vultr.py), + [hotaisle.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/hotaisle.py), [vastai.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vastai.py). ### 1.4. Register the provider with the catalog @@ -57,7 +66,8 @@ Add your provider in the following places: - Either `OFFLINE_PROVIDERS` or `ONLINE_PROVIDERS` in `src/gpuhunt/_internal/catalog.py`. - The `python -m gpuhunt` command in `src/gpuhunt/__main__.py`. - (offline providers) The CI workflow in `.github/workflows/catalogs.yml`. -- (online providers) The default catalog in `src/gpuhunt/_internal/default.py`. +- (online providers) `ONLINE_PROVIDER_MODULES` in `src/gpuhunt/_internal/default.py`, which is what + `default_catalog()` loads. ### 1.5. Add data quality tests diff --git a/contributing/GPUHUNT.md b/contributing/GPUHUNT.md index 33e3150fa..360e97366 100644 --- a/contributing/GPUHUNT.md +++ b/contributing/GPUHUNT.md @@ -4,10 +4,12 @@ An offer is a possible configuration. It consists of: - Provider (or backend in dstack) +- CPU architecture - CPU count - RAM size - Disk size - GPU count +- GPU vendor (if any) - GPU model name (if any) - GPU VRAM size (if any) - Is interruptible (or spot) @@ -15,11 +17,14 @@ An offer is a possible configuration. It consists of: - Instance name or ID (provider-specific) - Price per hour +Offers are represented by `CatalogItem`. Providers construct them directly, setting `provider` to +their own `NAME`, and `gpu_vendor` whenever `gpu_count` is non-zero. + ## Catalog Some providers don't have a suitable API for querying all offers in real-time. That's why gpuhunt has two types of providers: -- Online — offers can be queried in real-time +- Online — offers can be queried quickly in real-time - Offline — offers must be loaded from a precomputed catalog file The `Catalog` class hides those details from the user, reading offers from the file for offline providers or querying online providers. @@ -28,7 +33,16 @@ The `Catalog` class pulls the latest catalog from the S3 bucket and caches it fo ## Provider implementation -Providers must implement a single method `get`. It has the same name for both online and offline providers but works differently. +Providers subclass either `OnlineProvider` or `OfflineProvider` from `src/gpuhunt/providers/base.py`. + +Both implement `get`. It has the same name for both online and offline providers but works differently. + +In addition: + +- Online providers implement the `from_env` classmethod, since `default_catalog()` constructs them + in the user's process. +- Offline providers may override `filter` to omit some offers from the published catalog. + Credentials are passed in by the caller, so a missing one is an error rather than a skip. ### Offers sorting @@ -100,7 +114,7 @@ These mechanisms are used to preserve backward compatibility: - **`gpuhunt` version**: The interfaces in the `gpuhunt` package preserve backward compatibility within a minor version (`X` in `0.X.Y`). -- **Offer flags**: If an offer breaks older `dstack` versions, it is marked with a flag in `RawCatalogItem.flags` +- **Offer flags**: If an offer breaks older `dstack` versions, it is marked with a flag in `CatalogItem.flags` and the flag is added to the list of supported flags in `dstack`. Older `dstack` versions that don't support this flag will not see the respective offers. - **Offline catalog versions**: If a breaking change in the structure or content of an offline catalog is unavoidable,