OpenAI-Compatible API

Training artifacts on MinT can be mounted onto the inference service with one click, giving you an OpenAI-compatible endpoint. Business systems integrate simply by switching the base URL.

Two ways to call

The same LoRA weight can be called in two ways:

• Direct SDK sampling. Through sampling_client.sample(prompt, sampling_params, num_samples), for scenarios like canary comparison, automatic evaluation, and A/B launch.
• OpenAI-compatible HTTP. As described, "the trained LoRA weights can be mounted onto the inference service with one click, giving you an OpenAI-compatible endpoint." The business side only needs to switch the base URL and can call it with the OpenAI SDK.

Direct SDK sampling

After training, get a sampling client from training_client:

sampling_client = training_client.save_weights_and_get_sampling_client(name='my-run-v1')

prompt_ids = tokenizer.encode('3 * 7 =')
samples = sampling_client.sample(
    prompt=types.ModelInput.from_ints(prompt_ids),
    sampling_params=types.SamplingParams(max_tokens=16, temperature=0.7),
    num_samples=4,
)
for s in samples.sequences:
    print(tokenizer.decode(s.tokens))

OpenAI-compatible HTTP

A trained model yields an OpenAI-compatible endpoint. The specific base URL, model identifier, and authentication method are provided by sampling_client after training, or viewable in the console. Business systems can integrate using the standard OpenAI SDK usage:

from openai import OpenAI

client = OpenAI(
    base_url="<your-mint-oai-compatible-endpoint>",
    api_key="sk-your-api-key-here",
)
resp = client.chat.completions.create(
    model="my-run-v1",     # the name used during training
    messages=[{"role": "user", "content": "3 * 7 ="}],
    max_tokens=16,
    temperature=0.7,
)
print(resp.choices[0].message.content)

Deployment mode and inference adaptation

There are two deployment modes, and your choice affects the inference form:

• LoRA online mounting. Shares memory with the base model, fastest startup time, suited to parallel sampling across experiments.
• LoRA merged deployment. Merges the LoRA into the base model to produce full weights, which can be exported for standalone offline deployment, suited to private-delivery scenarios.