LoRA Deployment & Checkpoints

The training artifact on MinT is a LoRA incremental weight — right after training you can get a sampling-ready client via save_weights_and_get_sampling_client. Two deployment forms are supported: "online mounting" and "merged deployment."

Why LoRA is at the core

LoRA is one of MinT's core values: a built-in LoRA Manager gives you a lightweight incremental weight right after training, ready for immediate sampling and validation. Every training job centers on LoRA fine-tuning, and a full fine-tune averages about 1 to 1.5 hours (30B model, A3B structure).

Save the LoRA and get a sampling client

After the training loop ends, one call saves the weights and returns a sampling client:

sampling_client = training_client.save_weights_and_get_sampling_client(name='my-run-v1')

prompt_ids = tokenizer.encode('3 * 7 =')
samples = sampling_client.sample(
    prompt=types.ModelInput.from_ints(prompt_ids),
    sampling_params=types.SamplingParams(max_tokens=16, temperature=0.7),
    num_samples=4,
)
for s in samples.sequences:
    print(tokenizer.decode(s.tokens))

The resulting sampling_client supports the .sample(prompt, sampling_params, num_samples) interface, usable for canary comparisons, automatic evaluation, A/B launches, and similar scenarios.

Two deployment forms

LoRA weights can be mounted onto the inference service with one click, giving you an OpenAI-compatible endpoint; deployment supports two modes:

Save a checkpoint

After training finishes, check this run's consumption on the usage page, and save the current model checkpoint for deployment or a later RL stage. Each checkpoint carries metadata (Recipe, data version, hyperparameters, eval scores) for easy traceability.

LoRA hosting under private deployment

Private delivery explicitly includes "private model hosting: the LoRA and merged weights produced by training can be packaged directly as offline deployment bundles." The Enterprise console and SDK come from exactly the same source as the public-cloud version, with no version fork.