MPT (MosaicML Pretrained Transformer)
MosaicML's 2023 commercially usable open LLMs, now a legacy family
Overview
MPT (MosaicML Pretrained Transformer) is a family of open language models released by MosaicML in 2023: MPT-7B in May, with Instruct, Chat and 65k-context StoryWriter variants, and MPT-30B in June. Base models are Apache 2.0, the Chat variants non-commercial. Databricks bought MosaicML in 2023 and has since retired MPT from its model serving.
Pricing
Pricing shown for reference only. These figures reflect RECATOOLS research as of 14 Oct 2026 and may be out of date or incomplete. This is not financial or purchasing advice — always confirm the current price on the provider’s official website before making any decision.
Use cases
What you can produce with MPT (MosaicML Pretrained Transformer)
- MPT-7B and MPT-30B base models, released under Apache 2.0 for commercial use
- Instruct, Chat and 65k-context StoryWriter fine-tunes, with the Chat models limited to non-commercial use
- The llm-foundry training and fine-tuning code (Apache 2.0) used to build the models
- A documented 2023 training run: one trillion tokens in 9.5 days for about $200,000
- An early worked example of ALiBi extrapolating beyond the trained context length
- A dated reference point for how open LLMs were built before the Llama 2 era
MPT was an important 2023 release: an Apache 2.0 base model trained from scratch for about $200,000, with a 65k-context StoryWriter variant when most open models stopped at 2k to 4k tokens. Mind the licence split, though: both Chat models are non-commercial.
Today it is history. Databricks retired MPT from its serving APIs in 2024 and 2025, the official Hugging Face repositories no longer open publicly, and llm-foundry is barely maintained. Read it to understand the period; deploy something current.
What we found
MPT, short for MosaicML Pretrained Transformer, is a family of open language models MosaicML released in 2023. MPT-7B arrived on 5 May 2023: a 7-billion-parameter model trained from scratch on one trillion tokens of text and code in 9.5 days, at a cost MosaicML put at about $200,000. The base model was Apache 2.0 and usable commercially at a time when Meta's original LLaMA was not. It shipped with three fine-tunes: Instruct, Chat and StoryWriter-65k+, a long-context variant trained on 65,000-token inputs that, thanks to ALiBi attention biases, MosaicML showed generating up to 84,000 tokens.
MPT-30B followed on 22 June 2023, trained with an 8,000-token context, and MosaicML reported more than three million downloads of the 7B models by then. Licences were not uniform. The base and Instruct models allowed commercial use, but MPT-7B-Chat and MPT-30B-Chat carried the non-commercial CC BY-NC-SA 4.0 licence, so the chat variants could not simply be dropped into a product.
Four days later Databricks agreed to buy MosaicML for about $1.3 billion including retention packages, and MPT has since been wound down. Databricks retired the MPT models from its pay-per-token Foundation Model APIs on 30 August 2024 and from provisioned throughput on 19 December 2025, pointing users to Llama models instead. The MPT repositories under MosaicML's Hugging Face organisation now return an authorisation error, and its public listing shows only Mosaic BERT models. llm-foundry, the Apache 2.0 training code released alongside MPT and now described as training code for Databricks foundation models, last tagged a release in July 2025 and removed its CI workflows in March 2026.
This is a dated page, not a recommendation. MPT matters as the 2023 family that showed a commercially usable open model could be trained for a six-figure sum, and llm-foundry remains readable for anyone studying how. For a model to deploy today, look at current open-weight families.
Compiled by RECATOOLS from public sources: vendor documentation and pricing, code repositories and published reporting.
Notable facts
- MosaicML was acquired by Databricks for $1.3 billion in June 2023, just weeks after releasing MPT — one of the fastest acquisitions of an AI lab after a major model release.
- MPT models can be extended to handle 84,000 token contexts during inference even when trained on 2,048 tokens — a significant practical advantage for document analysis.
- The ALiBi attention mechanism used in MPT was invented by researchers at UNC and Facebook AI Research and represents a fundamentally different way of encoding positional information.
Frequently asked questions
About this listing
This entry was compiled from publicly available data including MPT (MosaicML Pretrained Transformer)'s official website, press releases, documentation, and reputable third-party publications. RECATOOLS is not affiliated with MPT (MosaicML Pretrained Transformer) unless explicitly stated.
Third-party AI tools update their pricing, features, availability, and policies frequently. Information here may be outdated by the time you read this — we make reasonable efforts to keep listings current, but cannot guarantee absolute accuracy.
For the latest details, please refer to MPT (MosaicML Pretrained Transformer) directly →
Spotted something out of date? Suggest an update →
Related reading
Alternatives to MPT (MosaicML Pretrained Transformer)
More in LLMs & Chat
