FlowtandBriefAI · Software · ResearchMeet Flowtand

Writer releases three Palmyra-mini open models with GGUF and MLX quantizations

Announcement

Hugging Face

Browse Models topic page →

Writer announced the Palmyra-mini family: three open models sized 1.5B–1.7B, including a non-reasoning base model and two Chain-of-Thought variants. GGUF and MLX quantizations are available. Writer reports benchmarks such as 52.6% on Big Bench Hard, 82.87% on GSM8K and 92.5% on AMC23, and notes that RL tuning on one variant improved single-shot accuracy while reducing sampling diversity.

What Writer released

Writer announced three open models in the Palmyra-mini family, sized between 1.5B and 1.7B parameters. The lineup is a non-thinking base model (palmyra-mini) and two reasoning variants: palmyra-mini-thinking-a, trained with a Chain-of-Thought approach for logic tasks, and palmyra-mini-thinking-b, positioned for mathematical and reasoning problems. Writer published GGUF and MLX (BF16) quantizations for all three, which widens the range of local runtimes that could load them. Writer reports 52.6% on Big Bench Hard for the base model, 82.87% on GSM8K for thinking-a, and 92.5% on AMC23 for thinking-b, and states that all base models use Qwen architecture so inference should be runnable on frameworks such as vLLM, SGLang, TRTLLM and TGI. Writer also discloses that RL fine-tuning of thinking-b improved pass@1 but reduced sampling diversity, lowering majority@64 scores versus the SFT base.

A concrete test for an ML engineer

An ML engineer on a small team maintaining a classification-and-extraction pipeline could try swapping a larger hosted model for palmyra-mini on a held-out sample of their own task data, since Writer positions the base model as an all-rounder for generative tasks. Separately, an engineer working on multi-step math or logic prompts could try palmyra-mini-thinking-b where the source note says it performs best among the release on math-focused benchmarks. Both moves are conditional experiments: they should be scoped to a specific task slice and compared against the current model, not adopted because of headline benchmark scores on datasets that may not reflect production inputs.

Trial steps and failure checks

A team could start by recording a baseline on 100–300 real task examples: exact-match accuracy, latency per request, and token counts for reasoning variants, since CoT output is typically longer. Then run each Palmyra-mini variant with the published sampling settings (temperature 0.6, top_p 0.95) on the same set and compare. Because Writer reports both pass@1(avg-of-1) and pass@1(avg-of-64) results, teams relying on repeated sampling should check majority-vote behavior rather than single-shot accuracy alone — the source notes RL reduced diversity on thinking-b. A human reviewer should spot-check reasoning traces for plausible rather than lucky answers. Treat a drop in baseline accuracy beyond an agreed threshold, or unstable outputs across repeats, as a stop signal.

Trade-offs and when not to use it

The reasoning variants trade latency and output length for accuracy on math and logic tasks, so they are a poor fit for simple high-throughput generation where the base model suffices. Writer's own note on thinking-b shows a diversity trade-off: gains in single-shot accuracy can erode majority-vote reliability, which matters if a pipeline leans on self-consistency. Prerequisites include a runtime that supports the Qwen architecture, which Writer expects but does not guarantee, and a willingness to validate licensing and deployment terms not covered in this excerpt. The published benchmarks are Writer's own and cover standard datasets, so they are not evidence of performance on domain-specific work. Teams needing proven behavior on regulated or safety-critical tasks should not adopt these models without their own evaluation and review.

Context and limitations

  • Based on a short source excerpt; the full article was not analyzed.
  • All benchmark numbers are self-reported by Writer; no independent evaluation is described.
  • Licensing and deployment terms for the models are not covered in the excerpt.
  • Based on bounded source text, not independent testing. Practical suggestions are editorial proposals.