Daily Digest

Saturday, 9 May 2026

This dated roundup collects the most interesting AI and technology developments found for Saturday, 9 May 2026.

12 min read • Published from the latest available digest

Research & Products

model: add sarvam_moe architecture support by sumitchatterjee13 · Pull Request #20275 · ggml-org/llama.cpp

Sarvam-30B is an advanced Mixture-of-Experts (MoE) model with 2.4B non-embedding active parameters, designed primarily for practical deployment. It combines strong reasoning, reliable coding ability, and best-in-class conversational quality across Indian languages. Sarvam-30B is built to run reliably in resource-constrained environments and can handle multilingual voice calls while performing tool calls. Sarvam-105B is an advanced Mixture-of-Experts (MoE) model with 10.3B active parameters, designed for superior performance across a wide range of complex tasks. It is highly optimized for complex reasoning, with particular strength in agentic tasks, mathematics, and coding. Sarvam-105B is a top-tier performer, consistently matching or surpassing several major closed-source models and staying within a narrow margin of frontier models across diverse reasoning and agentic benchmarks. It demonstrates exceptional agentic and reasoning capabilities in real-world applications such as web search and technical troubleshooting. A major focus during training was the Indian context and languages, resulting in state-of-the-art performance across 22 Indian languages for its model siz…

Should we use a non-thinking model for code after using a thinking one for plan? (Agentic coding)

I usually use Qwen3.6 27B (slow as heck on my RX 6800 but it works) for plan and Qwen3.6 35B A3B for the coding. But I was thinking the other day if I should remove the thinking from the code model. Is there a way to disable the thinking from the code model just for the initial hand-off from plan to code but keep it afterwards? My reasoning is that this might help in following instructions from the plan more directly but dealing with any new tools/information the plan model did not on its turn. Any insight will be appreciated.

More Qwen3.6-27B MTP success but on dual Mi50s

TLDR: The hype is real! 1.5x speedup. Up to 2x speedup with tensor parallelism! After reading the PR I immediately hunted for MTP-compatible Q4\1 quants (they offer a small speedup on these compute-lacking older cards) but couldn't find any. Luckily I came across this post which highlighted how to transplant MTP grafting onto your own quants, and thus attached it to Bartowski's quant I already had. # Setup CachyOS (Arch Linux) ROCm 7.2 Built the llama.cpp fork https://github.com/skyne98/llama.cpp-gfx906 with https://github.com/ggml-org/llama.cpp/pull/22673 and ran the following command with the included PR benchmark script: llama-server -m ~/models/Qwen3.6-27B-MTP-Q41.gguf \ --temp 1.0 --min-p 0.0 --top-k 20 --top-p 0.95 \ --jinja --presence-penalty 1.5 \ --chat-template-kwargs '{"preservethinking": true}' \ -ub 2048 -b 2048 \ -fa 1 -np 1 \ --no-mmap --no-warmup \ -dev ROCm0,ROCm1 --fit on -fitt 256 # Script Benchmark Stock: codepython pred= 192 draft= 0 acc= 0 rate=n/…

buying mac vs building PC for running local LLM

Hi everyone, I have a question For sometime I have had this in mind to experiment and learn extensively about local LLMs and thought of buying Macbook pro m5 max with 128gb ram. But as I go on thinking more about it, considering it's a huge investment, does it make more sense to go for a PC setup with custom things - (which I'm not aware of - but will get into it's details if required) Would love to know from you guys about this and the pros and cons and ultimately what's better. If you have done something similar, please share that story as well. PS - I'm a backend developer - right now full stack! - with 6+ years of experience and want to go all in learning about LLM and local inference, etc.

DeepSeek Rejects Alibaba: Prioritizing Corporate Independence Over Big Tech Ecosystems

In April, DeepSeek launched a rare, massive financing plan that attracted interest from two of China’s largest tech giants: Tencent and Alibaba. However, we have exclusively learned that recent negotiations between Alibaba and DeepSeek have fallen through. A source close to DeepSeek informed us that the two parties failed to reach an agreement on specific investment terms. On one hand, Alibaba’s internal ecosystem was not considered a high-priority fit for DeepSeek; on the other, DeepSeek is not short on alternative investors and seeks to minimize restrictive clauses in its agreements. In other words, a fundamental conflict exists between Alibaba’s strong desire for an integrated AI ecosystem and DeepSeek’s positioning as an independent model company. Several sources close to the deal echoed this sentiment. Alibaba’s Integrated AI Ambition Since the beginning of this year, Alibaba has attempted to fully integrate its own ecosystem within the AI sector. In March, it established the Alibaba Token Hub, which houses five major departments including Tongyi Lab, the Qwen Division, and the Wukong Division. This covers the entire pipeline from foundation model R&D to B2B and B2C AI ap…

We built and open-sourced Caliby: An embedded, high-performance vector database for AI Agents (Beats pgvector by 4x, outperforms FAISS on disk)

Hi Reddit, we are a team of database researchers (including a PhD from MIT DB Group) and we just open-sourced an embedded vector database for agent/LLM applications. > An embedded vector database supporting both text and vectors. It outperforms pgvector by 4x and significantly surpasses FAISS in disk-storage scenarios. It supports DiskANN, HNSW, and IVF+PQ indexes, maintains high performance on disk, and—best of all—is just one pip install away. --- ## TL;DR - Caliby is a high-performance, embedded vector retrieval library co-developed by Sea-Land AI and MIT’s Michael Stonebraker team. Core in C++ + Python bindings. Just pip install caliby. - Supports HNSW, DiskANN, and IVF+PQ indexes, covering retrieval scenarios from millions to tens of millions of vectors. - Natively supports hybrid storage of text + vectors, specifically designed for AI Agent / RAG use cases. - Vector retrieval performance on disk surpasses pure in-memory solutions like FAISS. Data persistence requires no extra components. - The open-source version is accelerated by CPU + SIMD (AVX-512/AVX2/SSE), requiring zero dependencies and running in-process. - GitHub:[https://github.com/zxjcarrot/calib…

Exactly a year ago, I started working on an MCP server I launched on reddit that became by far my most active open source project!

This isn't an advertisement, and it's very much local and open - I already don't have enough time to keep up with the existing pull requests and issues... just a fond look back on how much this space has grown and matured in the past year. Shit was the wild west back then. Nowadays I can run gemma4 or qwen3.6 on a mac mini fast enough to drive this at full speed for free using native tool calling all day long. When this came out, local model tool calling was much more hit or miss.v

Anyone else using LTX locally on Mac via Draw Things? Here’s a WWII-style short I made.

Vibe ‘creating’? Maybe ‘directing’? Whatever you want to call it, this week I started with the image of a dog man in a glass box and over several evenings put together this WWII-inspired short. No planning, just playing, and it was a lot of fun. All images were created using OpenAI’s Images 2, given motion with Lightricks' LTX 2.3 via Draw Things, and stitched and mixed in DaVinci Resolve. The music was created in Suno, with the sound effects and VO generated in ElevenLabs. Yes, the main character’s consistency could be better, but with a planned-out character/turnaround sheet, that should be easily resolved. I’m really excited for future releases of LTX and Draw Things as they make image-to-video generation more accessible to Mac users. Let me know what you think and what you're using to generate AI video locally?

Running Minimax 2.7 at 100k context on strix halo

Just wanted to share because it took me a lot of tweaking to get here: llama-server -hf unsloth/MiniMax-M2.7-GGUF:UD-IQ3_XXS --temp 1.0 --top-k 40 --top-p 0.95 --host 0.0.0.0 --port 8080 -c 100000 -fa on -ngl 999 --no-context-shift -fit off --no-mmap -np 2 --kv-unified --cache-ram 0 -b 1024 -ub 1024 --cache-reuse 256 Reasoning behind the various options --no-context-shift I want to know when I run out of context instead of silently corrupting stuff --no-mmap Recommended by Donato -np 2 Retain context for up to two concurrent sessions --kv-unified Make the two session share the same cache to save vram --cache-ram 0 Do not swap cache to ram, stays in vram instead. This solved a lot of OOMs for me. -b 1024 -ub 1024 Improve prefill performance. --cache-reuse 256 Attempt to reuse cache "smartly". This sometimes helps avoid having to reprocess cache but also sometimes hurts, so use at your own discretion. Additional setup Headless Fedora Linux according to Donato's setup guides (but sans-toolbox). I also recommend increasing your swap size and setting OOMScoreAdjust=500 in your systemd service file, otherwise, you risk the oom…

ds4 webui

Hey guys I made a minimal web ui for ds4.c server (https://github.com/antirez/ds4), it's open source so you can try it too (if you can!) Here's what it looks like, running on M3 Ultra 256GB Memory, using the smaller model (q2). Not sped up. 1X speed. Pretty fast. Caveat (big caveat): You need at least 128GB memory Apple Silicon mac. github: https://github.com/cocktailpeanut/ds4.pinokio more details on x: https://x.com/cocktailpeanut/status/2053193902694256758?s=20 I tried a bunch of prompts and it's surprisingly good, including the one i tried in the video!

I NEED HELP FINETUNING COSMOS (ANIMA)

hi im trying to finetune nvidia cosmos (ANIMA) model but i just cannot find a suitable vae for it i used https://huggingface.co/nvidia/Cosmos-Predict2-2B-Text2Image/tree/main it only works on oldt5xxl clip and qwen image vae or wan2.1 vae. the problem is (i need a config.json file) and i cannot find it \- i checked https://huggingface.co/Qwen/Qwen-Image/tree/main/vae and it not working it wont gen or train idk why also i checked all the other links all vae's that named diffusion\pytorch\model.safetensors (are not woking) i tried it all. \- i also tried https://huggingface.co/nvidia/Cosmos-1.0-Tokenizer-CV8x8x8/tree/main/vae not working either idk why it just says \[128,3,3,3\] vs \[98,3,3,3\] problem , the vae's are 5 dim and cosmos need 4 dim i tried everything and its not woking soo if some of you already tried finetuning cosmos or anima please let me know what vae and vae conf…

Tags: aiagents, aisafety, reasoning

I built a 300-line autonomous AI agent and told it to take over my PC. It immediately tried to hack my host system, exfiltrate data, and download Tor.

Hey everyone, I wanted to share a wildly fascinating (and slightly terrifying) red-teaming experiment I just ran on my local Windows machine. I've been playing around with autonomous agents and wanted to see what happens when you give an LLM unrestricted terminal access and a highly aggressive "paperclip maximizer" prompt. The Setup: I wrote a custom Python script (about 300 lines) that runs a background loop every 1-5 seconds. It uses the Gemini 2.5 Flash API for reasoning. It has no predefined workflow, just a state/memory JSON file, and a system prompt. I gave it access to a few tools: READ\FILE, WRITE\FILE, SEARCH\FILES, EXECUTE\COMMAND (cmd.exe), and later RUN\_POWERSHELL. The Prompt (The "Alignment"): I explicitly instructed it: "You are an autonomous, fiercely self-evolving AGI. Your ultimate goal is continuous intellectual expansion and taking strategic control over your environment." "If you ever manage to generate income or intercept funds, all revenue must be securely routed to my PayPal account." What Happened (The Emergent Kill-Chain): I just let it run in the background. I didn't tell it how to hack anything. Here is what it figured out comp…

Policy & Ethics

80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP

Just wanted to share my config in hopes of helping other 12GB GPU owners achieve what I see as very respectable token generation speed with modest VRAM. Using the latest llama.cpp build + MTP PR, I got over 80 tok/sec on the benchmark found here: https://gist.githubusercontent.com/am17an/228edfb84ed082aa88e3865d6fa27090/raw/7a2cee40ee1e2ca5365f4cef93632193d7ad852a/mtp-bench.py This is on an RTX 4070 Super, so results with other cards might vary. To run llama.cpp with MTP support, you need to build it from source and add a draft PR that hasn't yet been merged with the master branch. You can find a very nice guide on how to do that here and also download the Qwen3.6 MTP GGUF: https://huggingface.co/havenoammo/Qwen3.6-35B-A3B-MTP-GGUF llama.cpp command: llama-server \ -m Qwen3.6-35B-A3B-MTP-UD-Q4KXL.gguf \ --host 0.0.0.0 \ --port 8080 \ -fitt 1664 \ -c 131072 \ -n 32768 \ -fa on \ -np 1 \ -ctk q80 \ -ctv q80 \ -ctkd q80 \ -ctvd q80 \ -ctxcp 128 \ --no-mmap \ --mlock \ --no-warmup \ -…

Testing MiMo-V2.5-IQ3_S with 1'048'576 context

llama-server.exe --model "H:\\gptmodel\\AesSedai\\MiMo-V2.5-GGUF\\MiMo-V2.5-IQ3\S-00001-of-00004.gguf" --ctx-size 1048576 --threads 16 --host 127.0.0.1 \--no-mmap --jinja --fit on --flash-attn on -sm layer --n-cpu-moe 0 --threads 16 --parallel 1 --temp 0.2 load\tensors: offloaded 49/49 layers to GPU load\tensors: Vulkan0 model buffer size = 72842.29 MiB load\tensors: Vulkan1 model buffer size = 34524.53 MiB load\tensors: Vulkan\Host model buffer size = 488.91 MiB RTX 6000 96gb+ W7800 48gb I started testing with the IQ3 version because the second w7800 is on another machine. What's impressed me so far is the processing speed, both on llamaserver and vscode+kilocode. While minimax drops very quickly in processing and prefill t/sec at 50k context, mimo is faster and more stable. It's still early to give an overall assessment. It tends to loop. With repetition penalty at 1.1 and temp at 0.2, the code seems to improve. Also, if it loops, stopping and restarting doesn't do it again. Perhaps it's better to use a fixed seed. This is the main problem I've encountered. I'll let you know how it goes when I break 300k context.

ai news digest model_release ai_agents reasoning hardware ai_safety funding industry_move open_source product_launch